How do I properly quote this bash pipeline for watch?

Asked
Active3 hr before
Viewed126 times

9 Answers

properly
90%

It is all about how to escaping single quotes inside single quotes, there is a good explanation "BASH, escaping single-quotes inside of single-quoted strings" , Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers ,The problem is about the single quotes for awk, you could fix it by escaping single quotes. , Kevin: I saw that hint, actually! And so I tried replacing the single quotes around {print $2} with double quotes. No luck. I believe the single quote in awk is a token that's non-equivalent to the double quote – Dmitry Minkovsky May 6 '13 at 21:02

The problem is about the single quotes for awk, you could fix it by escaping single quotes.

watch 'echo "scale=2;$(cat io | grep wchar | awk '
"'"
'{print $2}'
"'"
')/(1024^3)" | bc'
load more v
88%

Stack Exchange network consists of 178 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers. , 1 @RegisteredUser I recommend to accept this (steeldriver's) answer instead of mine because it gives right solution for awk and also sort syntax by stat! – Pandya Jul 20 '14 at 11:02 , Student asked me if it is necessary to simplify fractions at the end of answering a question. I'm not sure how to respond ,Ask Ubuntu is a question and answer site for Ubuntu users and developers. It only takes a minute to sign up.

One option would be to escape the $ in your awk expression

watch "ls -hal ./file |awk '{print \$5}'"

Alternatively, you could avoid the issue altogether by using stat instead of parsing the output of ls

watch stat - c '%s'. / file
load more v
72%

I've built up this pipeline:,Now I'm trying to watch it. My knowledge of Bash is really ad-hoc, and so I'm not having success. Tried things like:,Characters which have a special meaning in Makefile and that need to be escaped are:,Okay, it turned out that Makefiles need little escaping for itself, but the commands which are executed by the shell interpreter need to be escaped.

I've built up this pipeline:

echo "scale=2;$(cat io | grep wchar | awk '{print $2}')/(1024^3)" | bc

Now I'm trying to watch it. My knowledge of Bash is really ad-hoc, and so I'm not having success. Tried things like:

watch echo "scale=2;$(cat io | grep wchar | awk '{print $2}')/(1024^3)" | bc # I understand why this fails

watch 'echo "scale=2;$(cat io | grep wchar | awk ' {
   print $2
}
')/(1024^3)" | bc'
# Not enough bash understanding to understand why this fails

Sample output from cat io is

rchar: 36713294562
wchar: 36788363400
syscr: 27050
syscw: 2314540
read_bytes: 36709928960
write_bytes: 0
cancelled_write_bytes: 0
load more v
65%

I guess you have to escape the $ sign:,otherwise it would be interpreted by the shell which would result in an empty string ("") - i.e. awk would print the whole line., Size of set of integers with all sums of two distinct elements giving squares , Tour Start here for a quick overview of the site

I guess you have to escape the $ sign:

watch "ps -ef | awk -F' ' '{print \$2}'"
load more v
75%

The Bourne shell does allow quotes to extend beyond lines: ,This is part of my Unix tutorials series. ,(There are 7 spaces between 'a' and 'b'.) This is ugly but works. It is easier to use pairs of quotation marks to indicate the start and end of the characters to be quoted: ,Note that this technique works for any shell using either one of the quotes.

There are three different "quotation" marks on the keyboard.. Two of them use marks used for quotations in English usage, and are sometimes called the single quote and double quote.. The third quotation mark is the back quote (more properly called backtick or grave) character: "`". It looks like the single quote and some times people get them confused in shell scripts. The first two are used for quoting phrases in Unix. The back quote is not used for quoting characters. That character is used for command substitution, where the characters between them are executed by the shell and the results is inserted on that line. Example:

% echo the date is `date`
load more v
40%

Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.,Thanks for contributing an answer to Unix & Linux Stack Exchange!,Connect and share knowledge within a single location that is structured and easy to search.,Use a combination of single quotes (') and double quotes ("). For example:

watch 'command | othertool | yet-another-tool'
load more v
22%

I want to include special characters like quotes, backslashes, dollar signs and newlines in my pipe variables. How do I properly quote and escape them so that they survive processing by the yaml parser and variable expansion?,The dollar sign is escaped to prevent the shell from treating it as a variable expansion.,How do I properly quote Pipelines pipe variables and escape special characters like dollar signs?,Even though we are enclosing our value with single quotes here (which stops the yaml parser from processing backslash characters) we still need to escape the backslash to avoid special processing by the shell in the second round.

The second round of processing actually happens within your pipeline and is performed by the shell that is running your build script (usually bash). Pipe variables are passed to the pipe using command-line arguments to a "docker run" command. For example, if you define a pipe variable like this in bitbucket-pipelines.yaml:

variables: MY_PIPE_VARIABLE: 'my-pipe-variable-value'

Then it will get passed into the pipe like this:

docker run...--env = MY_PIPE_VARIABLE = "my-pipe-variable-value"...

Note the double quotes around the value. This tells your shell to do a limited amount of variable expansion on the value that you've defined. That's what allows you to write something like this in your pipeline:

variables: MY_PIPE_VARIABLE: '${MY_REPOSITORY_VARIABLE}'

which gets passed into the pipe like this:

docker run...--env = MY_PIPE_VARIABLE = "${MY_REPOSITORY_VARIABLE}"...

hello

variables: VAR1: 'hello'

price: $100

variables: VAR2: 'price: \$100'

string with internal "double quotes"

variables: VAR3: 'string with internal "double quotes"'

string with internal 'single quotes'

variables: VAR4: 'string with internal '
'single quotes'
''

string with internal `back ticks`

variables: VAR5: 'string with internal \`back ticks\`'

string with a backslash \ character

variables: VAR6: 'string with a backslash \\ character'

string with a      tab character and a

newline

variables: VAR7: "string with a \ttab character and a \nnewline"

string with $ multiple ' difficult " characters \ that need

escaping

variables: VAR8: "string with \\$ multiple ' difficult \" characters \\\\ that need\nescaping"

 

variables: VAR9: 'string with escaped \\t sequences \\n that \\" remain \\'
' escaped in the \\\$ final variable inside the pipe'
load more v
60%

If the pipeline is not executed asynchronously (see Lists), the shell waits for all commands in the pipeline to complete. ,A pipeline is a sequence of one or more commands separated by one of the control operators ‘|’ or ‘|&’. ,Each command in a pipeline is executed in its own subshell, which is a separate process (see Command Execution Environment). If the lastpipe option is enabled using the shopt builtin (see The Shopt Builtin), the last element of a pipeline may be run by the shell process. ,When the shell is in POSIX mode (see Bash POSIX Mode), time may be followed by a newline. In this case, the shell displays the total user and system time consumed by the shell and its children. The TIMEFORMAT variable may be used to specify the format of the time information.

[time[-p]][!] command1[ | or | & command2]
48%

bash$ ps ax | grep clock 765 tty1 S 0:00 xclock 901 pts/1 S 0:00 grep clock ,Simple-minded file formatter, used as a filter in a pipe to "wrap" long lines of text output.,The expand filter converts tabs to spaces. It is often used in a pipe.,This filter removes duplicate lines from a sorted file. It is often seen in a pipe coupled with sort.

cat list - 1 list - 2 list - 3 | sort | uniq > final.list
# Concatenates the list files,
# sorts them,
# removes duplicate lines,
# and
finally writes the result to an output file.
load more v

Other "properly-undefined" queries related to "How do I properly quote this bash pipeline for watch?"