bash: how to pass command line arguments containing special characters

Question

I've written myself a linux program program that needs a regular expression as input.

I want to call the program in the bash shell and pass that regular expression as a command line argument to the program (there are also other command line arguments). A typical regular expression looks like

[abc]\_[x|y]

Unfortunately the characters [, ], and | are special characters in bash. Thus, calling

program [abc]\_[x|y] anotheragument

doesn't work. Is there a way to pass the expression by using some sort of escape characters or quotation marks etc.?

(Calling program "[abc]\_[x|y] anotheragument" isn't working either, because it interprets the two arguments as one.)

Answer 1

You can either:

1. Escape each single special symbol with a backslash (as in \[abc\]_\[x\|y\]) or
2. Double-quote the entire argument (as in "[abc]_[x|y]").

EDIT: As some have pointed out, double-quoting does not prevent variable expansion nor command substitution. Therefore if your regex contains something that can be interpreted by bash as one of those, use single quotes instead.

Answer 2

Use single quotes. Single quotes ensure that none of the characters are interpreted.

$ printf %s 'spaces  are  not  interpreted away
neither are new lines
nor variable names $TESTING
nor square brackets [TESTING]
nor pipe characters or redirection symbols | > <
nor the semicolon ;
nor backslashes \a \b \c \\
the only thing that does not work is the single quote itself
'

There are two solutions if you need to embed a single quote:

$ printf '%s\n' '[ Don'"'"'t worry, be happy! ]'
[ Don't worry, be happy! ]
$ printf '%s\n' '[ Don'\''t worry, be happy! ]'
[ Don't worry, be happy! ]

Answer 3

Per man bash

There are three quoting mechanisms: the escape character, single quotes, and double quotes.

A non-quoted backslash (\) is the escape character. It preserves the literal value of the next character that follows, with the exception of <newline>. If a \<newline> pair appears, and the backslash is not itself quoted, the \<newline> is treated as a line continuation (that is, it is removed from the input stream and effectively ignored).

Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.

Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, `, \, and, when history expansion is enabled, !. The characters $ and ` retain their special meaning within double quotes. The backslash retains its special meaning only when followed by one of the following characters: $, `, ", \, or <newline>. A double quote may be quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.

The special parameters * and @ have special meaning when in double quotes (see PARAMETERS below).

Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. Backslash escape sequences, if present, are decoded as follows:

       \a     alert (bell)
       \b     backspace
       \e
       \E     an escape character
       \f     form feed
       \n     new line
       \r     carriage return
       \t     horizontal tab
       \v     vertical tab
       \\     backslash
       \'     single quote
       \"     double quote
       \nnn   the eight-bit character whose value is the octal value nnn
              (one to three digits)
       \xHH   the eight-bit character whose value is the hexadecimal value HH
              (one or two hex digits)
       \uHHHH the Unicode (ISO/IEC 10646) character whose value is
              the hexadecimal value HHHH (one to four hex digits)
       \UHHHHHHHH
              the Unicode (ISO/IEC 10646) character whose value is
              the hexadecimal value HHHHHHHH (one to eight hex digits)
       \cx    a control-x character

The expanded result is single-quoted, as if the dollar sign had not been present.

A double-quoted string preceded by a dollar sign ($"string") will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.

Answer 4

You can use a backslash ( \ ) in front of special characters to escape them like so:

john@awesome:~ # echo \&
&

Answer 5

Although it might not be useful as a regex, some character sequences may be interpreted as Bash variable names. To prevent this from occurring and avoid having them expanded, use single quotes instead of double quotes:

program '[abc]_[x|y]' anotherargument

Quote each argument separately (if they need quoting) so they are interpreted as independent arguments. You can also use arrays in some cases:

param_array=('[abc]_[x|y]' anotherargument)    # create an array
param_array+=(yetanother)     # append another element to the array
program "${param_array[@]}"   # use the array elements as arguments to program

Answer 6

program "[abc]_[x|y]"
program "[abc]_[x|y]" anotherargument

Answer 7

Escaping them should work fine:

  programm \[abc\]_\[x\|y\]

Answer 8

Where does the pattern come from? Is it fixed or from a user? Is it the user who is invoking the script on the local system, or someone remote?

You use quotes to wrap data to keep the shell from interpreting it. There are two options:

1. Double-quotes, which still permit some interpretation ($expand and `backticks`)
2. Single-quotes, which pass everything through literally

Because $ is a valid character in regexps (end-of-line/buffer) you probably want to use single-quotes to hold the regexp, unless you're storing in in a variable. If you're taking arbitrary data from someone untrusted, you'll need to replace ' with '"'"' and then wrap in single-quotes.

Note that [abc]_[x|y] looks like you want to match x or y, while it's actually matching one of the three characters xy|. The square brackets match the characters within and only - for ranges and a ^ at the start for negation. So, [abc]_(x|y) might be what you meant, and the parentheses are the characters which are special to shell. Square-brackets are not special to shell, it just looks like they are. Double-square brackets [[ ... ]] are special.