Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

How do I get Cygwin's grep to work properly in a regular cmd.exe?

> grep -o 'ProductVersion\".*\".*\"' foo.txt | grep -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'
foo.txt:ProductVersion" Value="59.59.140.59"
grep: |: No such file or directory
grep: grep: No such file or directory
grep: [0-9]+\.[0-9]+\.[0-9]+\.[0-9]+: No such file or directory

and

> grep -o 'ProductVersion\".*\".*\"' foo.txt >> blah.txt
foo.txt:ProductVersion" Value="59.59.140.59"
grep: >>: No such file or directory
grep: blah.txt: No such file or directory

Will gladly accept someone else's answer, but modifying my command to not use escaped quotes solved my issue. Thanks, @barlop.

In my particular search, I was able to change

grep -o 'ProductVersion\".*\".*\"' foo.txt >> blah.txt

to

grep -o 'ProductVersion.*Value.*' foo.txt | grep -v Name >> blah.txt

I would call this more of a workaround.

share|improve this question
    
That is correct. Testing in cmd.exe, eventually to be used in a .bat –  Thomas Nov 17 '11 at 16:25
    
Did you read the question? It's Cygwin grep and I showed output samples from grep. –  Thomas Nov 17 '11 at 16:28
    
Hmmm right.... sorry.. –  Diogo Nov 17 '11 at 16:30
1  
It's a mixture of escaped quotes and the pipe that is causing an issue, but i've no idea why.. If I take the example that doesn't work, and I put ( ) around the whole chunk before the pipe, it should work but doesn't. But does if there are no escape quotes there. –  barlop Nov 17 '11 at 16:36
1  
stackoverflow.com/questions/2737107/piping-findstrs-output shows the same issue. Wrapping the grep in a .bat would work, it seems. findstr supports ^" for escape as well, it seems. –  Thomas Nov 17 '11 at 16:52

2 Answers 2

up vote 0 down vote accepted

For Cygwin's grep

A workaround, is that you can specify the ASCII value in Bash. " is 22 in hex.

Two points: You have to remove the single quotes from around the first part, so that $'\x22' is interpreted as special, not as literal.

And for the second part of the expression you can't just use -o, it has to be -oE.

Because + is part of ERE, and without -E, it's just BRE. It thinks + is literal.

Proof + is literal there.. 55.55.55.55 won't match but this will:

$ echo 3+.3+.3+.3+ | grep -o [0-9]+\.[0-9]+\.[0-9]+\.[0-9]+
3+.3+.3+.3+

So here's the line you had but adjusted..

Using Bash's feature of expanding ASCII codes, instead of using quotes. Removing quotes from around the first part, and adding -E to second part:

$ grep -o ProductVersion$'\x22'.*$'\x22'.*$'\x22' foo.txt | grep -oE [0-9]+\.[0
-9]+\.[0-9]+\.[0-9]+
59.59.140.59

ADDED

If you replace [0-9]+ with [0-9][0-9]* (which is the same), then you can use grep without the -E.

You can use grep -P and then you can use \d for [0-9], but you have to use quotes around the second part. Or \\d.

In fact, here is a great solution that totally solves your original problem.. You only need a quote around the problematic bit. (By the way, I could make the regular expression in the second half more efficient using the repetition operator, but that's not relevant to the issue we've had with quotes which I'm focussing on).

This works. Dropping the single quotes from the first bit, and using \" to make them literal quotes. This gets round the bug of the double quotes needing to be single quoted. (Weird bug if Windows NT's findstr has something like it, though not with single quotes no doubt.)

grep -P in the second part, allows us to use \d. We could put quotes around the regular expression in the second half. Or, we can just put quotes around the '\d\ or, we can do as I have done and use \\d. (\d alone -unescaped and unquoted, won't match because it gets interpreted by Bash and reduced to d when grep gets it.)

$ grep -o ProductVersion\".*\".*\" foo.txt | grep -oP \\d+\.[0-9]+\.[0-9\]+\.[0
-9]+
59.59.140.59

Now that we've dealt with the quotes issue, I'll make it more efficient with the repetition operator. The regular expression of 3{4} means 3333. The regular expression of (fg){4} would mean fgfgfgfg.

$ grep -o ProductVersion\".*\".*\" foo.txt | grep -P '(\d.){4}'
ProductVersion" Value="59.59.140.59""

$ grep -o ProductVersion\".*\".*\" foo.txt | grep -P '('\\d.')'{4}
ProductVersion" Value="59.59.140.59""

$ grep -o ProductVersion'"'.*'"'.*'"' foo.txt | grep -P '('\\d.')'{4}
ProductVersion" Value="59.59.140.59""
share|improve this answer
    
Cool, thanks for the -E flag, I'd ended up making a much uglier flavor of that version regex to get it to work. –  Thomas Nov 17 '11 at 19:44
    
@Thomas can you show us the ugliness? what flavor version and regex you used? By the way, I just updated the answer to nail it a bit better! –  barlop Nov 17 '11 at 23:23
    
My ugly version used the [0-9][0-9]* construct you mentioned. The new versions of the ProductVersion grep are not working for me. The first 3 never execute (waiting for stdin) and the last won't pipe. This is what I'm going with for now: c:\eddynet\projects>grep -o 'ProductVersion.*Value.*' foo.txt | grep -v Name | grep -oE [0-9]+\.[0-9]+\.[0-9 ]+\.[0-9]+ >> blah.txt –  Thomas Nov 17 '11 at 23:48

You aren't using a Unix shell. Quoting is different.

For some reason, you think that single quotation marks are metacharacters in Microsoft's command interpreter. They are not. They have no special significance. Furthermore, backslashes have no special significance to Microsoft's command interpreter, either. They are not the escape character for quoting metacharacters. That is the caret. It's the double quotation marks that matter: they quote metacharacters such as <, >, and | to stop the command interpreter recognizing them, and any backslashes preceding them are irrelevant.

So your command line breaks down like this, with the quoted strings emphasized:

grep -o 'ProductVersion\".*\".*\"' foo.txt | grep -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'

As you can see, what you thought was a pipeline is in fact an improperly terminated quoted string starting from your third double quotation mark and extending to the end of the line. You're actually running just the one grep command, and you are giving it that entire quoted string at the end of its command tail. Your grep command knows about single quotation marks, and is breaking the command tail that it receives from the command interpreter, which still contains the double quotation marks (because the command interpreter recognizes, but does not remove, them), into seven words:

  1. -o
  2. ProductVersion\".*\".*\"
  3. foo.txt
  4. |
  5. grep
  6. -o
  7. [0-9]+\.[0-9]+\.[0-9]+\.[0-9]+

Hence the error messages about those files. But this is your grep command's doing. The command interpreter doesn't operate in terms of words, and Win32 programs are passed a single command tail, not an argument vector as in the Unix paradigm. It's up to the called program to split the command tail into words, if it wants to operate in Unix (and C language) fashion. (The runtime support libraries of most C and C++ language implementations for Win32 do this splitting behind the scenes. It's still the invoked program doing it, not the command interpreter, though.)

Indeed, many Win32 C and C++ programs that do not use the Cygwin libraries don't treat single quotation marks specially, any more than the command interpreter itself does. They'll mostly end up splitting the command tail into just two words:

  1. -o
  2. 'ProductVersion\.*".*"' foo.txt | grep -o [0-9]+\.[0-9]+\.[0-9]+\.[0-9]+

The preceding is what a program compiled with Microsoft's C/C++ compiler would do, for instance. Ironically, such C/C++ programs will recognize backslashes within quoted strings, even though the command interpreter does not. So they think that their command tail looks like this, with one big improperly terminated quoted string, instead of two quoted strings:

 -o 'ProductVersion\".*\".*\"' foo.txt | grep -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'

Yes, this is a command script writing nightmare. Essentially, you have to know what conventions the program you are running adheres to in order to decide how to quote the argument vector that you wish to pass to it. Cygwin has one convention. C and C++ programs compiled with commercial C and C++ compilers for Win32 have others. (In the 1980s and 1990s, Borland, Watcom, and Microsoft disagreed about command tail lexing in their C/C++ compilers for DOS, and there was a very subtle difference in backslash character handling among programs as a consequence.) Other programming languages do things in yet other subtly different ways.

You know that your grep command is a Cygwin program, so you need to construct a command line that (a) the command interpreter will correctly recognize as two simple commands joined into a command pipeline, and that then (b) the grep command will correctly split into words using the Cygwin algorithm. Here's one way:

grep -o 'ProductVersion^".*\^".*\^"' foo.txt | grep -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'
share|improve this answer
    
Could be i'm misunderstanding you but I just tried the line you suggested and in pure cmd.exe C:\>c:\cygwin\bin\grep -o 'ProductVersion^".*\^".*\^"' foo.txt | grep -oE '[0-9]+\. [0-9]+\.[0-9]+\.[0-9]+' (I had to use the full path to avoid running the gnuwin32 one which I also have). <-- and it doesn't match.(even with grep -oE).Similarly if loading the cygwin shell and running it it doesn't match.By the way, Your post is interesting, are you assuming he's running a cygwin program without loading cygwin(given his first line)?I suppose if he was,that's simply a very wrong way for one to do it. –  barlop Nov 18 '11 at 5:37
    
also, are lots of commands written in a language other than C and C++ that don't split behind the scenes? What language(that they're commonly written in) did you have in mind? or is it just other (a minority of) implementations of C and C++? –  barlop Nov 18 '11 at 5:41
    
I had given +1 though it may be your answer is in error. –  barlop Nov 18 '11 at 20:11
    
Your focus is in error. The questioner explicitly stated twice that xe is using cmd. No assumptions on my part are required. The questioner also asked about getting the pipe/redirection to work; not about tweaking the regular expressions used. So what regular expressions match against data that you've picked out of thin air is irrelevant. The focus here is getting the actual commands to work in a pipeline and with redirection. It's in the title, for goodness' sake! As for the rest: You should, by now, know how these sites work. Comments are not for StackOverflow questions. –  JdeBP Nov 19 '11 at 12:41
    
It could've been an issue had the mismatch been down to an error you made with escapes you used, though in this case, it wasn't, and the mismatch was down to an error in my comment, of a stray space and not using the full path of grep in the second part. As for you not thinking it appropriate to address a "stackoverflow question" in a comment.. You could also decide it inappropriate to give a "stackoverflow answer" in your post as in you writing "command tail lexing in their C/C++ compilers for DOS". I was merely asking a question on what you had written. –  barlop Nov 19 '11 at 13:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.