Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a html-page url and I want to grep it. How can I do it by wget someArgs | grep keyword?

My first idea was wget -q -O - url | grep keyword, but wget's output bypass grep and arise on the terminal in its original form.

share|improve this question
grep selects lines delimited by (e.g.) carriage return and linefeed characters, an HTML response doesn't have lines it has text with markup like <br> or <p> so the whole web-page could look like one line to grep – RedGrittyBrick Jun 1 '12 at 19:44
@RedGrittyBrick The OP's command works flawlessly for me. – slhck Jun 1 '12 at 19:47
up vote 3 down vote accepted

The easiest way is to use curl with the option -s for silent:

curl -s | grep whatever
share|improve this answer
@slhck: Both commands do exactly the same for me. – Dennis Jun 1 '12 at 21:39
@Dennis Try curling For whatever reason I tested it with this particular URL and got no output. Dunno what I'm missing. – slhck Jun 1 '12 at 21:45
@slhck: Curl doesn't follow redirects by default. It does with the -L switch. – Dennis Mar 31 '13 at 20:15
@Dennis Didn't know what you were talking about without seeing the deleted comments – but yeah, that makes sense. Thanks for clearing it up. – slhck Mar 31 '13 at 20:27

Keeping this around for the sake of completeness.

Your example should actually work. The syntax is correct, and here's a screencast I just took demonstrating it, with a good old GNU wget 1.13.4.

wget -q some-url -O - | grep something

So assume your pattern is wrong and grep will just output everything it got.

share|improve this answer
It could also be a typo in the URL. With -q, there is no error message. – Dennis Jun 2 '12 at 0:44

This bug was in v1.12.1 fixed in another version. Currently I use v1.15 and it works as expected.

share|improve this answer

If you are looking to grep or pipe headers, they are standard directed to stderr so you need to redirect them. Eg:

wget -O - > /dev/null 2>&1 | grep HTTP
share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .