Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to scrape a website for all sites and append an Paramater, e.g.

Now what I got is wget --recursive to get all sites, but that doesn't append the parameter and the content is useless. I need to do this in a 2nd step:

curl | grep -c MyString

So in essence I am looking for a particular String in a entire Website to which I got only access via HTTP.

share|improve this question
up vote 1 down vote accepted

I managed now by using brackets and curl like this:

curl{+cat1,+cat2,+cat3,+cat4,+cat4}?myPara=us | grep -c myString

Of course this won't get me all sites, but one could work out a script with wget, regex, collections and loops.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.