Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a webpage that basically consists of a list of links to other pages. Using wget I would like to download all the pages listed.

Using "wget -r -l1 URL" I basically get what I want.

But how to do the same if the list is split over several pages (with URLs ending in "?page=3", "?page=4"....).

share|improve this question

migrated from Apr 7 '13 at 1:56

This question came from our site for system and network administrators.

up vote 1 down vote accepted

If you know the number of pages, you could use a for-loop:

for i in {1..5}; do wget -r -l1 URL?page=$i; done
share|improve this answer
Thanks. That could work. In fact I have several such lists, but I could perhaps just pick some very large number and see what happens if the for loop hits a non-existant page. – user1583209 Apr 6 '13 at 14:35

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.