Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a webpage that basically consists of a list of links to other pages. Using wget I would like to download all the pages listed.

Using "wget -r -l1 URL" I basically get what I want.

But how to do the same if the list is split over several pages (with URLs ending in "?page=3", "?page=4"....).

share|improve this question
add comment

migrated from serverfault.com Apr 7 '13 at 1:56

This question came from our site for professional system and network administrators.

1 Answer

up vote 1 down vote accepted

If you know the number of pages, you could use a for-loop:

for i in {1..5}; do wget -r -l1 URL?page=$i; done
share|improve this answer
    
Thanks. That could work. In fact I have several such lists, but I could perhaps just pick some very large number and see what happens if the for loop hits a non-existant page. –  user1583209 Apr 6 '13 at 14:35
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.