Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The url of the forum thread I'm trying to get access to is in the form

http://domain.com/showthread.php?threadid=3333333&userid=0&perpage=40&pagenumber=1

I've tried

wget --user-agent=Mozilla/5.0 -k -m -E -p -np -R http://domain/showthread.php?noseen=0&threadid=3333333&pagenumber=1

and I've had no luck.

share|improve this question

Why not just in a for loop:

for pageno in {1..1000000}; do
    wget ... http://domain/showthread.php?noseen=0&threadid=3333333&pagenumber=$pageno || break
done

or perhaps a while loop is better, if a little longer to write:

i=1
while true; do
    wget ... http://domain/showthread.php?noseen=0&threadid=3333333&pagenumber=$pageno
    if test $? -ne 0; then
        break
    fi
    i=$((i+1))
done
share|improve this answer
    
OP here, thanks Mikel. Actually I'm having problems just downloading the contents of the one page in the thread. Once I have that going I was thinking of the first way you suggested, but there seems to be something wrong with my wget paramaters (? maybe? I don't know) – user65999 Feb 4 '11 at 2:01

Might be worth checking if the forum supports rss feeds of sections / threads. It'd save you bother.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .