Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to make a download of a forum I moderate, before it closes for good. There's some useful info on it I want to save for myself and I don't want to export the data to another webserver, I just want the pages. Mind you, I'm a user at the forum, not the admin. Now, I googled this and found it can be easily done with wget: How can I download an entire (active) phpbb forum?

I used:

wget -k -m -E -p -np -R viewtopic.php*p=*,memberlist.php*,faq.php*,posting.php*,search.php*,ucp.php*,viewonline.php*,*sid*,*view=print*,*start=0* -o log.txt http://www.example.com/forum/

I experimented with this, but I can only achieve downloading the publicly visible sections, not the sections you have to log in for. I tried to achieve this by using a Firefox plugin to make a cookies.txt (while my session is logged into the forum) and add --load-cookies file cookies.txt to the command, but still I only get the publicly visible sections.

Any suggestions to make this work?

share|improve this question
    
Do you know any Python? –  paradroid Dec 30 '10 at 13:48
1  
One thing I assume is happening is that wget follows all the links on the pages it finds. If you start at the index page of a forum, I assume it goes through the code top to bottom. One of the links on the index page is the 'logout' link. Whenever it hits that, it automatically logs out the session. I've tried adding and exclude for login.php* or something like that, but so far I keep getting the same results. Anyone know how I can tell wget to exclude any page that starts with http://www.example.com/forum/login.php*? so that it doesn't log itself out while it is busy? –  user60891 Feb 4 '11 at 7:17
add comment

1 Answer

You may need to set up cookies for the session because many web sites use cookies to make the login and logout functions work.

The "--load-cookies" option might help you here.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.