Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I want to make a download of a forum I moderate, before it closes for good. There's some useful info on it I want to save for myself and I don't want to export the data to another webserver, I just want the pages. Mind you, I'm a user at the forum, not the admin. Now, I googled this and found it can be easily done with wget: How can I download an entire (active) phpbb forum?

I used:

wget -k -m -E -p -np -R viewtopic.php*p=*,memberlist.php*,faq.php*,posting.php*,search.php*,ucp.php*,viewonline.php*,*sid*,*view=print*,*start=0* -o log.txt

I experimented with this, but I can only achieve downloading the publicly visible sections, not the sections you have to log in for. I tried to achieve this by using a Firefox plugin to make a cookies.txt (while my session is logged into the forum) and add --load-cookies file cookies.txt to the command, but still I only get the publicly visible sections.

Any suggestions to make this work?

share|improve this question
Do you know any Python? – paradroid Dec 30 '10 at 13:48
One thing I assume is happening is that wget follows all the links on the pages it finds. If you start at the index page of a forum, I assume it goes through the code top to bottom. One of the links on the index page is the 'logout' link. Whenever it hits that, it automatically logs out the session. I've tried adding and exclude for login.php* or something like that, but so far I keep getting the same results. Anyone know how I can tell wget to exclude any page that starts with*? so that it doesn't log itself out while it is busy? – user60891 Feb 4 '11 at 7:17

You may need to set up cookies for the session because many web sites use cookies to make the login and logout functions work.

The "--load-cookies" option might help you here.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .