Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I hope I may ask this here.

I need to extract the contents of an existing web site (in charge of the web site owner) to word (or text) documents. For this, I only need the content from one DIV with a given ID.

Is there any tool for windows that can do this for me (ideally recursively)? I know wget and Web Site Downloader but both can "only" save the complete HTML.

share|improve this question
up vote 1 down vote accepted

Your best bet would be to build your own toolchain for this:

  1. Use a tool such as wget to recursively download the HTML files from which content is needed. Pay special attention to options -r to specify recursive downloading, and -l to specify depth of the recursion. wget outputs plain text.
  2. Use a tool such as grep to filter out everything except the line(s) containing the <DIV> you need. Pay special attention to options -r to specify recursive searching, and -e to specify a regular expression. Pipe grep's output to a file of your choice. grep outputs plain text if it is fed plain text.

Hint: It may be simpler to use grep multiple times to filter out things in smaller chunks. This depends entirely on how similar all of the various pages are, and how clean the code is.

Edit: Then again, perhaps using a regex is not a good way to parse HTML.

share|improve this answer
Another Hint: You may want to consider sed if grep doesn't do all of what you need. Not all web sites conveniently place carriage returns to put each DIV on a line by itself. – Chris Nava Nov 23 '09 at 14:58
This is exactly what I wanted to avoid. – Martin Nov 23 '09 at 15:21

I do not think something like this already exists. I think your best option would be to code something up yourself.

BeautifulSoup is a... beautiful Python library that will let you do this in very minimal code. For more help, I suggest you head over to Stack Overflow

share|improve this answer
Well, if I code it myself, I would look there, sure. Thanks anyway. – Martin Nov 23 '09 at 15:21

I'm lazy. In the time it would take you to research and set up a special-purpose tool, surely you can just highlight the required text with a mouse, copy it, and paste it into a text editor?

share|improve this answer
This is what I'm doing already. But it is boring and error-prone. That's why I'm searching for a tool to help me. – Martin Nov 23 '09 at 15:24

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.