1

I hope I may ask this here.

I need to extract the contents of an existing website (in charge of the website owner) to Word (or text) documents. For this, I only need the content from one DIV with a given ID.

Is there any tool for Windows that can do this for me (ideally recursively)? I know wget and Web Site Downloader, but both can "only" save the complete HTML.

3 Answers 3

1

Your best bet would be to build your own toolchain for this:

  1. Use a tool such as wget to recursively download the HTML files from which content is needed. Pay special attention to options -r to specify recursive downloading, and -l to specify depth of the recursion. wget outputs plain text.
  2. Use a tool such as grep to filter out everything except the line(s) containing the <DIV> you need. Pay special attention to options -r to specify recursive searching, and -e to specify a regular expression. Pipe grep's output to a file of your choice. grep outputs plain text if it is fed plain text.

Hint: It may be simpler to use grep multiple times to filter out things in smaller chunks. This depends entirely on how similar all of the various pages are, and how clean the code is.


Edit: Then again, perhaps using a regex is not a good way to parse HTML.

2
  • 1
    Another Hint: You may want to consider sed if grep doesn't do all of what you need. Not all web sites conveniently place carriage returns to put each DIV on a line by itself.
    – Chris Nava
    Nov 23, 2009 at 14:58
  • This is exactly what I wanted to avoid.
    – Martin
    Nov 23, 2009 at 15:21
0

I do not think something like this already exists. I think your best option would be to code something up yourself.

BeautifulSoup is a... beautiful Python library that will let you do this in very minimal code. For more help, I suggest you head over to Stack Overflow

1
  • Well, if I code it myself, I would look there, sure. Thanks anyway.
    – Martin
    Nov 23, 2009 at 15:21
0

I'm lazy. In the time it would take you to research and set up a special-purpose tool, surely you can just highlight the required text with a mouse, copy it, and paste it into a text editor?

1
  • This is what I'm doing already. But it is boring and error-prone. That's why I'm searching for a tool to help me.
    – Martin
    Nov 23, 2009 at 15:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .