Question

I hope I may ask this here.

I need to extract the contents of an existing web site (in charge of the web site owner) to word (or text) documents. For this, I only need the content from one DIV with a given ID.

Is there any tool for windows that can do this for me (ideally recursively)? I know wget and Web Site Downloader but both can "only" save the complete HTML.

Answer 1

Your best bet would be to build your own toolchain for this:

1. Use a tool such as wget to recursively download the HTML files from which content is needed. Pay special attention to options -r to specify recursive downloading, and -l to specify depth of the recursion. wget outputs plain text.
2. Use a tool such as grep to filter out everything except the line(s) containing the <DIV> you need. Pay special attention to options -r to specify recursive searching, and -e to specify a regular expression. Pipe grep's output to a file of your choice. grep outputs plain text if it is fed plain text.

Hint: It may be simpler to use grep multiple times to filter out things in smaller chunks. This depends entirely on how similar all of the various pages are, and how clean the code is.

---

Edit: Then again, perhaps using a regex is not a good way to parse HTML.

Answer 2

I do not think something like this already exists. I think your best option would be to code something up yourself.

BeautifulSoup is a... beautiful Python library that will let you do this in very minimal code. For more help, I suggest you head over to Stack Overflow

Answer 3

I'm lazy. In the time it would take you to research and set up a special-purpose tool, surely you can just highlight the required text with a mouse, copy it, and paste it into a text editor?