Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm looking for an easy toll which can crawl the web I give it to and extract all text elements from that. It will be nice if it can do: plain text, alt and title for images, header section. All of the separately if possible. The output should be somehow searchable or text files (xml) for every page it crawled. I need these text to pass them to translators.

share|improve this question
There are plenty of web crawlers. Here are a few - opensource and python – Praveen Oct 18 '12 at 13:09

Good old Lynx can provide you most features you requested. Try lynx -dump, for example.

You can also use wget for recursively crawling the sites you need, and then process the files with various converters available, such as htmltidy.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.