Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

How do I extract all the external links of a web page and save them to a file?

If you have any command line tools that would be great.

share|improve this question

4 Answers 4

up vote 10 down vote accepted

You will need 2 tools, lynx and awk, try this:

$ lynx -dump http://www.google.com.br | awk '/http/{print $2}' > links.txt

If you need numbering lines, use command nl, try this:

$ lynx -dump http://www.google.com.br | awk '/http/{print $2}' | nl > links.txt
share|improve this answer
  1. Use Beautiful Soup to retrieve the web pages in question.
  2. Use awk to find all URLs that do not point to your domain

I would recommend Beautiful Soup over screen scraping techniques.

share|improve this answer

if command line is not a force you can use Copy All Links Firefox extension.

share|improve this answer

Here's an improvement on lilton's answer: you don't need awk at all for lynx's got some useful options.

lynx -listonly -nonumbers -dump http://www.google.com.br

if you want numbers

lynx -listonly -dump http://www.google.com.br
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.