Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I would like to retrieve all .html files from a website that has a certain text on its name:

eg. this_is_good_site.html

So, I would like to download .html files with a word "good" on its name. I tried wget and curl, but I did not understand how can I choose those files using a regular expression? Is there a Python or Perl solution, if command line tools on Unix can't do this?

share|improve this question
up vote 1 down vote accepted

Well, if you're wanting to do it with Python you might look into using urlib2 - you would also probably have better luck with this question on StackOverflow.

share|improve this answer

As you're using a Unix environment, try this using wget's Recursive Accept/Reject Options;

wget -r -A "*good*" <site_to_download>

This will perform a recursive (-r) download of the site, and only accept (-A) paths which match the pattern ("*good*")

share|improve this answer

Try HTTrack website copier or a similar program, better than command line. download it all to a directory, sort by .html copy and paste them all somewhere else, delete the leftovers

share|improve this answer

You must log in to answer this question.