Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I would like to retrieve all .html files from a website that has a certain text on its name:

eg. this_is_good_site.html

So, I would like to download .html files with a word "good" on its name. I tried wget and curl, but I did not understand how can I choose those files using a regular expression? Is there a Python or Perl solution, if command line tools on Unix can't do this?

share|improve this question

3 Answers 3

up vote 1 down vote accepted

Well, if you're wanting to do it with Python you might look into using urlib2 - you would also probably have better luck with this question on StackOverflow.

share|improve this answer

As you're using a Unix environment, try this using wget's Recursive Accept/Reject Options;

wget -r -A "*good*" <site_to_download>

This will perform a recursive (-r) download of the site, and only accept (-A) paths which match the pattern ("*good*")

share|improve this answer

Try HTTrack website copier or a similar program, better than command line. download it all to a directory, sort by .html copy and paste them all somewhere else, delete the leftovers

http://www.httrack.com/

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.