Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

When performing a recursive download, I specify a pattern via the -R parameter for wget to reject, but if this file is a HTML file, wget downloads the file regardless of whether or not it matches the pattern.

e.g.

wget -r -R "*dynamicfile*" example.com

still retrieves files such as example.com/dynamicfile1.html

Is there a way to prevent this?

share|improve this question

1 Answer 1

It does this because wget uses the html files to know where to scan next as it crawls through the webpage. I would just let wget do its business and then do a rm *.html after it is done, or something similar.

EDIT: Doing an rsync *dynamicfile* /foo/bar to a second directory might be a better way to filter your files to only keep the ones with the correct name (assuming that you want to keep some of the html files if they have the right name)

share|improve this answer
1  
I'm trying to filter the file because it causes wget to get stuck in an infinite loop, so this won't work. –  Mike B Apr 20 '10 at 18:05
    
Sounds like your infinite loop is the true issue your trying to deal with. This is different enough that you should probably just post a new question instead asking about preventing infinite loops with wget. –  Jarvin Apr 20 '10 at 18:36
    
You should add a depth limit to wget. This will make sure it isn't an infinite loop. –  Jarvin Apr 20 '10 at 18:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.