Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

How do you use wget to download an entire site (domain A) when its resources are on another domain, (domain B)?
I've tried:
wget -r --level=inf -p -k -E --domains=domainA,domainB http://www.domainA

share|improve this question
    
Wow! No one after all this time? –  Parsa Oct 1 '10 at 8:03
add comment

3 Answers 3

up vote 3 down vote accepted

wget --recursive --level=inf --page-requisites --convert-links --html-extension --span-hosts=domainA,domainB domainA

share|improve this answer
    
I get: wget: --span-hosts: Invalid boolean domainA,domainB'; use on' or `off'. After changing to on, it does not work. –  Matthew Flaschen Feb 14 at 1:26
    
@MatthewFlaschen What I've written here worked for me. Could you provide the arguments you've used? –  Parsa Feb 26 at 2:04
    
I don't have the exact command I ran before. However, I have the same problem with: wget --recursive --level=inf --page-requisites --convert-links --html-extension --span-hosts=example.org,iana.org example.org I'm using GNU Wget 1.13.4 on Debian. –  Matthew Flaschen Feb 28 at 5:42
add comment

wget --recursive --level=inf --page-requisites --convert-links --html-extension -rH -DdomainA,domainB domainA

share|improve this answer
    
This partly works. However, for some reason, it doesn't seem to work if the URL (at the end) is a redirect. Also, it downloads links too, not just page requisites. Also, -r and --recursive are the same. –  Matthew Flaschen Feb 14 at 1:44
add comment
wget --page-requisites --convert-links --adjust-extension --span-hosts --domains domainA,domainB domainA

You might need to ignore robots.txt (note, this may be a violation of some terms of service, and you should download the minimum required). See https://www.gnu.org/software/wget/manual/wget.html#Robot-Exclusion .

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.