0

Linux centos 6-64

What should I add to wget for it only to read the links on the domain requested and not go to other domains mention inside the site being downloaded?

Ex: while mirroring site1.com it is going to site2.com because there is a link to site2.com inside site1.com

As I dont know which are the site2.com, site3.com, etc he will find, so I didnt find suitable to use the domain exclude option.

Is there an option for it to read only the site.com links?

Thanks

1

man wget says

  -D domain-list
   --domains=domain-list
       Set domains to be followed.  domain-list is a comma-separated list
       of domains.  Note that it does not turn on -H.

Note also GNU wget spanning hosts says

Wget’s recursive retrieval normally refuses to visit hosts different than the one you specified on the command line. This is a reasonable default; without it, every retrieval would have the potential to turn your Wget into a small version of google.

So maybe you are using -H for some reason?

Limit spanning to certain domains—‘-D’

The ‘-D’ option allows you to specify the domains that will be followed, thus limiting the recursion only to the hosts that belong to these domains. Obviously, this makes sense only in conjunction with ‘-H’. A typical example would be downloading the contents of ‘www.example.com’, but allowing downloads from ‘images.example.com’, etc.:

  wget -rH -Dexample.com http://www.example.com/
  • Hello Brick, yes, indeed was missing the -D, now is ok, thank you – Joao Jul 31 '18 at 21:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.