4

I'm new to using wget, so I hope this is not a dumb question :

How can I save a file to a local directory that has been named after the link to the file ?

It does not seem to work if a redirection was involved in the download.

For instance: say I'm downloading a file from www.abc.com/news/xyz.jif, I can get wget to save a local copy to a directory named www.abc.com with sub directory news and then file xyz.jif.

If however the file xyz.jif was moved to www.qwerty.com/old/xyz.jif and the search was re-directed there, the file is still saved under www.abc.com...etc

As I said, I'm new to this, so I hope I've explained the problem correctly.

Here's a simple real world example (using wget):

wget --force-directories www.smugmug.com/photos/1235566-S.jpg

it is saved to : c:\www.smugmug.com/photos/1235566-S.jpg , which is incorrent. It should have saved it to c:\suz.smugmug.com/photos/1235566-S.jpg, where it was redirected to.

1 Answer 1

0

Sorry, didn't catch the point initially. So, a complete new answer (quick and dirty - and since it's bash programming, it works only with linux): use cURL to determine if there's a redirect (using -w parameter); depending on the result, use wget to get either the original url or the redirect target:

url=www.smugmug.com/photos/1235566-S.jpg; redirurl=`curl -w "%{redirect_url}" --output /dev/null --silent ${url}`; if [ -z $redirurl ]; then wget -x $url; else wget -x $redirurl; fi

Or, on multiple lines:

url=www.smugmug.com/photos/1235566-S.jpg;
redirurl=`curl -w "%{redirect_url}" --output /dev/null --silent ${url}`;
if
    [ -z $redirurl ];
then wget -x $url;
else wget -x $redirurl; fi

redirurl is set to the redirection target if there is any. If not, wget is called with the original url; if yes, wget calls the redirection target. The -x parameter makes wget create the subdirs.

Of course, this handles only one level of redirection. To make it more robust, you'ld have to check recursively if there's a redirection. So, it isn't just a matter of command line options if you want to solve this issue in a reliable way.

5
  • I tried that already , it does not work unfortunately. It still saves the file under the original URL , not the redirected one :0(
    – Robin
    Jun 4, 2015 at 9:49
  • I don't think I've been clear in my explanation , sorry. I want wget / curl to automatically save the file with the same directory names from where it got the file. From my first post get file from www.abc.com/news/xyz.jif , but file has been moved to www.qwerty.com/old/xyz.jif , unknown to me. At the end of the download file will be in c:\www.qwerty.com/old/xyz.jif , assuming I ran wget/curl from C drive
    – Robin
    Jun 4, 2015 at 12:00
  • Here is a simple example (using wget) that you can reproduce quickly. wget --force-directories www.smugmug.com/photos/1235566-S.jpg it is saved to : c:\www.smugmug.com/photos/1235566-S.jpg , which is INCORRECT. It should have saved it to c:\suz.smugmug.com/photos/1235566-S.jpg , where it was redirected to. In the curl example you have been given , you are telling it where to save the files. It must automatically grab the redirected url , which you wont know obviously.
    – Robin
    Jun 4, 2015 at 12:55
  • Sorry, now i understand - i've removed my comments and changed my answer.
    – tohuwawohu
    Jun 4, 2015 at 19:18
  • I'm not using linux , but that has pointed me in the right direction!! That "write_out" function of curl is pretty powerful. Must spend some time getting to know it. I have what I need now. Thanks again!!
    – Robin
    Jun 5, 2015 at 8:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .