24

If I write wget "no such address" -o "test.html" it first creates the test.html and in case of failure, leaves it empty. However, when not using -o, it will wait to see if the download succeeds and only after that, it'll write the file.

I'd like the latter behavior to be applied to -o too, is it possible?

4 Answers 4

27

wget returns a non-zero exit status when the URL is not found, so you can append a remove command on failure:

wget "url" -O file || rm -f file

Or create a temporary file and only move it where you want on success:

wget "url" -O /tmp/wget && mv /tmp/wget file

The second has the benefit of not deleting an existing file on failure, but be sure to use unique temporary names (see man tempfile) if you're running multiple instances in parallel.

2
  • Also, adding --retry-connrefused can help in preventing the empty file in the first place.
    – akom
    Dec 29, 2017 at 15:40
  • If this is happening in an exec in a puppet manifest, changing creates => file to unless => "[ -s file ]" can make it self-healing.
    – akom
    Dec 29, 2017 at 15:41
19

As written in the comments, wget -O is more like a shell redirection which always writes into the file regardless of errors.

You can use curl -f instead:

curl -f http://nonexistent/file.jpg -o localfile.jpg

It will not touch the local file if there is an error fetching the file.

4

The correct syntax is

wget "url" -O file

notice the UPPERCASE O. The -o options tells wget to write a log file, that's why it's always written even on failure.

6
  • At first I thought it was working, but then I found it didn't. try wget "host.does.not.exist" -O "emptyFile" An error is returned, yet the emptyFile is created.
    – akurtser
    Jul 22, 2010 at 9:38
  • 1
    @akurtser you're right. I think there's no way to tell wget not to create the file. I found this thread: mail-archive.com/[email protected]/msg08586.html in which they discuss the matter. The baseline is that you can have MULTIPLE downloads to the same file so it gets created because wget cannot be shure that ALL urls will fail. Jul 22, 2010 at 11:41
  • Well thanks, it's a part of a bash script I'm writing, so I'll just try to first save it a temp file, which in case of successful download, will be be renamed. Not very elegant, but can't think of anything better.
    – akurtser
    Jul 22, 2010 at 12:19
  • 1
    @akurtser Surely you can check the return code from wget then... it should tell you if you can delete the file "if not found". So no need for temp/renaming. Jul 22, 2010 at 12:42
  • 1
    The -O option is a redirection, which redirects the downloaded content to a file, even in cases when there are no contents. Therefore, a file is always created, even if the download failed.
    – Quan To
    Sep 15, 2017 at 9:13
0

According to the help doc(wget -h), you can use --spider option to skip download(version 1.14).

Download:
  -S,  --server-response         print server response.
       --spider                  don't download anything.

1
  • So then we use wget --spider "url" && wget "url" -O file Jul 11, 2022 at 10:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .