I am using the following code to download a list of pdf files:
wget -i list.txt -A .pdf
Some pdf files are downloaded properly. However, some pdf files are not downloaded properly. When I check the log, I see the following report:
--2013-04-09 11:25:42-- http://amazon.com/111.pdf Reusing existing connection to amazon.com:80. HTTP request sent, awaiting response... 200 No headers, assuming HTTP/0.9 Length: unspecified Saving to: `111.pdf' [ <=> ] 1,045 --.-K/s in 2m 9s 2013-04-09 11:27:51 (8.11 B/s) - Read error at byte 1045 (Connection reset by peer).Retrying. --2013-04-09 11:27:52-- (try: 2) http://amazon.com/111.pdf Connecting to amazon.com (amazon.com)|00.00.55.888|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2680728 (2.6M) [application/pdf] Saving to: `111.pdf' 61% [==============================> ] 1,649,221 10.0K/s in 2m 41s 2013-04-09 11:30:41 (10.0 KB/s) - Read error at byte 1649221/2680728 (Connection reset by peer). Retrying. --2013-04-09 11:30:43-- (try: 3) http://amazon.com/111.pdf Connecting to amazon.com (amazon.com)|00.00.55.888|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2680728 (2.6M) [application/pdf] Saving to: `111.pdf' 100%[==================================================>] 2,680,728 10.1K/s in 4m 22s 2013-04-09 11:35:11 (10.0 KB/s) - `111.pdf' saved [2680728/2680728]
I wonder why I can not open the pdf file 111.pdf? The above report says that it is 100% downloaded. Is it because of the connection reset by the peer?
I wonder if it is possible to put the wget in a loop for every file, in such a way that it does not exit the loop, until the download is done with no error? I found the following loop, however, it gives an error. The code and the error is shown below:
while read -r link do wget -A .pdf until [ $? = 0 ] do wget -A .pdf done done < ./list.txt
Try `wget --help' for more options. wget: missing URL Usage: wget [OPTION]... [URL]...
I am using Cygwin on Windows.
Please let me know if you have other suggestions as well.
Thank you for your help.