Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

For a long time I have assumed that it is not really possible to download a corrupted file via http as long as it's not corrupted on the server and the implementation of http protocol is correct, which is most likely the case for modern mainstream software.

So I always chuckled when I saw a download site offer an md5 hash of a file they provide for download. I haven't seen a case before, where I download a file, the size is correct but the content is not.

Well, today, I had a first case of this. I downloaded an iso of Ubuntu, tried to install it, it failed and after a long research (I just could not believe that the reason could be a corrupted download) I checked the MD5 and what do you know, it was wrong (size was correct). So I re-downloaded it and got yet another wrong md5. Only on my third download the md5 was correct.

So my question is, is it possible in principle to get corrupted download over http, assuming that the implementation is correct, the transfer has finished successfully and that the file is correct on the server. If this is possible, then how can this happen?

share|improve this question
1  
Also see: Can a slow internet connection corrupt a file? –  slhck May 14 '11 at 8:08
add comment

2 Answers

up vote 4 down vote accepted

Yes, it's possible, especially on poor quality Internet connections – usually wireless, but some wired connections (such as the one I have) also have high error rates at high speeds.

The HTTP protocol does not have any provisions for ensuring data integrity. On transport layer, TCP does have error detection by using a checksum, but it's not very reliable.


There is another reason for providing hashes or digital signatures. Often, the actual files are distributed over many mirror servers, which cannot be guaranteed to be 100% secure. If there's no hash or signature to verify, someone with access to a mirror (not necessarily legitimate) could replace the files and remain undetected, without having to break into a completely different server where the website is hosted.


You can get automatic verification of files if you download Ubuntu over BitTorrent instead of HTTP. (Each piece is verified at download time, so you never have to re-download the entire thing.)

share|improve this answer
1  
Good answer. However I would like to explore the topic a bit more, if you don't mind. The "high speed" part turned out to be very relevant. I never had any problems at home, but this Ubuntu ISO downloads that I did was at work an the speed was about 5Mb/s. It's hard for me to accept that TCP is not reliable, because almost everything around is based on TCP. Is there anything else but this wikipedia article about TCP unreliability? Do you know exactly how high speed affects the issue? Thank you in advance. –  zespri May 16 '11 at 1:05
1  
@zespri: TCP is supposed to be reliable, but the checksum cannot catch 100% of all errors. High speed is only a factor with certain connection types, or over unreliable links... Which unfortunately can occur anywhere between you and the server, not necessarily immediately at your end. (I'd test downloading from mirrors located at different countries.) There's of course also the possibility that your file was already corrupted on the server's filesystem... –  grawity May 16 '11 at 4:14
1  
if there is a small probability of corruption, then increasing the number of hops you have in between you and the server would increase the chance of corruption??? –  Trevor Boyd Smith Sep 14 '11 at 14:07
1  
@Trevor: Depends on the link quality. Ten hops over Ethernet are far more reliable than a single hop over WiFi. (The routers/switches themselves almost never corrupt data; it's usually the connection that does it.) –  grawity Sep 14 '11 at 14:13
add comment

As Grawity said, it is possible, but in addition, what I have noticed:

Even with stable internet connections, it is possible for a download to finish early without any sort of valid reason - it can just happen.

And most importantly, if you have bad memory, it is possible that the download caches in memory before being written to disk, and as the memory is bad, it is writing the file incorrectly. If you constantly have bad downloads, this is a high posibility.

share|improve this answer
    
Oh, I hate webservers that randomly cut off downloads at ~50% without any support for resuming... –  grawity May 14 '11 at 14:43
1  
Wil: Yep, cut offs are often, and because of this I specifically excluded them from my question, saying that I assume that the sizes are exactly right. Thank you for you comment. –  zespri May 16 '11 at 1:08
    
Yep that is a key statement for this question "the sizes are exactly right" –  Trevor Boyd Smith Sep 14 '11 at 14:08
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.