Why do sites offer an MD5 hash of a file? How does that help you verify the integrity/source of the file? Wouldn't paying attention to your URL be enough security?
2
-
This is not a programming question. Maybe better for Server Fault? The idea of the hash is that you can better detect corruption of the file -- either deliberate or bit rot.– GrayOct 18, 2010 at 13:17
-
Every 2^10 to 2^12 bit is messed up, data is sent in packages and thy also contain hash (if it does not match package is resent.), because of this normal people do not even notice this. On heavy network load or couple more reasons failing rate increases. MD5 shows up little changes what passed the tests (with big difference).– MargusOct 18, 2010 at 13:23
Add a comment
|
7 Answers
It helps to verify that once you are done downloading the file, that you have the same file that is sitting on their server.
After running your local file through a hash function, if the slightest difference is found between your copy and theirs, it will result in you getting a different hash than what they have posted.
An MD5 hash doesn't protect you if the site gets hacked because the MD5 sum an be changed as well by the hacker. A GPG signature (in a separate file) would protect against this if you have a copy of the public key that you've gotten from somewhere else, or verified that it's the correct key by checking the signatures on the key.
What an MD5 hash is good for is making sure the file didn't get corrupted in the download process. TCP checksums don't all of the errors that could occur while your file is in transit (though those errors are rare), and I have seen an ISO image get corrupted during the download process before.
Security is one (important) thing, but I think it is also to make sure you downloaded the file in its entirety.
-
That's also checked when hashing the file, but filesize is much easier to use for that purpose.– Roger PateOct 20, 2010 at 9:24
Well in theory both files should produce the same hash if they are exactly the same (though collisions do occur).
You can provide it easily with a server side language, such as ol' trusty PHP...
echo md5_file('my_file.zip');
-
Although that'll regenerate the MD5 hash for each pageview (if you don't have a cache in place), and generating an MD5 hash for a big file (hundreds of megabytes, for example a Linux distribution) can take a few minutes - more than PHP will usually swallow, and you don't want your users to wait for minutes for a page to generate. There's md5 commandline tools available for all OSes, better to use those and 'statically' add those to the list 'o downloads (or the public ftp folder).– cthulhuOct 18, 2010 at 14:26
-
@Cthulhu Yep, I was trying to keep the example relevant - but you will want to cache that hash, or better still calculate it once on acquiring the file - or have a background process do it.– alexOct 18, 2010 at 15:14
It offers a defence against a possible man-in-the-middle attack. Suppose a malicious party had somehow been able to hijack the request for the file, but had not been able to alter the text of the web page itself. The MITM would be able to get you to download a different sequence of bytes, but would not be able to make the hash of his file match the hash quoted on the qweb page, that of the real file.
-
2You only get protected against a MITM attack if you're sure you've obtained the reference hash from a trusted source.– BrunoOct 18, 2010 at 13:21
-
@Bruno: However, though not often the case nowadays with higher connection speeds, you could get the hash from the official site and the file on local media from someone else (i.e. a real man in the middle ;). This is also the principle behind BitTorrent.– Roger PateOct 20, 2010 at 9:21
There are several reasons:
- a Man-in-the-Middle attack could possible alter your download and inject a virus or trojan or other malicious code in the download
- It's also possible that some corruption occurred in the bitstream while downloading and the downloaded file might be broken.
- There might be a proxy server involved which might cutoff the connection early.
- Some older browser couldn't download files bigger than 2GB, so downloading bigger files would also mean a corrupted download.
- It also allows a user to check if the "same" file on a mirror server is really the same file and not an altered version.
Sometimes, files transferred may be corrupt, either intentionally by an attacker or due to a faulty connection.
Cryptographic hash functions (such as MD5) are designed to change if there's a change in the original file. As such, you can detect such alterations.
Note that, if you want protection against intentional modification (made by an attacker), you need a way to verify that the reference hash you obtained is correct and from a trusted source (e.g. signature via PGP or X.509 certificates for example). You may also need a better hash algorithm (e.g. SHA-1, SHA-256), as collisions attacks have been demonstrated against MD5.
