12

I downloaded a huge amount of files, but discovered the use of md5 and sha as integrity checkers quite recently. From then I always prefer to check it for big downloaded files, even if I never found them to be corrupted.

Do we really need to check the integrity of downloaded files?

Pick as example a Linux distribution that I have just downloaded, which is 1GB, if you want.

Thank you

Improve this question
7
  • Many downloads are http downloads and not few of the big ones stop prematurely due to server time limits or connectivity issues, without giving any error. Torrents and ftp on the other hand should make it clear if the file is complete.
    – Dan
    Feb 24, 2015 at 18:53
  • 5
    The point of comparing the hash can rather be "has someone altered the file (maliciously)" than "has the file been downloaded correctly". One prime example is to download a windows ISO from a non-official/not trustworthy source and check the integrity of the file by comparing the MD5 hash with the one Microsoft published on MSDN.
    – TheUser1024
    Feb 24, 2015 at 18:54
  • 1
    @TheUser1024 this assume that your two sources are the same, the MSDN version may be different.
    – Eric G
    Feb 24, 2015 at 19:04
  • 1
    This question may get more responses on security.SE
    – Eric G
    Feb 24, 2015 at 19:08
  • 3
    It was actually more on integrity than security, since I only download from trusted sources, even if it is an important point in file download
    – maxpesa
    Feb 24, 2015 at 19:10

5 Answers 5

Reset to default
6

It depends on a few factors.

  1. Do you have a stable internet connection?
    If you have a stable internet connection, you do not need to check the integrety of the file as it will most likely be correct. I never check the hash and I also never have had corrupt files. Or maybe one time when the remote server disconnected.

  2. Do you want to verify the file based on security reasons?
    If you are concerned about the safety of the file you're downloading, you can use the MD5 hash to verify that the file was not somehow altered. You download a file and if the MD5 hash doesn't match, it means the file on the server is different than the MD5 hash, and somehow something is wrong. This would only be valid if you don't trust the server you're downloading from, but usually if someone provides a hash, they usually also try their best to keep things updated. But if their site got hacked, and you did check the MD5 hash, then you got the little bonus.

Overall, these 2 will give a no to most people. If its a no to you, thats entirely up to you of course.

Improve this answer
5
  • 2
    A stable internet connection is not sufficient. You also have to be sure your RAM and storage drive do not corrupt the files. But yeah, pretty unlikely. If your machine isn't BSOD'ing, chances are you would only care about the security.
    – ChrisInEdmonton
    Feb 24, 2015 at 19:01
  • 6
    A file checksum is not a security check since an attacker that can modify the file could likely also modify the checksum.
    – Johnny
    Feb 24, 2015 at 19:58
  • 1
    If I were to hack a site and replace the file with a malicious one, I think I'd be smart enough to know to change the hash listed.
    – Cole Tobin
    Feb 24, 2015 at 21:32
  • 3
    MD5 is no longer sufficient to verify that the contents of a file have not been intentionally altered. It's only good for unintentional corruption. The odds of an attacker compromising both vary. The storage for the hash, in many cases, is not accessed through the same channels as the bulk file storage for a large download. For security, cryptographic signatures are better than simple hashes, especially if you don't have a secure channel for distributing the hashes.
    – Perkins
    Feb 24, 2015 at 22:42
  • I think the idea of hashes was when the file supplied was hosted on a different site to the hash. Eg a developer's site with a hash links to Sourceforge or somewhere.
    – Matthew Lock
    Feb 25, 2015 at 2:51
6

The answer and choice one makes is going to be based on his/her risk tolerance and considerations of time and effort in verification.

Checking MD5/SHA1 hashes is a good first step and you should do it when you have time. However, you must consider your ability to trust the hash provided. For example, if the the author's website with the hash is hacked, then the attacker can change the hash, so you would not know. If the hash you calculate is not the same as the hash provided, you know something is up. However, just because the hashes match does not guarantee the file is good.

A better alternative for a software author to provide integrity and authenticity is through digitally signing the files being distributed. This attaches the authenticity information to the file and does not rely on trusting some website. If an author digitally signs the file, the only way for this to be faked is a compromised certificate authority or if the developer's signing key was stolen. Both of these cases are far less likely than a website on the Internet being hacked.

Ultimately, you must do your own due diligence to determine if you want to trust something and then take countermeasures (run in a sandbox, a virtual machine, etc.) to mitigate any unknown factors or miscalculations you made when deciding whether or not to trust.

Improve this answer
4

For security reasons, YES. Consider that a Tor exit node was found to be patching binaries during download, then remember that your ISP may or may not have the slightest morals, and that they are in complete control of your internet connection.

Improve this answer
6
  • Your complain on ISP is on the fact that they can change my connection speed and priority and other things that can corrupt files?
    – maxpesa
    Feb 24, 2015 at 19:08
  • @maxpesa - They can modify the stream if they want. The file won't be corrupt just modified.
    – Ramhound
    Feb 24, 2015 at 19:29
  • @maxpesa - I was pointing out that your ISP is in the same position that the Tor exit node in the linked article was - it has the ability to control what comes over your wires. If they wanted to, they are in position to modify the binaries you download on-the-fly.
    – Michael Kohne
    Feb 24, 2015 at 19:32
  • 2
    If the attacker can modify the binary, the attacker can most likley also modify the hash. That would require a more targeted attack than patching all binaries though, but is still very possible. While checking the hash can avoid damage, there is no guarantee. If you care about security, the first step is to only download over HTTPS - especially when you are using proxies like with tor!
    – kapex
    Feb 24, 2015 at 19:45
  • 1
    To add to @kapep's comment, it's even better to validate using cryptographical signatures (using keys distributed through a secure channel). That way an attacker would have to compromise both the keystore and the repository in order to alter the binary.
    – Johnny
    Feb 24, 2015 at 19:57
4

For files where integrity is critical, yes

I often find it necessary when I am doing something where it is critical that the file has a high integrity.

One example is flashing a router with OpenWRT. If the file is corrupt, then that router would be bricked, and then I would either have to either:

  • Replace it (expensive)
  • Fix it (time consuming. Especially if I need to solder a serial cable/JTAG)

Both of these are inconvenient, compared to the simplicity of checking a hash. Therefore I would strongly recommend doing it for critical files.

Improve this answer
0

I usually only bother to check if my download was interrupted and I resumed it later, because HTTP requests with a seek offset are not as widely used, so something silly might have happened.

In many cases, the download is a compressed file that won't decompress if it's broken. If that's NOT the case, checking is not a bad idea. I might check an ISO before burning it to write-once media.

Verifying it with a hash from the same site I got it from is of little use, security-wise, as other answers and comments have said. It can be more useful if you downloaded from a mirror, but the hash is from the original upstream. Or if the filename is ambiguous, and for some reason you aren't sure you got the exact version you wanted.

The point is, publishing hashes is useful for lots of special cases other than just downloading the file from the same site hosting the hash.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.