Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I tested the speed of md5sum on a few Ubuntu 8.04 servers

  • Pentium III 700 MHz: 52 MB/s
  • Atom 1.6 GHz, 32-bit: 119 MB/s
  • Core 2 (Yorkfield) 2.5GHz, 32-bit: 194 MB/s
  • Core 2 (Yorkfield) 2.5GHz, 64-bit: 222 MB/s

Then I downloaded a tool (by apt-get install) called md5deep and found that it's roughly 20% faster (as tested on the 32-bit Core 2 server). This makes me feel that the "vanilla" md5sum included in Ubuntu isn't the fastest one.

Questions:

  1. Other than md5deep, are you aware of any MD5 calculators that are potentially faster than md5sum? (Answers for software from other OS are also welcomed.)
  2. If I want to compile md5sum myself, what compiler options would you suggest for the Core 2 server? (note: gcc 4.2.4 in Ubuntu 8.04 does not seem to support -march=core2)

Edit: I downloaded and compiled coreutils-8.4 on the 64-bit Core 2 server and the speed increased to 320 MB/s... Unbelievable!

FYI, the one included in Ubuntu 8.04 is coreutils-6.10.

Is it safe to replace coreutils with a newer version?

share|improve this question
    
Usually md5sum is faster than my disk...eg I don't have a HDD that can read at 200MB/sec, so a faster than that md5sum doesn't help any. Do you have a really fast SSD RAID array? Or are you running md5sum on files stored on a ramdisk (tmpfs) or something? –  davr Apr 1 '10 at 22:26
    
At first I used a 1GB file of random bytes, loaded completely in the memory cache. Then I found that the md5sum speed is the same if I pipe null bytes into md5sum on the fly. I decided to use the latter method so that servers without 1GB RAM can also be tested. –  netvope Apr 1 '10 at 22:35
    
I'd be curious to know how OpenSSL's md5 implementation compares to the others you've benchmarked. –  Spiff Apr 2 '10 at 4:33
add comment

1 Answer 1

Just thought I'd chime in to add some updated speed benchmarks.

Using md5sum on Ubuntu 10.10, which uses coreutils 8.5:

  • Core 2 Quad (Kentsfield) 2.66GHz, 64-bit: 404 MB/s
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.