Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm interested in storing an indicator of file / directory integrity between two archived copies of directories. It's around 1TB of data stored recursively on hard drives. Is there a way using OpenSSL to generate a single hash for all the files that can be used as a comparison between two copies of the data, or at a later point to verify the data has not changed?

share|improve this question

4 Answers 4

up vote 2 down vote accepted

You could recursively generate all the hashes, concatenate the hashes into a single file, then generate a hash of that file.

share|improve this answer
    
For a single command, something like md5 -q <(find . -type f 2>/dev/null | xargs md5 -q | sort) works well in Bash and doesn't require a temp file. Alter if your system uses md5sum instead of md5. Also be aware that sort can behave differently on different platforms which will affect the final checksum if the order is different. Add flags like ! -name ".DS_Store" to the find component to ignore certain files, like the .DS_Store files on Mac OS X that can throw off the checksum since they're generated by the OS. –  Justin Mrkva Mar 2 at 22:37

You can't do a cumulative hash of them all to make a single hash, but you can compress them first then compute the hash:

$tar -czpf archive1.tar.gz folder1/
$tar -czpf archive2.tar.gz folder2/
$openssl md5 archive1.tar.gz archive2.tar.gz


to recursively hash each file:

$find . -type f -exec openssl md5 {} +
share|improve this answer
1  
1TB of data - no room to tar them. Is there a way to recursively generate hashes of all files? –  Kieveli Nov 19 '09 at 19:00
    
yes, added it to my answer. –  John T Nov 19 '09 at 19:54
    
nice tar idea, but not always applicable. the 'find' method is better in general. if there is 'no room' for the tarball: % tar -cf - folder | openssl md5 –  akira Nov 20 '09 at 7:22

Doing a md5 sum on the tar would never work unless all of the metadata (creation date, etc.) was identical as well, because tar stores that as part of its archive.

I would probably do an md5 sum of the contents of all of the files:

find folder1 -type f | sort | tr '\n' '\0' | xargs -0 cat | openssl md5
find folder2 -type f | sort | tr '\n' '\0' | xargs -0 cat | openssl md5
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.