Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm looking for a tool to maintain the integrity of a filesystem and it's contents using checksums. Effectively storing a list of checksums/filename pairs somewhere on the filesystem in a way that can be verified later if files are somehow damaged or lost.

Git does what I want, but because it stores the contents of every file in it's object database, the disk usage will at least double. And the fact that it does not provide a progress bar when scanning files tells me it was not designed for the multi-terabyte filesystem I have in mind.

I can do this crudely by storing the output of md5deep, but is there a tool specifically designed for this purpose, using whatever smarts possible to make the process efficient?

share|improve this question
    
what filesystems/OS? –  Sathya Aug 27 '11 at 14:13
    
NTFS, Mac OSX . –  Jesse Aug 27 '11 at 14:26

1 Answer 1

up vote 2 down vote accepted

md5deep and related tools are in fact designed to verify file integrity. If MD5/SHA hash algorithms are too slow, it's because they were designed for cryptographic security first – if that is not required, CRC32 checksums are the next best choice. (For example, the Btrfs filesystem uses CRC32 for data integrity.) Search for "SFV" utilities (iSFV, MacSFV) that will automate the checking.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.