How can I copy files to an external drive and verify their integrity in OS X?

Question

I'm moving large amounts of data from one external drive to another larger one. The files are important and the smaller drives need to be cleared and reused (HD camera). Is there some utility for moving files and verifying their integrity?

I've been using this command

find . -type f -exec md5 '{}' \; > md5list.txt

in the terminal to create a list of MD5s for each file then using diff to compare the two. However, I am moving 320GB at a time, which takes a while by itself. Computing the checksums takes another hour or so. It would be much more efficient to do this on the fly, during the copy. I'm just hoping someone has already written the software...

Answer 1

rsync

Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking its whole-file checksum...

Answer 2

We use ShotPut Pro for this exact task - it copies media from SxS (and other) HD camera media, puts it into more than one place at a time if you want, and verifies the data.

It's made the process of keeping on top of footage so much easier for us.

If it's for cameras, then this is the best bet I think. If it's for copying other data, I'm not sure (I found this question while trying to find a decent filemanager for OS X!)

ShotPut Pro™ for Macintosh is an automated copy utility application for HD video and photo files. ShotPut Pro is the industry de-facto standard offloading application for professionals. The simple user interface and robust copy speeds make it indispensable for todays tapeless HD workflows.

Security Verify file-by-file sizes, MD5 or CRC checksums, or byte-to-byte comparisons of your media copies to ensure they exactly match the originals.