Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm moving large amounts of data from one external drive to another larger one. The files are important and the smaller drives need to be cleared and reused (HD camera). Is there some utility for moving files and verifying their integrity?

I've been using this command

find . -type f -exec md5 '{}' \; > md5list.txt

in the terminal to create a list of MD5s for each file then using diff to compare the two. However, I am moving 320GB at a time, which takes a while by itself. Computing the checksums takes another hour or so. It would be much more efficient to do this on the fly, during the copy. I'm just hoping someone has already written the software...

share|improve this question

3 Answers 3

up vote 1 down vote accepted

rsync

Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking its whole-file checksum...

share|improve this answer
    
I'd like a GUI for this (so I can delegate). I see there are multiple ones. Which is the best? –  jedavis Jan 8 '11 at 19:18
    
Not a clue. I don't actually run OS X (and I've never used a GUI for rsync). –  Ignacio Vazquez-Abrams Jan 9 '11 at 0:29
    
This is not correct or rather taken out of context. Rsync only verifies that the transfer was done correctly. In the case of a local copy there is no transfer over the network, so the actual file-writes are not verified. –  user329023 Jun 3 at 15:40

We use ShotPut Pro for this exact task - it copies media from SxS (and other) HD camera media, puts it into more than one place at a time if you want, and verifies the data.

It's made the process of keeping on top of footage so much easier for us.

If it's for cameras, then this is the best bet I think. If it's for copying other data, I'm not sure (I found this question while trying to find a decent filemanager for OS X!)

ShotPut Pro™ for Macintosh is an automated copy utility application for HD video and photo files. ShotPut Pro is the industry de-facto standard offloading application for professionals. The simple user interface and robust copy speeds make it indispensable for todays tapeless HD workflows.

Security Verify file-by-file sizes, MD5 or CRC checksums, or byte-to-byte comparisons of your media copies to ensure they exactly match the originals.

enter image description here

share|improve this answer

Rsync only verifies that the transfer was done correctly. In the case of a local copy there is no transfer over the network, so nothing is verified. The actual file-writes would not be considerer transfer. The other poster noted this when he compared the times vs using cp. You must re-read the file from the disk. The fastest way to do this is CRC (less CPU intensive).

crc32 * > ~/Desktop/crc1.txt
crc32 * > ~/Desktop/crc2.txt

then run crc32 on those files and you will see if the copy was exact.

See the quote from the other answer at http://manpages.ubuntu.com/manpages/lucid/man1/rsync.1.html.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.