0

My uncle is a photographer with lots and lots of photoshoots. He stored (duplicates of) backups across a variety of external hard drives. 90% of all backups are the same, but some have had some edits, such as added files or photoshopped images. Now I got him a NAS with RAID setup and an off-site NAS for additional backup of the main machine, to store all his data on one single device. He asked me to deduplicate all backups and only keep the latest version, deleting all copies.

There are already a few applications (sure, recommend one if you know one (Mac/Windows) to identify copy images when file size & file name are equal. But I was wondering..


TL;DR
What are the odds of an image to keep the same file size when it's photoshopped?

//

Alternatively: What's the best way to identify perfect duplicates of images?
In the case my previous work method (filename/size) isn't the best way to go at it.


I had the assumption that if the odds are very low (<1%) I could simply search & destroy and keep 1 of every file of which both file name and file size is equal.

1 Answer 1

1

You can compute checksums on originals/copies and compare to see if anything has been changed.

This link explains it a bit more and offers some tools to help.

3
  • Before actually checking your link: How easily is that, 200 images at a time, with a total of about 100.000 images? Update: I read the page. Checksums are a great way to verify differences. Remaining question: How easily is that per batch Mar 23, 2016 at 15:41
  • I'm sure you could script it, if you really wanted to. However there is a superuser question which may help - superuser.com/questions/137155/…
    – Hefewe1zen
    Mar 23, 2016 at 15:44
  • the thing is, most backup-oriented programs (such as the windows built-in robocopy) already use checksums to determine if files are changed. I think the real problem is a reliable dating of files, since checksums will not tell you which one is the original file and which one is the altered file. (edit: robocopy actually may not, but most still do!)
    – Yorik
    Mar 23, 2016 at 16:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .