I have a set of backups of mostly photos. The directory looks sort of like this:
backup/Day1/photos/1.jpg .../2.jpg backup/Day2/photos/2.jpg .../3.jpg .../4.jpg backup/DayN/photos/2.jpg .../3.jpg .../9.jpg
Files with same name are identical. There are many duplicates. Due to the way the backup system works, it's not possible to create incremental backup directly. I always get the entire dump each day.
If I want to create a compressed archive for a date range, say Day 5~9, what is the best tool/compression algorithm to do that， which does deduplication well? (I'm not counting on compressing the jpeg itself)