Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Possible Duplicate:
Duplicate file finder

Is there a tool that can identify/manage duplicates at the directory level? I've tried using duplicate file removal software to weed out duplicates, but these applications are too granular. I am looking to manage my duplicates at the folder level. Most of my duplicate files are also in (nearly) duplicate folder structures, so I want to to be able to identify the root (nearly) duplicate folder and delete/merge/update the root folder, instead of managing duplicates on a file-by-file basis.

One of the ways these duplicate folders was created was because whenever I upgraded to a larger harddrive, I backed up the old harddrive contents to a folder on the new harddrive. I have several directories filled with duplicate files and directories. Some directories are super-sets of other directories (contain the same files/directories plus additional files) Others contain a common subset of files/directories, but each contain additional files.

share|improve this question

marked as duplicate by studiohack Feb 10 '12 at 23:30

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

9 Answers 9

I second the WinMerge suggestion, especially if you are uncomfortable with the command line.

WinMerge is not totally automated, but you can delete whole sets of duplicate folders at once by control-clicking on them.

You can limit your search to only folders by changing the File -> Open... dialog box Filter: field from "." to "*".

You can control-select all the duplicate folders and then delete the ones from the right or left side, or both.

-steve

share|improve this answer

If you have access to the (excellent) Beyond Compare, check out the Folder Sync feature.

alt text

share|improve this answer

i wonder if maybe this would help you, its called winmerge. seems like you can set it up to compare 2 folders and take an action on the folder depending on the results of the comparison.

http://lifehacker.com/software/geek-to-live/compare-and-merge-files-and-folders-with-winmerge-290657.php

share|improve this answer
    
This could be very helpful, but I am looking for a tool that also finds similar/duplicate folders (I may not realize where all the dupes are located, beforehand) –  Leftium Aug 25 '09 at 15:56

One command line tool I've used and like very much because duplicates can be turned into hard links is FINDDUPE by Matthias Wandel. http://www.sentex.net/~mwandel/finddupe/

Hardlinks make duplicates appears as if the file still exists in two places, but only one copy exists on the disk. But not that this only works on NTFS file systems, and you must run it with administrator privileges.

Bets of all, it's totally free. :)

share|improve this answer

If you got suspected folders, you can use a trick.

I'm using the well known xcopy trick to copy everything in the suspected folder to a file. Then, I compare these two files to see if they're intact.

share|improve this answer

I'd recommend checking the site commandlinefu. Your on windows so you'd have to use cygwin.

I found this entry at commandlinefu.com that might be usefull

find . -type f -print0|xargs -0 md5sum|sort|perl -ne 'chomp;$ph=$h;($h,$f)=split(/\s+/,$_,2);print "$f"."\x00" if ($h eq $ph)'|xargs -0 rm -v --
share|improve this answer
    
I prefer UnxUtils and/or MSys instead of Cygwin. I've always found Cygwin to be very slow, due to its emulation layer. –  davr Aug 27 '09 at 16:26

It's hard to find software with the right level of grandularity that does what you want. I'd recommend learning a scripting language ( try MS Shell + VBScript , perl, python ) and write a little command line utility to do this yourself. You can make it as grandular as you like.

Whoever wrote w/e software is suggested above, probably did not have the exact same idea as you, so you will probably have to do some manual looking/deleting.

share|improve this answer

Have you tried AusLogics? They currently supply some free Duplicate File Finding software, here's a link:

http://www.auslogics.com/en/software/duplicate-file-finder/download

Hope this helps :)

share|improve this answer

I'm not sure what you mean by "granular" when referring to other duplicate removal software, but after trying a couple dozen software programs that do this, I have found that Glary Utilities suits my needs perfectly. It is very fast searching a folder or even a drive or comparing one folder to another for duplicates based on exact size, or time stamp, and or name. You can choose which file extensions to filter in or out of the search. And you can edit the results manually, or let the program delete one of the duplicates in every group. Might I add best of all, it's free and very very fast.

share|improve this answer
    
Thanks for the suggestion, but this one is too "granular." I am looking to manage my duplicates at the folder level. Most of my duplicate files are also in (nearly) duplicate folder structures, so I want to to be able to identify the root (nearly) duplicate folder and update/delete that one. –  Leftium Aug 20 '09 at 21:44

protected by studiohack Mar 27 '11 at 7:46

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.