Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Is there any GUI application (either for Windows or Linux) that implements fuzzy hashing algorithms like ssdeep or sdhash to find near duplicate files (mostly text files)?

share|improve this question
add comment

3 Answers 3

You can try Near Duplicates Finder. It is not exactly what you are asking for, but it is Java based, so it will work on any platform where Java works (including Windows, OS X and Linux). Is has no GUI interface, but it will find near duplicates for text based files, like TXT, DOC, HTML, PDF, etc.

Example Usage

Near Duplicates Finder v.0.1.0
Usage: java com.softcorporation.neardup.DuplicatesFinder parameters ...

Parameters format: -parameter [value]
    -start filename[,filename]  directory / file(s) to search for duplicates (mandatory)
    -report filename            report file (by default report goes to ./report.log file)
    -score                      the score to report the duplicate (default is 0.6)
    -onlynew                    find the duplicates only for new documents
    -gram                       number of words in a phrase
    -purge                      clear files list from past runs
    -db                         location of db directory with files list
    -delete criteria            remove duplicates by criteria (old, new, small, large)
    -deletepath pattern         remove only matching pattern files (mandatory for delete)
    -verbose                    display progress information (on standard output)

Example: Find duplicates from text files in directory 'docs' and save report in 'report.log'
    java com.softcorporation.neardup.DuplicatesFinder -start docs -report report.log

For more information visit web site: http://www.softcorporation.com/products/neardup
share|improve this answer
add comment

I've found SSDeepFE for ssdeep.

Also, sdhash seems to have a web-based GUI but it isn't very useful yet.

share|improve this answer
add comment

I think this is the duplicate file finder that you are looking for. There is similarity mode which works very nice on source code files too (text files), but it is available in the commercial PRO version.

share|improve this answer
    
Have you tried it? –  Alix Axel May 22 '13 at 23:45
    
Yes I have. I have already shared that it works very nice on source files (text). The similarity mode does not use the so called FUZZY search, but rather than that it performs some deep analysis and can find similar documents even if entire paragraphs are swapped. This tool worked the best for me, but the similarity mode is slower and should be used on a smaller subset of files. The similarity mode works in the FREE version with some limitations:link –  Jack Dorsey May 23 '13 at 11:24
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.