I have hundreds of thousands of files scattered across the hard drives of multiple systems, all running Linux, and I need to organize them (classify, dedupe, etc.). I have all sorts of files, audio, video, text, etc. Can anyone suggest a solution any better than "spend a few months writing the ideal program for this" and "just delete them all"?

| improve this question | | | | |
  • 5
    i am tempted to "close question: too broad" ... :) – akira Jul 4 '13 at 6:06
  • I've faced the deduping challenge myself in the past and developed some lines of bash scripting code. Shall I post it? – artistoex Jul 4 '13 at 6:12
  • @artistoex: Simple deduping is a solved problem. I'm looking for a larger solution, one that collects the statistics of files from multiple systems and allows me to perform various operations (move, delete, tag, etc.) on the data/files and then push the changes back out. – Ignacio Vazquez-Abrams Jul 4 '13 at 7:20
  • Do you know tool that can help you in the same situation on another operation system? Windows, Mac, etc...? What criteries for sorting ? – september Jul 4 '13 at 17:13
  • Could you please give a bit more detail on what your requirements are? As it is the question is either all-out off topic as a software request, or too vague as you can see from the one answer you have received. What do you mean exactly by "tagging"? Is this a tag that can be pushed to the OS, or is it specific to the software you are envisioning? Some kind of database of all your files? – terdon Jul 7 '13 at 1:50

What criteria for sorting ? Video/Audio/text... ? If yes, try something like:

~# find / -name "*.avi" -exec mv -i {} ~/video/ \;
~# find / -name "*.mkv" -exec mv -i {} ~/video/ \;
~# find / -name "*.wav" -exec mv -i {} ~/audio/ \;
~# find / -name "*.mp3" -exec mv -i {} ~/MP3/ \;
| improve this answer | | | | |
  • This is not a bad answer per se, but it my case it doesn't handle duplicate filenames and duplicate contents, nor multiple systems. – Ignacio Vazquez-Abrams Jul 4 '13 at 18:02
  • OK, added -i option for mv. You will be able to compare content before overwrite it. – september Jul 4 '13 at 18:37

Not the answer you're looking for? Browse other questions tagged or ask your own question.