Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Just bought a NAS. So I thought I'd get all our photos, media files and pdfs consolidated, de-duplicated, de-junked and virus-checked and stick them all on it.

We have 3 laptops, one running Windows, the others OSX. We have a file server running Windows - it was the result of an earlier attempt at a networked fileserver - and a Mac Pro that is also kind of a server (previous attempts at this job have resulted in most of our stuff being on it). Also memory cards/sticks, cd backups and so on.

I would be grateful if anyone could suggest a strategy or, ideally, tool(s) I could use to solve this problem. It is probably no more than one or two terabytes of data in total, but I can imagine that going through it all manually, file-by-file may well drive me insane.

share|improve this question
1  
You might have to resort to a manual process. I've done deduplication from multiple sources, and frankly, the error margin is too high to let software do it for you, unless it's a very narrow field (e.g. large video files). – user3463 Jul 22 '12 at 5:12

its a manual process. there are limits to what things like image magic can do from scripts to try and weed out duplicates for images/videos. if just by name is reliable then you could use a copy software to merge them all. as for duplicate pdf's dont know of a software that would read through it and see if they are duplicates.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .