Deduplication is the finding and removal of duplicate files.

learn more… | top users | synonyms (1)

1
vote
2answers
131 views

Find unique files across 2 drives

I'm trying to compare 2 volumes, which mainly have the same content, but in completely different folder structures. (Around 2 million files.) I want to see which files are ONLY in 1 folder, not the ...
1
vote
2answers
599 views

Windows 7 file-based backup service

I'm looking for a good replacement for Lazy Mirror, since it doesn't support Windows 7 well. Pros: One of the things I really loved about Lazy Mirror is that it always maintains a "full" backup, but ...
0
votes
2answers
73 views

Duplicate Deletion Script does nothing

An associate of mine wrote a script for me that should theoretically delete files that have the same name, regardless of their file type. Specifically, it checks mp3, m4a, wma, wav files. However, ...
0
votes
2answers
3k views

How to Enable Data Deduplication in Windows 8.1 client

How can i enable Data Deduplication in Windows 8.1 x64 client. http://weikingteh.wordpress.com/2013/01/15/how-to-enable-data-deduplication-in-windows-8/ The above link discusses the same for a ...
2
votes
1answer
384 views

Deduplicate files using the command line in OSX Lion

I am using OSX 10.8.4 and would like to remove duplicate files in a single flat directory if an md5 hash matches. I have looked up several answers but none of them work for me, I think because of ...
2
votes
1answer
274 views

Deduplicate Files inside multiple .tar.gz archives

I take a nightly backup from my Webhosting service as .tar.gz. The Data changed is very little each night (<10MB). I would like to archive these files and deduplicate (solid compression) files ...
1
vote
1answer
257 views

What is the best way to merge two sets of files, but for duplicates to select the older file in Windows?

I have stumbled into a dumb usage case. I want to merge two folders in a particular way (I guess these are called folders now, I still think of them as directories, but whatever). The folders have ...
0
votes
1answer
45 views

Deduplicating Multiple File Types Across Multiple Sources

We are faced with a situation where data has been backed-up to several external mediums and we are undergoing an exercise to consolidate the data. The data is comprised of binary files, audio, video, ...
0
votes
1answer
292 views

avoid duplicate files during backup

I'm backing up 2 drives to an external backup drive. The 2 source drives have a lot of duplicate files (name/size) between them (for work). One of the source drives is already backed up to the ...
0
votes
1answer
1k views

WMP 12.0 adds duplicate entries for same file mapped network drive

Windows Media Player 12 is adding duplicate entries for each file on my mapped network drive. When I try and rebuild the index by going to %UserProfile%\Appdata\Local\Microsoft\Windows Media, I am ...
3
votes
0answers
232 views

Rsync, how can I delete remote duplicates?

I'm trying to delete remote duplicate files without transferring the non-existing files, and without updating the existing files. If I specify both --existing and --ignore-existing (along with -av ...
1
vote
0answers
181 views

OSX MAIL APP: how can i merge several 'archived' MAIL folders into one single new (filesystem) folder

over the last 5 years i have accumlated several partial MAIL folders scaterred across several different volumes (representing 10GB++ of mail that i really dont want to have to download all over ...
1
vote
0answers
373 views

How to deduplicate files when storing in the SDFS file system using OpenDedup

I followed the link, http://www.opendedup.org/quickstart for installing OpenDedup in my linux machine (Ubuntu 12.04 - 64 bit). I created a SDFS volume and mounted it. I copied two similar Ubuntu VMs, ...
1
vote
0answers
213 views

Duplicate Images / Tags - sorting and removing

I have folders full of camera images that I have copied and backed up between my laptop, computer and external hard drive(s). I want to organise and sort them, and remove duplicates, so I have used a ...
0
votes
0answers
16 views

Is there a diff-like tool that searches the directory for existence of the file or folder?

In an attempt to de-duplicate the data on my system, I've used tools such as Gemini for finding duplicate files across directories and diff tools like Kaleidescope for understanding how the ...
0
votes
0answers
80 views

Deduplication vs Incremental Backups

I need good arguments for that question(s): Why would you take a daily full backup and deduplicate the volume, instead of taking a single full backup and afterwards continuing with incremental ...
0
votes
0answers
27 views

mv backup but instead of creating tilde directories, create tilde files in subdirectories?

I have 2 deep directory structures that are (probably) identical and I want to end up with a single one, but not auto-resolve existing conflicts (I'm guessing rsync would clobber something). I just ...
0
votes
0answers
39 views

Copy files to location, skipping duplicates (in source and dest. location)

I have an external hardrive I think is running down breaking (unable to delete some files from it). I copied a bunch of files over already but ran out of space. How do I dedup while copying/How to I ...
0
votes
0answers
72 views

How to stop Deduplication in Windows 2012 R2

Deduplication has been enabled on my server and I would like to stop it, since I don't need it to run. I have disabled the deduplication on the drive, but the deduplication process continues. How can ...
0
votes
0answers
101 views

command line extracting especified file from 7z file

I would like to extract a specific file from a 7z file, but that filename exist twice on different directories. How can I choose one of them? I tried using compressed path but I received an error ...
0
votes
0answers
93 views

Copy only files that don't exist regardless of file name?

My dropbox account apparently renames all auto uploaded images. I'm about to clear my phone and I would like to verify that all images are in the dropbox folder before deleting them. If they had their ...