Deduplication is the finding and removal of duplicate files.

learn more… | top users | synonyms (1)

2
votes
2answers
330 views

Deduplicate Files inside multiple .tar.gz archives

I take a nightly backup from my Webhosting service as .tar.gz. The Data changed is very little each night (<10MB). I would like to archive these files and deduplicate (solid compression) files ...
1
vote
2answers
159 views

Find unique files across 2 drives

I'm trying to compare 2 volumes, which mainly have the same content, but in completely different folder structures. (Around 2 million files.) I want to see which files are ONLY in 1 folder, not the ...
1
vote
2answers
612 views

Windows 7 file-based backup service

I'm looking for a good replacement for Lazy Mirror, since it doesn't support Windows 7 well. Pros: One of the things I really loved about Lazy Mirror is that it always maintains a "full" backup, but ...
0
votes
2answers
76 views

Duplicate Deletion Script does nothing

An associate of mine wrote a script for me that should theoretically delete files that have the same name, regardless of their file type. Specifically, it checks mp3, m4a, wma, wav files. However, ...
0
votes
2answers
3k views

How to Enable Data Deduplication in Windows 8.1 client

How can i enable Data Deduplication in Windows 8.1 x64 client. http://weikingteh.wordpress.com/2013/01/15/how-to-enable-data-deduplication-in-windows-8/ The above link discusses the same for a ...
1
vote
1answer
36 views

In Windows, is there any software to prevent duplication of files in a folder?

Let's say I have two files c:\f1\data.txt and c:\f2\data.txt. If I give the command copy c:\f2\data.txt c:\f1\data\ I will receive a warning about the potential owerwriting. I will receive the ...
1
vote
1answer
325 views

What is the best way to merge two sets of files, but for duplicates to select the older file in Windows?

I have stumbled into a dumb usage case. I want to merge two folders in a particular way (I guess these are called folders now, I still think of them as directories, but whatever). The folders have ...
0
votes
1answer
63 views

Deduplicating Multiple File Types Across Multiple Sources

We are faced with a situation where data has been backed-up to several external mediums and we are undergoing an exercise to consolidate the data. The data is comprised of binary files, audio, video, ...
0
votes
1answer
394 views

avoid duplicate files during backup

I'm backing up 2 drives to an external backup drive. The 2 source drives have a lot of duplicate files (name/size) between them (for work). One of the source drives is already backed up to the ...
0
votes
1answer
1k views

WMP 12.0 adds duplicate entries for same file mapped network drive

Windows Media Player 12 is adding duplicate entries for each file on my mapped network drive. When I try and rebuild the index by going to %UserProfile%\Appdata\Local\Microsoft\Windows Media, I am ...
3
votes
0answers
254 views

Rsync, how can I delete remote duplicates?

I'm trying to delete remote duplicate files without transferring the non-existing files, and without updating the existing files. If I specify both --existing and --ignore-existing (along with -av ...
1
vote
0answers
401 views

How to deduplicate files when storing in the SDFS file system using OpenDedup

I followed the link, http://www.opendedup.org/quickstart for installing OpenDedup in my linux machine (Ubuntu 12.04 - 64 bit). I created a SDFS volume and mounted it. I copied two similar Ubuntu VMs, ...
1
vote
0answers
216 views

Duplicate Images / Tags - sorting and removing

I have folders full of camera images that I have copied and backed up between my laptop, computer and external hard drive(s). I want to organise and sort them, and remove duplicates, so I have used a ...
0
votes
0answers
18 views

Efficient removal of duplicate records across multiple Excel sheets

I have a workbook with 2 sheets. Sheet 1 has 3 columns (1A, 1B and 1C) and sheet 2 has 2 columns (2A and 2B). Now I need to delete rows from sheet 1 if 1A and 1B match any row in sheet 2 (i.e. 1A=2A ...
0
votes
0answers
18 views

Is it possible to deduplicate a mounted truecrypt volume using Windows 2012?

An external volume has been encrypted using Truecrypt. Is it possible to run deduplication using Windows 2012 once the volume has been mounted?
0
votes
0answers
22 views

Is there a diff-like tool that searches the directory for existence of the file or folder?

In an attempt to de-duplicate the data on my system, I've used tools such as Gemini for finding duplicate files across directories and diff tools like Kaleidescope for understanding how the ...
0
votes
0answers
177 views

Deduplication vs Incremental Backups

I need good arguments for that question(s): Why would you take a daily full backup and deduplicate the volume, instead of taking a single full backup and afterwards continuing with incremental ...
0
votes
0answers
37 views

mv backup but instead of creating tilde directories, create tilde files in subdirectories?

I have 2 deep directory structures that are (probably) identical and I want to end up with a single one, but not auto-resolve existing conflicts (I'm guessing rsync would clobber something). I just ...
0
votes
0answers
46 views

Copy files to location, skipping duplicates (in source and dest. location)

I have an external hardrive I think is running down breaking (unable to delete some files from it). I copied a bunch of files over already but ran out of space. How do I dedup while copying/How to I ...
0
votes
0answers
111 views

How to stop Deduplication in Windows 2012 R2

Deduplication has been enabled on my server and I would like to stop it, since I don't need it to run. I have disabled the deduplication on the drive, but the deduplication process continues. How can ...
0
votes
0answers
119 views

command line extracting especified file from 7z file

I would like to extract a specific file from a 7z file, but that filename exist twice on different directories. How can I choose one of them? I tried using compressed path but I received an error ...
0
votes
0answers
93 views

Copy only files that don't exist regardless of file name?

My dropbox account apparently renames all auto uploaded images. I'm about to clear my phone and I would like to verify that all images are in the dropbox folder before deleting them. If they had their ...