I need to remove the duplicates in my computer, since i have around 5 copies of every single photo, in different folders. I want to delete the photos that also appear in a certain folder, in witch i sorted a bunch of them. I want it to keep the files in the folder, and also keep duplicates that do not also appear in that folder Ex: i want to keep the files on the desktop.

  • C:/Desktop/file1,file3
  • C:/Desktop/2016/file2
  • C:/Backup/file1(2),file4
  • C:/Backup2/file1(3),file2(2),file4(2)

The result should be

  • C:/Desktop/file1,file3
  • C:/Desktop/2016/file2
  • C:/Backup/file4
  • C:/Backup2/file4(2)
  • What have you already tried? – slhck Aug 8 '17 at 7:55
  • CCleaner's duplicate finder – cezieu Aug 8 '17 at 8:16

Assuming dir1 to be the path you want to search and dir2 to be the certain folder:

find dir1 -not -path "dir2/*" | grep -F -f <(ls -1 dir2) | parallel rm

or, if you only have xargs and not parallel:

find dir1 -not -path "dir2/*" | grep -F -f <(ls -1 dir2) | xargs rm


Let's say you want to look in the C: drive and the certain folder is C:\Desktop, then on cygwin:

find /cygdrive/c -not -path "/cygdrive/c/Desktop/*" | grep -F -f <(ls -1 /cygdrive/c/Desktop) | parallel rm

on Git-for-Windows:

find /c -not -path "/c/Desktop/*" | grep -F -f <(ls -1 /c/Desktop) | parallel rm

on WSL:

find /mnt/c -not -path "/mnt/c/Desktop/*" | grep -F -f <(ls -1 /mnt/c/Desktop) | parallel rm
  • 1
    Did not work as expected... (i am probably at fault here) . I used cygwin and $ find /cygdrive/c/dedup -not -path "/cygdrive/c/dedup/test/*" -delete | grep -F -f <(ls -1 /cygdrive/c/dedup/test) and it deleted all files except the ones in /test(the "certain folder") – cezieu Aug 8 '17 at 14:29
  • No, I made a mistake in placing the -delete option before grep: this way, all found files are deleted before the filter is applied! It should have been find dir1 -not -path "dir2/*" | grep -F -f <(ls -1 dir2) | parallel rm {}. – simlev Aug 8 '17 at 14:44
  • oh, now testing it – cezieu Aug 8 '17 at 14:45
  • Had quite some trouble installing parallel and whatever else it required, but now it works ;3! Thanks a lot ! – cezieu Aug 8 '17 at 15:24
  • @cezieu Sorry for posting a dangerous answer in the first place, good thing you tested it out. parallel does the same as xargs, which is usually available by default, so you should not have bothered. – simlev Aug 8 '17 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.