I have two directories that I would like to compare. The first directory contains an rsync backup of many thousands of folders and files. The second directory contains hard links to all of the files in the first, plus a few additional folders and files. I would like to be able to compare the directories in such a way that the hardlinked files are skipped or at least quickly decided to be the same file and only those files that are not hardlinks are further compared. Is this something diff can do, or is there a better way?
3 Answers
cmpdir is for this problem (Disclaimer - I wrote it).
It only compares inodes. Files that only exist in the first directory are marked as "Removed", whereas files that only exist in the second directory as "New". It does not compare the file content though.
Probably not fully what you mean, but if you want to skip files that are in both directories, you can use find with the -links
option.
find ./ -links 1
shows only files that have one hard link (and cannot therefore be present in both directories).
You can show inodes of your files with
ls -i -1
(-1 / "one" writes one line per file to output).Save the output for each of the two folders.
If the filename of "original" and hardlinked file differ, use
cut -f1
to filter the filenames out and retain only the inode numbers.Sort the files for inode number using
sort
. Now you can usediff
to view the differences.Use
grep "^<" diffOutput.list
to filter for lines starting with<
=> Inodes occurring only in first folder.grep "^>"
analogous for Folder B. Usecut -c2-
to delete the first character (<
or>
) from the output if you need the inode number.
If you previously have removed the filenames from the files, you have to search the filenames for a given inode number.
Save the output of ls -l -1
(with filenames) for each folder into one file. Now you can use grep
to get the filenames for given inode numbers in a bash loop:
Let InodesFilenamesA.list
be the file containing inode numbers and filenames for Folder A
12345 file1inBothFolders
99999 file2OnlyInFolderA
88888 file3OnlyInFolderA
...
Let inodesA.list
be a file containing Inodes occurring only in Folder A
99999
88888
....
Now loop over inodesA.list
.
cat inodesA.list | while read line; do
grep "$line" InodesFilenamesA.list
done
You get a list of folder