I have two directories that I would like to compare. The first directory contains an rsync backup of many thousands of folders and files. The second directory contains hard links to all of the files in the first, plus a few additional folders and files. I would like to be able to compare the directories in such a way that the hardlinked files are skipped or at least quickly decided to be the same file and only those files that are not hardlinks are further compared. Is this something diff can do, or is there a better way?
cmpdir is for this problem (Disclaimer - I wrote it).
It only compares inodes. Files that only exist in the first directory are marked as "Removed", whereas files that only exist in the second directory as "New". It does not compare the file content though.
You can show inodes of your files with
ls -i -1(-1 / "one" writes one line per file to output).
Save the output for each of the two folders.
If the filename of "original" and hardlinked file differ, use
cut -f1to filter the filenames out and retain only the inode numbers.
Sort the files for inode number using
sort. Now you can use
diffto view the differences.
grep "^<" diffOutput.listto filter for lines starting with
<=> Inodes occurring only in first folder.
grep "^>"analogous for Folder B. Use
cut -c2-to delete the first character (
>) from the output if you need the inode number.
If you previously have removed the filenames from the files, you have to search the filenames for a given inode number.
Save the output of
ls -l -1 (with filenames) for each folder into one file. Now you can use
grep to get the filenames for given inode numbers in a bash loop:
InodesFilenamesA.list be the file containing inode numbers and filenames for Folder A
12345 file1inBothFolders 99999 file2OnlyInFolderA 88888 file3OnlyInFolderA ...
inodesA.list be a file containing Inodes occurring only in Folder A
99999 88888 ....
Now loop over
cat inodesA.list | while read line; do grep "$line" InodesFilenamesA.list done
You get a list of folder