-1

I've used a file deduplicator on a large folder with hundreds of thousands of files. The deduplicator created hardlinks for all duplicate files that it detected.

If I was to attempt to merge some directories that contain some hardlinks to the same files and allowed overwriting of the targets will anything happen to the files concerned?

I don't want to accept what happens based on empirical evidence from the case of a test involving a few files. I want to know exactly the process involved to know if there is a risk of file corruption for various reasons like power interruption, or other unsolicited interruption.

2

rsync is a good tool you might want to use to recursively copy a directory to another. When rsync -a Dir1/ Dir2 encounters files with the same names, it unlinks* the file first, copies to a temporary file, and renames the file to the original name. If it is interrupted, you don't lose any files that you wouldn't have lost if the command has finished. Just run the same rsync command again again to finish it.

*Unlink means delete without touching the file content. Other hardlinks are unaffected.

If you use the --inplace argument, it can do bad things to your hardlinked files, as described in more details in man rsync. So don't use it.

I rsync copy over 400 GB in ~200000 files across several network-mounted SSH file systems with hardlink de-duplication into ext4 filesystem on Ubuntu operating system. I haven't lost a single file, even with network timeouts and other interruptions. No experience on Unix.

0

Of course there's a risk of file corruption for major (unspecified) file-system operations, on an unspecified file system, even without taking into account the gamut of possible hardware failures.

Ensure you have good backups.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.