File inconsistency.. doesn't appear in 'ls -a' but it exists!

Question

I'm really frustrated with this..

I run rsync 3.0.6 on OSX Snow Leopard to backup a folder form an external hdd (FAT32) to another external hdd (FAT32).

Somehow certain files (and at times they change!) get copied with a different file name. For example:

campo_2006_valcanale_127.jpg --> CAMPO~40.JPG

shrunk and uppercased.. of course rsync next time deletes the CAMPO~40.JPG and copies it again, but almost always it gets messed up again.. at times it works, but also files that were ok get corrupted..

Now. If I go in the destination directory and do a 'ls -a' I can't see the file campo_2006_valcanale_127.jpg because it has been somehow renamed. BUT THE FILE campo_2006_valcanale_127.jpg actually exists!! I can display it, I can write on it etc etc.. simply it's invisible! Maybe it's just the filename that's behaving oddly..

IMPORTANT: same behaviour when I copy files manually ('cp') and even when I copy them with the Finder! I drag and drop a file and its name changes!!

If you have ANY hint, please tell me.. I can provide more info if need be Thanks

Answer 1

The shortened filenames are artifact of FAT32; any file with a name longer than the old 8.3 limits gets given a shortened name, as you've seen. Use a proper filesystem, one that handles long filenames natively instead of using a bizarre hack, and you won't have this problem (eg, NTFS, HFS, ext - all handle long filenames natively). Wikipedia has more detail on the Long Filename hack that FAT32 uses

FAT32 also has other issues that make it hard for rsync to work - eg, it doesn't store timestamps with any great precision, so rsync can't use its normal heuristics.

The rsync FAQ has an entry on rsync copying the same files many times. In short, it's probably the timestamp that's causing most of the re-syncing; modify-window=1 or maybe modify-window=2 should fix this.