6

I have rsync'd a directory from a Mac to a linux box containing files and folders with special characters (umlauts). Unfortunately, the Mac uses NFD encoding (thanks, Apple!) which rsync didn't convert to the NFC standard file name encoding. rsync lists those files as e.g. Einversta?\#210ndniserkla?\#210rung.pdf what should be Einverständniserklärung.pdf during transfer. On the linux box, that leads to duplicate files, e.g. the folder containing two files of the same name Einverständniserklärung.pdf (one in NFD and one in NFC encoding).

How can I find all files in NFD-encoding and delete them? This must be possible, e.g. by using find.

1 Answer 1

7

There are two ways:

  • As asked: Try to convert the file name to NFD, and see if you get the same output as input. If you do, the file name is already NFD.

  • More useful: Try to convert the file name to NFC, and see if you get the same result. If you don't, the file name is either NFD or NFKC or NFKD or something mixed/malformed, but still not quite NFC.

To find and rename affected files, use convmv:

convmv -f utf-8 -t utf-8 --nfc ...

For something more easily scriptable you can use Perl's Unicode::Normalize:

find -type f | perl -C -MUnicode::Normalize -n -e'print if $_ eq NFD($_)'

To find all non-NFC names (including NFD and not-normalized):

find -type f | perl -C -MUnicode::Normalize -n -e'print if $_ ne NFC($_)'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .