As we know, Linux and Unix style filesystems store file name as byte stream, as apposed to Unicode streams on Windows NTFS/FAT.

So there is the question. If I get an old hard disk(from around 2002) with non-ASCII filenames encoded as "gbk" on its Ext3 partitions, and I want to attach that old disk to my modern Linux(utf8-encoded filenames) so that I can copy the old files out, preserving filename character meaning .

Then what's the best practice to do that?

Explaination from a programmers perspective is specially welcome.

  • I have an old laptop that I don't use anymore, but it has Linux and Windows XP and when Linux is running it can read the Windows partition. Why not just plug it in and see if it works? – Marichyasana Apr 27 '13 at 1:40
  • Thank you. I believe it can, because Windows explicitly/definitly stores Unicode character on FS so there is no ambiguation. I just want to know how guys in pure Linux world can cope with that problem. – Jimm Chen Apr 27 '13 at 2:56

No. Most Linux programs don't care about disk or mountpoint boundaries, and use the same encoding for all file names everywhere.

However, most utilities still treat filenames as byte strings, so there should be no problem with copying or using files named in weird encodings. (For example, I renamed a few my files to ISO-8859-13 and I can still open them in command-line or GNOME programs, although KDE has troubles.)

You can use tools such as convmv to rename all files to UTF-8 names – either in your copy, or directly on the original disk:

convmv -f gbk -t utf-8 -r /mnt/old-disk

(On Linux, it is possible to write overlay filesystems using FUSE, which could present the files from a given path using a different name encoding. I don't know if such a filesystem exists, though. bindfs has a similar purpose, but it only translates file ownership/permissions. Either way, I'm sticking with my recommendation to just convert all names to UTF-8.)

| improve this answer | |
  • Thank you for referring to convmv, and I find it's a perl script. After some trying, I'm starting to worry about its reliability. convmv's processing seems to rename on a file basis, instead of on a directory basis. So, when some gbk file name sequence happens to be UTF8 sequence( touch $(echo '0xE7 0x94 0xB5 0xE8 0x84 0x91'|xxd -r).txt ), it says Skipping, already UTF-8. – Jimm Chen Apr 28 '13 at 1:39
  • Use the --nosmart option. (The tool is most commonly used for fixing ISO-8859-𝒏 filenames, which don't have this problem.) – user1686 Apr 28 '13 at 8:31
  • Thank you. --nosmart does what I mean. But I'm not very sure whether convmv does it on a directory basis or a file basis. The man page seems not to be explicit on this. I can't figure it out myself due to not knowing Perl yet. – Jimm Chen May 10 '13 at 8:27
  • Sadly this doesn't work for me. When I run: convmv -f ISO-8859-1 -t utf8 Rkurr It says: Skipping, already UTF-8: ./Rökkurró Adding "--nosmart" makes it say: mv "./Rökkurró" "./Rökkurró" I think it should be "Rökkurró". I don't know why the filename is broken... the file was from windows. – Peter Jul 11 '14 at 9:35
  • 1
    @Peter: In your case, the name is UTF-8, but encoded twice (a program mistakenly thinking that the input isn't UTF-8 yet). Therefore you must convert it in the opposite direction. Try convmv -f utf-8 -t iso8859-1. – user1686 Jul 11 '14 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.