Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

As we know, Linux and Unix style filesystems store file name as byte stream, as apposed to Unicode streams on Windows NTFS/FAT.

So there is the question. If I get an old hard disk(from around 2002) with non-ASCII filenames encoded as "gbk" on its Ext3 partitions, and I want to attach that old disk to my modern Linux(utf8-encoded filenames) so that I can copy the old files out, preserving filename character meaning .

Then what's the best practice to do that?

Explaination from a programmers perspective is specially welcome.

share|improve this question

migrated from stackoverflow.com Apr 27 '13 at 8:16

This question came from our site for professional and enthusiast programmers.

    
I have an old laptop that I don't use anymore, but it has Linux and Windows XP and when Linux is running it can read the Windows partition. Why not just plug it in and see if it works? –  Marichyasana Apr 27 '13 at 1:40
    
Thank you. I believe it can, because Windows explicitly/definitly stores Unicode character on FS so there is no ambiguation. I just want to know how guys in pure Linux world can cope with that problem. –  Jimm Chen Apr 27 '13 at 2:56
add comment

1 Answer

up vote 2 down vote accepted

No. Most Linux programs don't care about disk or mountpoint boundaries, and use the same encoding for all file names everywhere.

However, most utilities still treat filenames as byte strings, so there should be no problem with copying or using files named in weird encodings. (For example, I renamed a few my files to ISO-8859-13 and I can still open them in command-line or GNOME programs, although KDE has troubles.)

You can use tools such as convmv to rename all files to UTF-8 names – either in your copy, or directly on the original disk:

convmv -f gbk -t utf-8 -r /mnt/old-disk

(On Linux, it is possible to write overlay filesystems using FUSE, which could present the files from a given path using a different name encoding. I don't know if such a filesystem exists, though. bindfs has a similar purpose, but it only translates file ownership/permissions. Either way, I'm sticking with my recommendation to just convert all names to UTF-8.)

share|improve this answer
    
Thank you for referring to convmv, and I find it's a perl script. After some trying, I'm starting to worry about its reliability. convmv's processing seems to rename on a file basis, instead of on a directory basis. So, when some gbk file name sequence happens to be UTF8 sequence( touch $(echo '0xE7 0x94 0xB5 0xE8 0x84 0x91'|xxd -r).txt ), it says Skipping, already UTF-8. –  Jimm Chen Apr 28 '13 at 1:39
    
Use the --nosmart option. (The tool is most commonly used for fixing ISO-8859-𝒏 filenames, which don't have this problem.) –  grawity Apr 28 '13 at 8:31
    
Thank you. --nosmart does what I mean. But I'm not very sure whether convmv does it on a directory basis or a file basis. The man page seems not to be explicit on this. I can't figure it out myself due to not knowing Perl yet. –  Jimm Chen May 10 '13 at 8:27
    
Sadly this doesn't work for me. When I run: convmv -f ISO-8859-1 -t utf8 R*kurr* It says: Skipping, already UTF-8: ./Rökkurró Adding "--nosmart" makes it say: mv "./Rökkurró" "./Rökkurró" I think it should be "Rökkurró". I don't know why the filename is broken... the file was from windows. –  Peter 2 days ago
    
And this was fun too, to guess the encoding, but didn't find one that works. for e in $(convmv --list); do convmv -f "$e" -t utf8 --nosmart R*kurr* 2>&1 | grep "mv"; done –  Peter 2 days ago
show 3 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.