Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have Windows 7 with NTFS filesystem. I have filenames and directory names like:

Kispál és a Borz - 02 - Tökéletes Helyettes

I want to transform them to:

Kispál és a Borz - 02 - Tökéletes Helyettes

The filesystem is capable of storing filenames like フリー百科事典, so it surely has unicode support.

As I imagine the story, a long time ago they were perfect. Then they were transferred from an UTF-8 to a Latin-1 filesystem, then back to this UTF-8 supporting filesystem. In theory, all information is there, I could write a program in C to fix these characters, but I assume someone somewhere already did it.

Do you know any utility that can do the transformation?

share|improve this question
add comment

1 Answer

up vote 0 down vote accepted

I wrote a C / C++ hybrid which does the translation part (does not rename anything, just converts bad byte sequences to good ones). You can download it using the link at the end of this post.

The input file is decoded as an UTF-8 stream into a UNICODE code position sequence which is then NOT converted to any other codepage. All code-positions are under 256, they represent the original UTF-8 string's byte sequence. So I just write these code positions as bytes to the output. The result is a correct UTF-8 string. It is still not an application for my problem, but the core of the solution.

The program is written and tested under Linux, but should work on any OS. Usage example:

nil@hippy:~/playground/c++$ g++ utf8decode.cpp -o utf8decode
nil@hippy:~/playground/c++$ cat > file
Kispál és a Borz - 02 - Tökéletes Helyettes
nil@hippy:~/playground/c++$ cat file | ./utf8decode
Kispál és a Borz - 02 - Tökéletes Helyettes
Characters found: 48
nil@hippy:~/playground/c++$

I wrote an UTF-8 character counter before, and I modified that. I havn't written the whole program in an hour. Source: http://pastebin.com/Hy7tVt5A http://pastebin.com/NFJUP0R5

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.