1

I am creating a CSV file with UTF-8 encoding.

It displays the Chinese characters properly if I open it using Notepad++.

It displays the Chinese characters properly if I open it using Notepad.

But, if I open it using Excel, I see garbage values.

If I open it using Notepad and press Control+S and close the file and reopen in Excel, it shows the characters perfectly.

Any clues whats going here?

I also tried to create a normal CSV file which has Encoding GB2312. I then used Unix command iconv to convert the file from "GB2312" to "UTF-8"

This command removes all the Chinese characters and shows only English characters. So, this is not a solution.

Please suggest.

Edit -

It doesn't show Garbage value, but shows ANSI encoding. Why does it picks ANSI encoding by default. And why after opening the same file in Notepad/Notepad++ it opens up with UTF-8.

Also, why just Control+s makes it open in Excel correctly.

  • For future reference: when using iconv, see the option --add-signature to add the BOM. – Arjan Aug 13 '15 at 8:28
3

In Linux UTF8 is usually saved without BOM since UTF8 BOM is optional, thus apps might have problem detecting proper encoding. You can try to save them in UTF8 with BOM or use UTF16 instead.

  • The problem for me was - I do not have any option to add data in UTF16 format in my programming language. Only UTF-8 is allowed and that goes without BOM. – Techidiot Aug 13 '15 at 11:48
0

I was aware of BOM sequence with UTF-8 encoding but was not aware of how to add it in my file.

Now, the BOM sequence is 0xEF,0xBB,0xBF, so I tried add this as plain text but it didn't worked. So, I added the ASCII values of these to my CSV and it works.

ASCII value is =

new.text = chr$(239) & chr$(187) & chr$(191) |Concatenated

Simply add this as 1st line and it works!

Thanks Chris for your suggestion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.