Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

One of our work steps involves saving an MS Excel worksheet as CSV and then using UltraEdit to convert the CSV to UTF-8 before importing it into a server system.

The problem is that, after the conversion to UTF-8, the file always contains 3 nonsense characters at the start of the file:

ENTITY_ID;FIELD2;FIELD3,FIELD4;(etc.)
value1;value2;value3;value4;(etc).

Observations:

  • As you can see, there are 3 characters that are noise and cause the server to reject the CSV import because the first column is not named "ENTITY_ID". The characters are always the same.

  • These characters are not shown after the conversion, but when we close and reopen the file in UltraEdit again, then we do see the characters.

  • These characters are only visible in UltraEdit. Windows Notepad or Notepad++ does not show them.

  • Using Notepad++ to convert the CSV to UTF-8 produces the exact same output: a file with the same 3 odd characters in the beginning. The only difference is that Notepad++ does not display these characters, even after closing and reopening the file.

Workaround:
We reopen the file in UltraEdit, delete the noise, and then the server accepts the CSV import.
This step needs to be eliminated by fixing the actual problem.

Question: How can we avoid these 3 characters?

share|improve this question

1 Answer 1

up vote 8 down vote accepted

That's the byte order mark, encoded as UTF-8. Tell your editor to not add it at the beginning, or use a real decoder in your server system.

share|improve this answer
    
Thank you! This prompted another question, of course :-) –  Torben Gundtofte-Bruun Jan 18 '12 at 12:21

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.