Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

The txt file with the content opens properly in the GUI editor (i am using sublime). But in vim, it appears like this, now i can't do the grep in this file. When i use less, it says "may be a binary file". How do i convert this to proper text file. Any idea why these strange characters are appearing?

Y^@ear|d^@o^@n^@o^@r^@c^@o^@d^@e^@|^@d^@o^@n^@o^@r^@n^@a^@m^@e^@|^@a^@g^@e^@n^@c^@y^@c^@o^@d^@e^@|^@a^@g^@e^@n^@c^@y^@n^@a^@m^@e^@|^@c^@r^@s^@i^@d ^@|^@p^@r^@o^@j^@e^@c^@t^@n^@u^@m^@b^@e^@r^@|^@i^@n^@i^@t^@

share|improve this question

1 Answer 1

It looks like a plain text file stored in Unicode UTF-16 LE, where each Unicode codepoint stored as one or two 16-bit code units.

If the text itself is primarily ASCII-only (all codepoints ≤ U+00FF), the result looks as if a NUL byte was inserted after every byte, if you're viewing the file from an UTF-16-incapable editor.

In command line, use iconv -f utf-16le -t utf-8 < file.txt > file-new.txt to fix it.

In Vim, you can use :e ++enc=utf-16le to re-open the file using the proper encoding.

If possible, tell Sublime to save files as UTF-8, which will be much more compatible with various tools. (In Vim: :set fileencoding=utf-8 | w)

share|improve this answer
1  
Actually, ASCII text would be all code points ≤ U+007F. –  dotancohen Apr 18 at 14:37
    
Right, miscalculated. –  grawity Apr 18 at 14:51
    
Thanks. I also found a command that stripped all the unknown char from the file tr -cd '\11\12\15\40-\176' < input-file –  Anjesh Apr 20 at 2:55
    
@Anjesh – That's not exactly a good way to "fix" it; it just happens to work because the text is mostly ASCII. For anything else it'd just end up as garbage. –  grawity Apr 20 at 9:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.