I currently work on a project where I need to write the Unicode characters ɛ and ħ, which I just added to my keyboard.

It works great almost everywhere, even in the command prompt, except for Vim who displays a question mark ? instead of ɛ and h instead of ħ.

The encoding is set to utf-8 and using the :digraphs isn't much of a help since even there, a lot of characters are shown simply as ?.

What can I do to fix this ?

I use the Consolas font which has both the aforementioned characters.

  • What about GVIM? Does it show those correctly? – Ingo Karkat May 24 '14 at 18:57
  • Nope, the same problem there too! – ahmed May 24 '14 at 18:59
  • 1
    What are the values of 'fileencodings', 'fileencoding', 'encoding'? – romainl May 24 '14 at 20:02
  • usc-bom, no value and latin1, should I set them all to utf-8? – ahmed May 24 '14 at 22:41
  • UPDATE: I set all these params to utf-8 but nothing happened. – ahmed May 24 '14 at 22:50

From the comment questions, even though the question states "encoding is set to utf-8", it appears Vim's encoding is NOT set to utf-8. It is still latin1 (ISO-8859-1). This means Vim does not know how to represent those special characters in memory, nor save them to a file.

For Vim to correctly write "special" characters, it needs 2 things:

  1. An 'encoding' setting that can represent that character in memory, usually utf-8
  2. An empty value, or a setting that can represent that character on-disk for 'fileencoding'. Usually this is also utf-8, or empty.

To properly read a file containing these characters, Vim again needs these two things. Vim can automatically detect the proper value for 'fileencoding', if your 'fileencodings' option is set properly.

'fileencodings' is a comma-separated list that tells Vim (in order) which encodings to try. "ucs-bom" as in the comments on your question will detect unicode, but only if there is a BOM (byte-order mark). Additionally since your Vim's 'encoding' is still "latin1" it wouldn't do you any good anyway if Vim DID detect unicode.

Note both 'encoding' and 'fileencodings' should be very close to the top of your .vimrc, before ANY mappings, menus, string options, or register settings. Otherwise existing text in those areas will be misinterpreted by Vim.

See the help for each of these options, and the wiki page on configuring Vim for Unicode, for details.

If Vim does't detect encoding properly on some files even with the correct settings, you can either:

  1. Set it manually, like :e ++enc=utf-8 myfile.txt
  2. Install a Vim plugin that will automatically detect file encoding

I work in debian and had the issue you describe. To fix it you need to be sure you have the locale you require installed on your operating system in debian bash this is how to fix it:

  1. check your locale

    locale -a

    in bash this reported: locale: Cannot set LC_VARIABLE to default locale: No such file or directory

  2. generate the required locale so step 1 no longer complains.

    you can do that with the locale-gen command if you know what locale you require

    sudo locale-gen "en_US.UTF-8"

    or just reconfigure locale by issuing the following command

    sudo dpkg-reconfigure locales

If done correctly : digraphs should show the proper characters now in vim


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.