Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

What are the standard encodings in Mac, Unix and others?

I know that if my friend who uses Windows sends me a file, it'll most probably be encoded in cp1251. So in emacs, I can do: revert-buffer-coding-system and will choose cp1251.

share|improve this question
add comment

1 Answer

up vote 7 down vote accepted

There is no "the standard encoding", you have many of them.

UTF-8, an Unicode encoding, is the most commonly used these days. It is the default on most Linux distributions, as well as many email clients on other platforms. On Windows, "Unicode" usually refers to UTF-16, which is just another representation of the same Unicode.

Other encodings have been slowly falling into disuse, due to their limitations and inability to use multiple languages in the same file.

  • The most "standard" of them would be the ISO 8859 encodings, mainly ISO 8859-1 for "Western Europe".

  • Their Windows counterparts are very similar; the most common is Windows-1252 aka cp1252, a superset of ISO 8859-1. (Your cp1251 is for Cyrillic script.)

  • Other encodings such as KOI8-R and Shift-JIS are also used.

See also a list of character encodings.

A Python script named chardet is able to accurately guess the encoding used on given files.

share|improve this answer
    
UTF-8 is the default on OSX –  Mark Sep 23 '11 at 11:47
    
@grawity: It looks like chardet doesn't have a CLI interface. Is that true? –  Adobe Sep 23 '11 at 12:13
    
@Adobe: True, but using chardet from Python interactive mode is easy. import chardet; data = open("myfile").read(); print chardet.detect(data) –  grawity Sep 23 '11 at 12:19
    
@grawity: Thanks - using Your solution I've made it a python script - so now it has a CLI interface. I've put the script here. Well - thanks again - it's so nice to have such an ability to find out the encoding automatically. –  Adobe Sep 23 '11 at 13:10
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.