Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am only using the 128 character set defined in the original ANSI standard.

But as a whole how are the files implmeneted differently.

I am not concerned with the display, i.e. if a tab is displayed with 6 or 8 characters but the actual internal representation in memory

One difference I've heard is the use of \r\n (Windows) vs. \n for line termination (Linux).

share|improve this question
    
I think the byte order mark is killing my #!(first line) in my php files I transferred over from windows to linux. The whole file works but it can not find the interpreter as it should. If I specefically make sure to encode in ANSI by selecting the encoding method in notepad is it true ASCII or does Windows do something else –  user73919 Jun 9 '11 at 23:52
add comment

3 Answers 3

up vote 5 down vote accepted

"Unicode" on Windows is UTF-16LE, and each character is 2 or 4 bytes. Linux uses UTF-8, and each character is between 1 and 4 bytes.

"The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)"

share|improve this answer
    
Windows wastes a byte? –  user73919 Jun 7 '11 at 21:02
    
If you're not using anything outside of Latin-1, yes. –  Ignacio Vazquez-Abrams Jun 7 '11 at 21:07
    
They're in the article I linked to. –  Ignacio Vazquez-Abrams Jun 7 '11 at 21:28
    
Ran a search for UTF-16LE but did not find it in the article. –  user73919 Jun 7 '11 at 21:32
    
It's not specifically called that in the article. Instead it talks about "low-endian" (or "little-endian" as we call it). –  Ignacio Vazquez-Abrams Jun 7 '11 at 21:35
show 2 more comments

One difference I've hear is the use of \r\n (Windows) vs. \n for line breaks (Linux).

Yes. Most UNIX text editors will handle this automatically, Windows programmers editors may handle this, general text editors (base Notepad) will not.

Windows seems to also need the EOF (Ctrl-Z) as END OF FILE in some contexts, whereas you'll probably never see it on UNIX.

Remember that MacOS X is now UNIX underneath, so it uses UNIX line endings. Though before OS X (MacOS 9 and below) it had its own ending (\r)

share|improve this answer
    
Where are \r\n and \n in the ASCII character set? en.wikipedia.org/wiki/File:ASCII_Code_Chart.svg –  user73919 Jun 7 '11 at 21:31
    
@Chris \n is ASCII 0x0A, Line Feed. \r is ASCII 0x0D, Carriage return –  Rich Homolka Jun 7 '11 at 21:36
    
LF and CR. Thanks. –  user73919 Jun 7 '11 at 21:38
    
@Rich What about EOF? Is this an ANSI character? –  user73919 Jun 7 '11 at 21:39
    
it appears to be the ASCII character SUB –  user73919 Jun 7 '11 at 21:45
show 1 more comment

Line breaks

Windows uses CRLF (\r\n, 0D 0A) line endings while Unix just uses LF (\n, 0A).

Character Encoding

Most modern (i.e., since 2004 or so) Unix-like systems make UTF-8 the default character encoding.

Windows, however, lacks native support for UTF-8. It internally works in UTF-16, and assumes that char-based strings are in a legacy code page. Fortunately, Notepad is capable of reading UTF-8 files; unfortunately, "ANSI" encoding is still the default.

Problematic Special Characters

U+001A SUBSTITUTE

Windows (rarely) uses Ctrl+Z as an end-of-file character. For example, if you type a file at the command prompt, it will be truncated at the first 1A byte.

On Unix, Ctrl+Z is nothing special.

U+FEFF ZERO WITH NO-BREAK SPACE (Byte-Order Mark)

On Windows, UTF-8 files often start with a "byte order mark" EF BB BF to distinguish them from ANSI files.

On Linux, the BOM is discouraged because it breaks things like shebang lines in shell scripts. Plus, it'd be pointless to have a UTF-8 signature when UTF-8 is the default encoding anyway.

share|improve this answer
    
Ctrl-Z works on windows just like Ctrl-D ( or whatever character you have bound to EOF with stty ) does on Linux: the console driver translates it to end of file. The literal character does not appear in the input stream; it just causes read() to return 0. –  psusi Jun 8 '11 at 1:22
    
I think the byte order mark is killing my #!(first line) in my php files I transferred over from windows to linux. The whole file works but it can not find the interpreter as it should. If I specefically make sure to encode in ANSI by selecting the encoding method in notepad is it true ASCII or does Windows do something else? –  user73919 Jun 9 '11 at 23:52
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.