Tell me more ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.
up vote 42 down vote favorite
11

When saving the text "Bush hid the facts" in notepad under Windows XP, how come when you reopen it shows squares instead of the text?

I saw it in this video if you need an example

http://www.youtube.com/watch?v=9bK9-sc_uus&feature=related

share|improve this question
22  
By the way, it's the same for any sequence of 4-3-3-5 letters, not just this one. – grawity Jul 31 '09 at 8:42
12  
Example: "John ate the bacon" – Troggy♦ Aug 18 '09 at 18:24

1 Answer

up vote 82 down vote accepted

This is due to a problem with the Win32 API function IsTextUnicode dating back to Windows NT 3.5. If a file is encoded in ANSI, the function will interpret it as UTF-16LE resulting in unreadable characters.

This fascinated me too back when I discovered it since I was kind of young and naive, I thought it was an actual conspiracy :)

There is actually a Wikipedia article on this you can find here.

share|improve this answer
17  
Interesting. +1 for the Wiki article that taught me the word "mojibake" and its particularly meta warning that "without proper rendering support, you may see question marks, boxes, or other symbols..." :-) – jtb Jul 31 '09 at 1:33
3  
+1 because, despite using Windows for as long as I can remember, I somehow never came across this! – Jared Harley Jul 31 '09 at 2:56
7  
Well, it's clearly a bug, because the software incorrectly. The best you can argue is that bugs like this are impossible to eliminate without losing other functionality. And, heck, Microsoft fixed it in Vista [according to Wiki], so someone there obviously thought it was a bug too. – John Fouhy Jul 31 '09 at 4:30
11  
It's not a bug if it does exactly what it advertise (i.e. documented) to do. It's specified precisely that it's a statistical test and not foolproof, and the shorter the input, the higher the error rate. It just so happens that in this case, it happens with a sentence that make sense to human. This particular sentence doesn't work with Vista & 7 because the implementation of IsTextUnicode have been changed and presumingly improved and it now report correctly for this sentence. What we have is better or worse false positive / negative rate, not bugs. – KTC Jul 31 '09 at 6:00
4  
"It's not a bug if it does what it's supposed to." Yeah maybe the technical term is 'design flaw' or something, but I think most people would still say it's ok to call it a bug. – davr Nov 18 '09 at 0:37
show 4 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.