Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

When saving the text "Bush hid the facts" in notepad under Windows XP, how come when you reopen it shows squares instead of the text?

I saw it in this video if you need an example

http://www.youtube.com/watch?v=9bK9-sc_uus&feature=related

share|improve this question
25  
By the way, it's the same for any sequence of 4-3-3-5 letters, not just this one. –  grawity Jul 31 '09 at 8:42
13  
Example: "John ate the bacon" –  Troggy Aug 18 '09 at 18:24

1 Answer 1

up vote 86 down vote accepted

This is due to a problem with the Win32 API function IsTextUnicode dating back to Windows NT 3.5. If a file is encoded in ANSI, the function will interpret it as UTF-16LE resulting in unreadable characters.

This fascinated me too back when I discovered it since I was kind of young and naive, I thought it was an actual conspiracy :)

There is actually a Wikipedia article on this you can find here.

share|improve this answer
17  
Interesting. +1 for the Wiki article that taught me the word "mojibake" and its particularly meta warning that "without proper rendering support, you may see question marks, boxes, or other symbols..." :-) –  jtb Jul 31 '09 at 1:33
3  
+1 because, despite using Windows for as long as I can remember, I somehow never came across this! –  Jared Harley Jul 31 '09 at 2:56
7  
Well, it's clearly a bug, because the software incorrectly. The best you can argue is that bugs like this are impossible to eliminate without losing other functionality. And, heck, Microsoft fixed it in Vista [according to Wiki], so someone there obviously thought it was a bug too. –  John Fouhy Jul 31 '09 at 4:30
11  
It's not a bug if it does exactly what it advertise (i.e. documented) to do. It's specified precisely that it's a statistical test and not foolproof, and the shorter the input, the higher the error rate. It just so happens that in this case, it happens with a sentence that make sense to human. This particular sentence doesn't work with Vista & 7 because the implementation of IsTextUnicode have been changed and presumingly improved and it now report correctly for this sentence. What we have is better or worse false positive / negative rate, not bugs. –  KTC Jul 31 '09 at 6:00
4  
"It's not a bug if it does what it's supposed to." Yeah maybe the technical term is 'design flaw' or something, but I think most people would still say it's ok to call it a bug. –  davr Nov 18 '09 at 0:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.