When saving the text "Bush hid the facts" in notepad under Windows XP, how come when you reopen it shows squares instead of the text?

I saw it in this video if you need an example


  • 28
    By the way, it's the same for any sequence of 4-3-3-5 letters, not just this one. – user1686 Jul 31 '09 at 8:42
  • 15
    Example: "John ate the bacon" – Troggy Aug 18 '09 at 18:24

This is due to a problem with the Win32 API function IsTextUnicode dating back to Windows NT 3.5. If a file is encoded in ANSI, the function will interpret it as UTF-16LE resulting in unreadable characters.

This fascinated me too back when I discovered it since I was kind of young and naive, I thought it was an actual conspiracy :)

There is actually a Wikipedia article on this you can find here.

| improve this answer | |
  • 17
    Interesting. +1 for the Wiki article that taught me the word "mojibake" and its particularly meta warning that "without proper rendering support, you may see question marks, boxes, or other symbols..." :-) – jtb Jul 31 '09 at 1:33
  • 3
    +1 because, despite using Windows for as long as I can remember, I somehow never came across this! – Jared Harley Jul 31 '09 at 2:56
  • 7
    Well, it's clearly a bug, because the software incorrectly. The best you can argue is that bugs like this are impossible to eliminate without losing other functionality. And, heck, Microsoft fixed it in Vista [according to Wiki], so someone there obviously thought it was a bug too. – John Fouhy Jul 31 '09 at 4:30
  • 11
    It's not a bug if it does exactly what it advertise (i.e. documented) to do. It's specified precisely that it's a statistical test and not foolproof, and the shorter the input, the higher the error rate. It just so happens that in this case, it happens with a sentence that make sense to human. This particular sentence doesn't work with Vista & 7 because the implementation of IsTextUnicode have been changed and presumingly improved and it now report correctly for this sentence. What we have is better or worse false positive / negative rate, not bugs. – KTC Jul 31 '09 at 6:00
  • 5
    "It's not a bug if it does what it's supposed to." Yeah maybe the technical term is 'design flaw' or something, but I think most people would still say it's ok to call it a bug. – davr Nov 18 '09 at 0:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy