3

I have this odd issue that I've been having for years now since Windows 8 (I'm now on Windows 10, if I remember right. The problem only seems to be on my work computer. My other personal computers don't seem to have a problem. I didn't think about asking for help at first because I found a workaround - which I'll explain later - but I think enough is enough.

Basically, whenever I copy and paste Unicode text (Japanese, Arabic, etc.) they appear as question marks on paste. Here's an example Japanese text that I'll copy:

何これ?!意味わかない!

And here's what it looks like after pasting:

????!??????!

Interestingly, if I copy the exact same text again for at least one more time, it will paste properly...

何これ?!意味わかない!

Removing even a single character from the selection before copy will cause the issue to "reset".

This was my workaround. It's not too difficult to do but I tend to always forget to do it because my other PCs work fine. This adds more steps and wastes precious seconds.

The problem is system-wide and affects all the programs and apps I use.

Any idea how to fix this permanently? Any help will be highly appreciated.

  • It would be helpful if you mention the programmes you copy from or paste into. Mention the Windows version and locale it is set to as well – Sanny Jan 14 '17 at 4:51
  • Every program reacts the same - whether I copy from and paste to them - and I experienced the issue since Windows 8 which is why I'm not mentioning any specific version numbers. I'm now using Windows 10 as mentioned. The system locale I use on all my devices is English (United States) yet this is the only device with the annoying copy-paste issue. – DeVilFisCh Jan 15 '17 at 5:46
  • Hmmm... maybe I'll try switching to a different locale then try the copy-paste thing. If it gets fixed, then the problem may lie within my locale settings and I'll try to switch back again hoping it will fix whatever it is. Makes me wonder why I never thought about doing that before. – DeVilFisCh Jan 15 '17 at 5:53
  • Okay. I feel stupid now. Switching the locale settings fixed the issue. I can copy paste Unicode text to my heart's content even when going back to English (US). – DeVilFisCh Jan 15 '17 at 6:36
  • Glad to hear that. Make sure you can copy and paste many languages safely (use non English Wikipedia pages for example) Post your solution in detail and accept it as the answer to your problem. – Sanny Jan 15 '17 at 6:44
2

I've "suffered" from this issue for years and I never knew the fix was so dead simple until Sanny menitoned "locale" in a comment above (Thanks Sanny!). Haha! Anyway, here's how to fix it if you come upon the same issue as I did:

This applies to Windows 10 (build 15002) but it may be similar to older (or newer) versions of Windows.

  1. Go to the Region settings in the Control Panel. There are several ways to do this and here's a few of them.
    • In the Search bar (Cortana) on the taskbar, search for "Control Panel". In the Control Panel, click on Change date, time, or number formats under Clock, Language and Region in category view or Region in icon list view
    • Windows 10 only: In the Search bar again, search for "region & language settings". This will open the Region & Language page in the Settings app. Scroll down until you find Additional date, time, & region settings. You may then select Region on the Control Panel window that opens.
  2. Open the Administrative tab and click on the Change system locale button. Choose a locale that is different to your current locale. I went with Japanese. I think choosing the language you will copy-paste often would be best, though it may be the same regardless. Acknowledge the change with OK.
  3. The system will ask you to restart which you'll obviously need to do to notice the changes.
  4. After restarting, test if copy-paste now works as intended. Upon success, you may re-do the above steps again and switch back to the locale you actually need to use.

That's it! Enjoy copy-pasting! ;)

2

Microsoft's products are all Unicode compliant. It doesn't make sense that you have to change your locale to fix the issue.

The ????? indicates that Unicode or UTF-8 is not being recognized properly (rather than being misdiagnosed as a different charset (perhaps between the program and the clipboard).

But it seems that this is an actual bug - it seems like the OS thought it was ASCII the first time, but then tried again with UTF-8. The Unicode world is very complex - to store full Unicode in every possible charset, you would need double the space and convert all your functions to be UTF-16 compliant - a massive undertaking and not very practical - imagine the storage and processing you will need to convert to UTF-16/32 - we're talking every document you own or view...so practically we use UTF-8 which encodes the standard UTF to 8-bit. But legacy functions and ASCII-based docs need to be converted to UTF-ASCII etc. What was going on here I surmise is that the OS 'guessed' incorrectly that the encoding was ASCII and used a separate function/class to copy and paste (or the default function/class). Once it 'realized' the encoding was Unicode it used that encoding. While Unicode UTF-8 is the standard there are on average about 3-4 different encodings per language for an OS to do deal with, and without knowing ahead of time what the encoding is - having to determine that is pretty hard.

From a computer's perspective, your character just looks like a pre-determined set of 1's and 0's and there is no way of objectively knowing what's the correct conversion that 01000001 is an 'A' for example. It could also be an א in Hebrew or some other character. Unicode changed all of that - each character has a unique 8-bit assignment which means you can determine what it is based on the encoding range.

So the misbehaving copy and paste probably has to do with legacy functionality with ASCII - upgrade and it should solve the problem!

  • I know it didn't make sense. I could copy the text properly when copying it at least twice (as mentioned in the question) and I also did mention that it is a system-wide issue and happens on only one computer that I use (although it has been permanently fixed by now by doing the steps in my answer). I think it was a very obscure bug of some sort that persisted even after upgrading to Windows 10 from 8 which is weird, to be honest. Either way, I have never encountered the problem again. – DeVilFisCh Oct 3 '18 at 20:33
  • Since I deal with Unicode a lot and deal with different charsets I do relate. There are other points. In order to enable that the filesystem is Unicode enabled, the regional settings must be set appropriately. There are the locale settings, which apps use and then there are the language settings and keyboard settings. If you have a different language you need to set that as well as the appropriate keyboard. It is quite possible that your keyboard types in ASCII and not Unicode ASCII which was causing an issue. – Danny F Oct 5 '18 at 9:42
  • Would that explain why I could properly copy the Unicode text if I copy it twice? Because it actually works if I do it that way. I just have to hit CTRL+C two times after highlighting the text to make it work. If I don't do so, I get the question marks when I paste the contents of the clipboard. I also get question marks if I change the highlighted text before I copy again. If the text was ASCII wouldn't it not work either way? – DeVilFisCh Oct 14 '18 at 19:52
  • Also, it seems I never mentioned that I had no trouble typing in Japanese or any other Unicode language. Regardless of the source - whether I typed it myself or if I got it from the web (Wikipedia, etc.), the results were the same - copy and pasting was the only thing that was borked if I don't do the copy-twice procedure. – DeVilFisCh Oct 14 '18 at 19:59
  • Correction: your text was Unicode, the OS's copy/paste functionality saw it as ASCII. – Danny F Oct 16 '18 at 2:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.