2

Here is what I want to do:

Redirecting properly the output of console applications (I use the term command in the rest of the message) into a file with 1252 encoding (to make it readable from any notepad soft in default configuration.

What I’ve observed:

Chcp is effective with internal commands and some external command (recent ones)

First of all it’s worth noticing CHCP operates differently under Win7 and Win 10.

If the following batch is run from a cmd prompt, you can notice the command outputs are displayed properly in win10 console whereas a win7 console renders characters out of ASCII badly.

for /f "tokens=2 delims=:" %%G in ('chcp') do Set _cp_=%%G
chcp 1252
@echo test an internal command
dir
@echo test an external (recent) command: Robocopy
robocopy .\ .\ /L
@echo test an external (legacy) command: Xcopy
xcopy test.txt 2>&1
chcp %_cp_%
echo end of test.cmd batch 

Incidentally, I am interested in knowing what causes such a difference although it’s not really the purpose of that message and since it’s easily fixable by adding a ps invoke “powershell [console]::outputencoding=[system.text.encoding]::getencoding(850)” in the batch after the 1st chcp command.

Whatever the real issue occurs when the batch output is redirected into a file: test.cmd > test.txt.

In that case the result is the same whatever OS. The output of Internal commands and new external commands (Robocopy, Bcdedit, etc) are properly 1252 encoded. Legacy commands (xcopy, chcp, etc) are not (output in OEM code page). In brief, most of commands are not affected by CHCP or equivalent [console] change thru powershell.

Various speculations about that mess:

  1. The legacy command code is based on CRT whereas internal commands and most recent external ones use Win32 API. It’s based on the last section regarding the console application development from MSDN Globalization Step-by-Step!

  2. Since at least win10 what is displayed in the console (same encoding for all command outputs) and stored a file is different (output encoding change depending the command), output/input streams may be handled differently depending on the type of handles they point. Console functions may be used for display and I/O file functions in case of redirection. Speculation based on High-Level Console Input and Output Functions!

  3. MS recommends the code of console applications forces OEM encoding of the output stream. Ref. Console Application Issues If MS suggestion is applied in the code of external commands that may explain why the redirection of their output streams into a file is always encoded OEM_CP whatever the console code page is applied. Oddly, readfile and writefile are not mentioned among functions affected by SetFileApisToOEM

Finally I don’t know if the difference between legacy commands and lately introduced ones is because their code respect MS suggestion and just because string literals are coded OEM vs ANSI..

Possible solutions/workaround

If 3 is correct, they are certainly very few.: It’s possible to change the value of registry key HKLM\system\currentset\control\NLS\codepage OEMCP=1252. It’s not safe (do not try to set Unicode 65001, your system may refuse booting) and inconvenient (reboot necessary). Or, filling the file with OEM encoded contents only and transcoding the file with PS script at the end of the batch. Simple but not very elegant if the file has to be accessed and checked periodically.

If 2 is correct, there may exist a function that controls the encoding of I/O file function readfile and writefile.

If 1 is correct, it should be possible to control the international settings or culture of the current user session and so control the code page of CRT application. Since Win8, it’s possible through Powershell Configure International Settings in Windows. Command line applications are also able to perform such things. Whatever, the difficulty here is about creating a “culture” with OEM code page set to 1252 as that doesn’t exist in the pre-defined set.

Even if there is no effective solution regarding that issue, do not hesitate to share your knowledges of that topic. I am just curious to understand how MS has implemented that stuff.

  • What does bash have to do with this question? Please remove the tag unless you can explain. – DavidPostill Apr 28 '16 at 16:22
  • Your test script on Windows 7 here does not display anything strange. – DavidPostill Apr 28 '16 at 16:26
  • Sorry for the bash tag nothing to do actually . The tag was suggested to me and I read batch. On my Win7 localized fr-fr, if I run the test script all non Ascii char are rendered badly: Here’s the result of first commands: >chcp 1252 Page de codes activeá: 1252 test an internal command >dir Le volume dans le lecteur C sÆappelle System Le numÚro de sÚrie du volume est xxx – user3528651 Apr 28 '16 at 16:57
  • Ah. Can't help you then. I have an English version of Windows with no funny characters ;) – DavidPostill Apr 28 '16 at 16:58
  • : Never mind, but yes, I forgot to mention all that stuff only makes sense for systems out of en-US (or similar) locale and using glyphs different of ASCII characters in their local language – user3528651 Apr 28 '16 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.