I tried to get robocopy in Windows 7 to generate a Unicode log, since I have files with Unicode characters. The command I used:
robocopy C:\mysource D:\mydest /mir /unilog:backup.log /tee
File the copy works and the onscreen output is correct, the log file itself just contains gibberish. This is regardless of whether I use the Command Prompt or the Powershell.
What gives? Am I doing something wrong?
It appears to be a bug in the XP27 version of RoboCopy (which comes with Windows 7).
In version XP26 (which comes with Windows Vista) /UNILOG produces a perfectly readable Unicode log file for me.
If you don't have a copy of Vista laying around EasyRoboCopy also comes with the XP26 version. (I haven't actually tried EasyRoboCopy itself, just extracted robocopy.exe out of its setup file using WinRAR.)
At a glance, I'd say the file written by Robocopy while using the /UNILOG and /TEE switches contains a UTF-16 little-endian byte order mark followed by an ISO-8859-1 terminal typescript.
To make it readable, I did the following in Ubuntu:
dd if=robocopy.log ibs=1 skip=2 obs=512 | # Strip the byte order mark
iconv --from-code ISO-8859-1 --to-code UTF-8 | # Convert to UTF-8
col -b > robocopy_utf-8.log # Interpret control characters
The resulting file matches what I saw in the Windows command prompt.
([System.Text.Encoding]::Unicode).GetString([System.Text.Encoding]::Convert([System.Text.Encoding]::GetEncoding(28591), [System.Text.Encoding]::Unicode, ([System.Text.Encoding]::GetEncoding(28591)).GetBytes($_)))
Apr 28, 2013 at 8:26
Looking at the (binary) file output on Win7, the /UNILOG option is useless. It writes the standard UNICODE BOM (FFFE), but then proceeds to write all narrow characters EXCEPT for the options line (e.g., /BYTES /S /COPY:DATS ...), which is actual unicode. After that, it reverts back to ANSI chars, and it is not UTF-8, either; i.e., if you have a filename with a wide character in the path, it is converted to a narrow '?' character.
Apparently no interest in fixing it from MSFT, since it's been this way for some time, and I have all updates.
I fixed my unreadable, Unicode-format Robocopy log files in Windows (which were accidentally created by appending normal Robocopy output to Unicode output from Out-File in PowerShell), as follows:
In PowerShell:
$bytes = [System.IO.File]::ReadAllBytes('C:\Temp\RoboCopyLog.txt')
$len = $bytes.Length
#Remove the Unicode BOM, and convert to ASCII
$text = [System.Text.Encoding]::ASCII.GetString($bytes,2,$len -2)
$text
The above code may not work for all file sizes!
(Credit for code: I adapted code from this post by Ferdinand Prantl: Stackoverflow - Read/Parse Binary files with PowerShell
If your file or directory names contain Unicode characters then before issuing the Robocopy command with the /unilog parameter use the chcp 65001 command. (Code page 65001 is UTF-8.)
Once you have the mangled Unicode log, just open it up in MS Word as Unicode (UTF-8) and save it:
In your case, the command in Powershell goes something like this:
robocopy C:\mysource D:\mydest /mir | Out-File backup.log
The workaround is that you use Out-File instead of built-in /unilog parameter. You will get exactly the same log file, but now it will be properly written in unicode.
Run the chcp command before robocopy command, with the right code page.
for UTF-8 (not working with robocopy & Hebrew and maybe more languages):
chcp 65001 | Out-Null
for Hebrew:
chcp 1255 | Out-Null
Full code page list: https://docs.microsoft.com/en-us/windows/desktop/intl/code-page-identifiers
