0

I have a couple of powershell automation scripts I wrote that sometimes require editing files used by linux OS family.

I know that windows ends strings in files using the CR LF flags, while linux uses only the LF flag. Both also use different versions of utf8 encoding. While windows is able to read linux-written files, the opposite is not the same, and this actually causes problems when reading windows-written files in linux.

Yet it seems there's more to it: I discovered windows uses utf8BOM, and actually does not fully support natively the simple utf8 encoding compared to linux.

My question is this: When using the set-content cmdlet, the default encoding used by powershell is utf8NoBOM. What is the actual difference between utf8, utf8BOM and utf8NoBOM when using the -Encoding parameter when writing to files?

From Microsoft's Set-Content manual page:

-Encoding

Specifies the type of encoding for the target file. The default value is utf8NoBOM.

  • utf8: Encodes in UTF-8 format.
  • utf8BOM: Encodes in UTF-8 format with Byte Order Mark (BOM)
  • utf8NoBOM: Encodes in UTF-8 format without Byte Order Mark (BOM)
0

This is not a Powershell specific thing. It is an industry format specification. So, just search the web for each encoding individually to get the details.

https://en.wikipedia.org/wiki/UTF-8

Such discussions are had for example:

MicrosoftDocs/PowerShell-Docs https://github.com/MicrosoftDocs/PowerShell-Docs/issues/4021

What's the difference between UTF-8 and UTF-8 without BOM? https://stackoverflow.com/questions/2223882/whats-the-difference-between-utf-8-and-utf-8-without-bom

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.