8

I use Emacs to edit an XML file that is also read by another application. The other application requires that the file start with a byte-order marker (BOM). However, Emacs seems to remove the BOM every time I edit the file. Is there a way to make Emacs leave the BOM alone?

  • Is this why when I edit an xml file for schtasks, it changes the encoding from unicode to unicode big endian and then doesn't work? – js2010 Jan 15 at 17:59
9

Emacs will write a BOM or not depending on what coding system it is using. Emacs automatically chooses the coding system it uses when visiting a file.

You can change the coding system to utf-8-with-signature, which will tell Emacs to write the BOM.

To change the coding system of a visited file:

C-x RET r utf-8-with-signature RET

You can set the coding system that Emacs uses for a particular file by setting a file variable. See the fine manual section 57.3.4 Local Variables in Files to learn how to do that.

  • Awesome, exactly the kind of answer I was hoping for! Thank you! – Vebjorn Ljosa Sep 17 '09 at 13:52
5

Followup on Richard Hoskins’s answer: if you never want the BOM to be hidden by emacs, you can disable the *-with-signature codings with this snippet:

(setq auto-coding-regexp-alist
  (delete (rassoc 'utf-16be-with-signature auto-coding-regexp-alist)
  (delete (rassoc 'utf-16le-with-signature auto-coding-regexp-alist)
  (delete (rassoc 'utf-8-with-signature auto-coding-regexp-alist)
          auto-coding-regexp-alist))))

The BOM is U+FEFF, the “zero-width non-breaking space”, and doesn’t show up as a box in my emacs 23.1.1—instead, the top line of the file’s moved slightly down, and a box sometimes appears around the first line—but you can see that the BOM’s there, and delete it if necessary.

1

Emacs "itself" should not mess with the BOM; if it's really doing that, then it would have to be the code implementing the Emacs "mode" you are useing to edit your XML files which removes the BOM. Since you don't say which one that is, I can only refer you to the documentation for that mode, or that you open the files in fundamental-mode (or similar non-destructive mode). Or try M-x find-file-literally if all else fails.

  • use xml-mode (i.e., sgml-mode), but that problem seems to be elsewhere: if I open the file with find-file-literally and then do M-x sgml-mode, the BOM is not removed. Because special characters are not UTF-8-encoded when visiting a file literally, it would be nice to figure out where in the underlying format conversion and character code conversion code the BOM is removed. – Vebjorn Ljosa Sep 15 '09 at 12:38
0

In my test, editing a UTF-8 file doesn't change the encoding and the BOM remains (efbb bf). (nxml-mode)

Well, this may vary between xml-mode and nxml-mode, or the version of emacs (24 vs 26). It says the mode on the bottom.

If you edit an Emacs an XML file encoded in unicode (UTF-16 little endian), it will change the encoding to UTF-16 big endian. Maybe that's what he's talking about.

But the BOM is still there, changed from fffe to ffef, and the nulls are on the odd byte instead of the even byte. You can see it in hexl-mode.

Sample xml file. The encoding attribute directs the encoding when emacs saves it in xml-mode or nxml-mode. A future version will be patched to check the BOM first.

<?xml version="1.0" encoding="UTF-16"?>
<hi />

It looks like Emacs takes UTF-16 as UTF-16BE, while Windows takes it as UTF-16LE (BE and LE don't work in Emacs for the encoding attribute). The encoding attribute is probably key to the problems here.

Saving it in powershell will convert it back to utf-16le.

[xml]$xml = get-content test.xml; $xml.save('test.xml')

With encoding="UTF-16LE" and encoding="UTF-16BE", the bom is removed, making the file unrecognizable in emacs. This is a confirmed bug that will be patched: http://lists.gnu.org/archive/html/bug-gnu-emacs/2019-05/msg00892.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.