Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I would like to convert (an) ANSI-encoded text file(s) (Win 1252) to UTF8 without BOM, ideally via a command-line call. My use-case: I export .tex files from Stata which I want to compile with LuaLaTeX. Stata apparently does not support UTF8, LuaLaTeX does not support anything but and thus chokes on some non-ASCII characters. From inside Stata I can call shell commands, so it would be nice if I could do the conversion on the fly from within my Stata scripts.

So ideally I would like to be able to call a command like e.g. convert2UTF.cmd file.tex. Another good option would be some batch conversion of files within a folder (e.g. convert all files with *stata.tex). In addition it would be great if the solution would work with default Windows tools (minimum Win 7, even better XP).

Similar questions have been asked here before. The Cygwin/GnuWin32 approach is problematic since I would like to be able to convert without having to install extra software on a machine. The powershell approach looks promising, but apparently out-file -en utf8 saves the file with BOM.

Another powershell approach that seems to convert to UTF8 without BOM is

foreach($i in ls -recurse -filter "*.*") {
    if (
        $i.Extension.ToLower() -eq ".tex"
    ) {
        $MyFile = Get-Content $i.fullname 
        [System.IO.File]::WriteAllLines($i.fullname, $MyFile)

Unfortunately I cannot figure out how to run it. I saved it as a powershell script into the same folder as the .tex files, but when I run it, it does not touch them. So apparently there is something missing. Needless to say that my powershell knowledge is close to nothing. Also, I would like to pass a filename as an argument, when calling it from Stata.

share|improve this question
There may be a way to use the powershell approach and not write the BOM. See Using PowerShell to write a file in UTF-8 without the BOM. – martineau Feb 27 '13 at 13:04
@martineau: That is quite similar to the code I posted, right? But how do I run it? E.g. when I paste [System.IO.File]::WriteAllLines(out.tex, $MyFile) to the powershell, I get a ParserError... – dapperdan Feb 27 '13 at 23:26
Yes, it's similar, however without seeing the exact ParserError you're getting it's hard to say exactly what's wrong -- my guess is you're passing the wrong arguments to WriteAllLines or passing them in the wrong order. – martineau Feb 27 '13 at 23:59
@martineau: Ok, I post $MyFile = Get-Content in.tex followed by [System.IO.File]::WriteAllLines(out.tex, $MyFile) and I get CategoryInfo: ParserError: (CloseParenToken:TokenId) [], ParentContainsErrorRecordExceptionFullyQualifiedErrorId : MissingEndParenthesisInMethodCall – dapperdan Feb 28 '13 at 0:16

I think you may try to use VBS script and ADODB.Stream object.

Google search: "vbs convert file ansi to utf-8"

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .