Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I Have text which looks like this:

supposed to undergo yearly cardiac exam in order to stay on transplant list. But, there are > patients who are missing important cardiac information. It is yo ur job as an intern on call to  make sure that you fin

As you can see, the first line is fine, but then the second line is corrupt. It looks like this even when I open it using Vim or LibreOffice. Is there a way to fix this? I've tried changing the encoding to UTF-8 but to no avail. Thanks!

share|improve this question
What is it a plain-text file? Word? PDF?… – Synetech Jun 18 '12 at 1:06
How much text like that do you have? – lornix Jun 18 '12 at 1:23
It's like that in a plain-text file. Even when I open it with Vim, LibreOffice, GEdit, you name it, it looks like that. There is a lot of text I need converted, which is what's stopping me from just typing it out. – oort Jun 18 '12 at 1:26
I was pleased to see you cut-n-paste to put it here, as I was able to tinker with it. Seeing the raw text file would be more helpful. Since it seems to be a mixture of regular ASCII (or UTF-8) and wide-chars, i imagine a 'filter' could be ginned up to fix it. – lornix Jun 18 '12 at 1:30

It looks to me like it may have started life as some kind of standard text with substitution markers, and was processed by a utility that didn't have enough defensive coding in it - a variation on this xkcd comic. It might, for example, have been generated using a buggy XSLT processor.

If that's what you see as plain text, there's probably not much you can do other than go back to the original source. Plain text files don't contain extra hidden information. A wrong encoding can cause problems, but if your editor loads assuming a byte-per-character encoding and that encoding is wrong you should still see more stuff in the form of garbage characters.

There's an off chance that there may be an end-of-file or null character preventing the tail end of the text from being shown, but no halfway decent text editor should be fooled by that these days.

share|improve this answer

Remember to always work on a backup copy until you're sure it works.

You could try loading the text file into vim, while forcing a different encoding method

start vim (or gvim, whichever)

:e ++enc=utf-8 textfilename.txt

vim knows several encoding types, try this repeatly cycling through them. From the Help Files:

Supported 'encoding' values are:

latin1, iso-8859-n, cp437, cp737, cp775, cp850, cp852, cp855, cp857, cp860, cp861, cp862, cp863, cp865, cp866, cp869, utf-8, ucs-2, ucs-2le, utf-16, utf-16le, ucs-4, ucs-4le

My question about the raw text file is to determine if it has a BOM at the beginning (Byte Order Mark), which would give us a hint as to what it thinks it is.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.