I have a text file in UTF-8 containing somewhere a Unicode character that is not present in CP1250*. I want to save the file as CP1250 but both Gedit and Leafpad fail to do so. I do not know which character is actually causing the problem: there is no way to use Ctrl+F to find “characters present in this file which do not exist in cp1250”. The document is long. How can I find the offending character? I know that I could always delete half of the text and proceed with the half that still cannot be saved, but that seems a bit cumbersome. Is there a better way?

* Perhaps, for example, the the file contains ɸ (U+0278), a character which does not exist in CP1250.

  • In the edited form, this question is about finding a tool that can convert from one character encoding to another and report or process specially those characters that cannot be represented in the target encoding. This is a rather broad question. Such tools exist both as standalone and as parts of other software, such as text editors. – Jukka K. Korpela Aug 5 '14 at 11:32
  • Well, what are those tools? I don't even know how to use Libreoffice to save in non-utf-8 (I would have to change locales I guess), simple text editors (or vim even) just throw an error without specifiing the culprit. – sup Aug 5 '14 at 11:38

Cut and paste the character “ɸ” e.g. from here into the Search box of your editor and search for it. Then you can delete it from the document.

However, just deleting a character is usually not recommendable. It is often better to replace it by U+001A SUBSTITUTE, indicating character-level data error, or (depending on the context and application) by a question mark “?” or some escape notation (like “U+0278” for “ɸ”), so that a problem is signalled to software or people who read the file. If “ɸ” is used properly in the original text, it is a phonetic (IPA) character used in the description of the pronunciation of a word. Hence, just omitting it would distort the information.

  • Ah, I worded the question wrongly. What can I do if I don't know the offending character? "ɸ" was just a n example. (The most often this happens with "…" and "..." typographical quotation marks against standard quotation marks. – sup Aug 5 '14 at 9:23
  • Please clarify this by editing the question. And note that CP-1250 alias Windows-1250 does contain the typographical (curly) quotation marks; it ISO counterpart, ISO-8859-2, does not. I had no difficulty in saving a page with “…” in Windows-1250 in Gedit (on Windows 7). – Jukka K. Korpela Aug 5 '14 at 10:45
  • Clarified. Yeah, with some characters it works, but not with all. – sup Aug 5 '14 at 10:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.