-1

I've inherited a big collection of web content claiming to use utf-8 encoding, yet clearly random gibberish that shows up when displaying the pages says they are some sort of windows encoding. I've used the command iconv -f Windows-1252 -t utf-8 *19* -o test.htm to change one file, and now I see valid characters that make sense, but in front of each of those characters there is now a 'Â' uppercase A with a circumflex. Examining the binary data shows that all the special characters are hex pairs such as C292 or C297. Is there some windows charset that uses C2 as an escape and I want something different than Windows-1252 in the iconv command?

6
  • 1
    It might help if you posted the header portion of the html file and a segment of the text in question. Jan 28 at 3:38
  • The file command, at least under unix, can sometimes show the charset, as an heuristic. Also "now I see": how do you look at the file? which tool? Jan 28 at 18:44
  • Take a look at the <meta> tags. Does any of them actually declare the UTF-8 encoding? (I don’t understand why this was migrated off Webmasters, it should have been perfectly on-topic there.) Jan 28 at 22:16
  • Wait, do you get C2 92 in the source or in the output? Jan 28 at 22:26
  • Don't add info as an answer. I suggest reading the Help section to see how this site works. We're a QA site, not a forum, and the difference is important. Use of the edit button is highly recommended. Jan 30 at 2:02

1 Answer 1

-1

I have a feeling the files were produced by someone using utf-8 escape sequences to embed Windows characters as though that made sense.

Doing the above iconv followed by a sed command to simply eliminate the 0xc2 characters left in the files after conversion created files which seem to display correctly now.

I did scan for legitimate instances of 'Â' which might be present, but I'm pretty sure there weren't any accented characters at all.

2
  • How does this answer the question. Jan 28 at 20:55
  • Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Jan 28 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.