Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [character-encoding]

A character encoding system consists of a code that pairs each character from a given repertoire with something else — such as a bit pattern, sequence of natural numbers, octets, or electrical pulses — in order to facilitate the transmission of data (generally numbers or text) through telecommunication networks or for data storage.

0
votes
1answer
19 views

Which type of character encoding consumes the most memory?

I will store a long (45132 character) string in a Postrgres database whilst preserving every character (including really rare ones). Postgres can store strings up to 1GB (see here). In terms of the ...
2
votes
1answer
36 views

What text encoding is used in “Zip archive data, at least v?[0x314] to extract”?

I tried multiple search terms on the usual suspect search engines as well as on stack overflow and superuser, but can't find anything on this. Normally a Zip archive magic pattern is "at least v1.0" ...
1
vote
2answers
41 views

FTP File transfer With Chinese Characters

When I'm trying to send files with Chinese characters over ftp, I am getting: 500 ERROR (Internal Server ERROR) through my backend code; It's working fine for english characters but not for ...
0
votes
1answer
44 views

Exporting from Excel to CSV replaces Japanese characters with ??? even though Windows, Office locale is Japan/Japanese

I am exporting an excel file (Excel 2016) containing Japanese characters into CSV. (Note : I am not exporting to CSV UTF-8 provided). In the process, all Japanese characters are replaced with '?' My ...
1
vote
0answers
14 views

Windows 10 OpenSSH - Character Encoding

Is there any setting on client-side such that I get the German Umlaut display correctly when connecting from my Win10, 64b, English to Linux servers by using Windows 10 OpenSSH client? While I like ...
0
votes
1answer
20 views

PuTTY not showing character | and _ correctly in pstree and tree command output

On Linux, tree and pstree commands show tree-like hierarchy of directories and processes respectively. Their output usually includes the characters | and _ which are used to draw the tree structure. ...
0
votes
0answers
30 views

Interpret text file with some hex codes?

I have a file with contents looking like PK\u0003\u0004\u0014\u0000\u0006\u0000\b\u0000\u0000\u0000!\u0000À¸<91><91>¢\u0001. However, I have a different version of the same file looking ...
2
votes
0answers
43 views

Execute sh file with Russian or Chines Chars - Saved as UTF-8 or Unicode

I have a file which has some russian chars in. Below is the content of by sh file. #!/bin/sh sed -i "s/\bVAR1\b/Привет, как ты/g" file1.txt When i save this file i had to save this as UTF-8 or ...
0
votes
0answers
39 views

Docker container in Windows Bash ignores non-ASCII characters, work's in PowerShell

I'm running a Python Docker container on my Windows 10 computer. If I add Norwegian specific characters in my Python code, such as print("båt"), and run the container in Windows Bash, the output is bt....
0
votes
2answers
166 views

TXT files: how to switch from weird characters back to normal?

So, I have on a flash drive a txt file generated in Cyrillic (my own work, own pen drive), a few years old. Now I needed to open it, only to see this kind of mess. I wonder why is this happening and ...
0
votes
2answers
66 views

What is causing these two apparently identical files to have different hashes?

I am unable to figure out why the following two files are yielding different hashes (SHA1, CRC32, SHA384, whatever): https://cdn.jsdelivr.net/npm/jsonify-error@1.2.1/dist/jsonify-error.js https://...
0
votes
0answers
19 views

mlterm not showing Korean Hangul alphabet

On this Lubuntu 16.04, LXTerminal can print any Latin and Hangul symbol, but mlterm shows Latin characters only. How can this terminal (version 3.5.0) be parameterized to display also Hangul? (I mean, ...
0
votes
0answers
18 views

Issue with character set after installing two os

I had Windows 10, there I had text file with cyrillic characters. Recently I reinstalled Win 10 as well as installed Ubuntu 16.04. Now when I open that file on both systems, I see some unreadable ...
0
votes
0answers
43 views

What encoding is needed to read this HTML file containing Spanish characters?

I downloaded this HTML file from archive.org, which is a digitilised book, written in Spanish. As such, it has plenty of accents and other unique symbols (e.g. ñ). Now, when I open the html file in a ...
2
votes
2answers
473 views

Why when copying a non-English URL from address bar it's URL encoded and not as the text I see (decoded)?

When I copy a link from the address bar that has characters that needs to be encoded usually, the copied link will contain the encoded text and not the decoded (which is what's visually seen). But if ...
0
votes
0answers
28 views

Special characters in filenames were replaced by underscores CentOS

I've trouble with special characters in filenames of foreign languages. When I generate a filename with special characters, such as 'araç temizliği', the special character is replaced by an underscore ...
0
votes
0answers
308 views

Windows RDP accent and special characters

Windows 10 RDP behaviour seems to have changed post April update. Same hardware, same mouse, same OS, same everything. I'm connected to a remote session (Windows Server 2012 R2) and am experiencing ...
3
votes
1answer
2k views

Proper encoding for file names in zip archives created in Windows and unpacked in linux

I have problems with different charsets in Windows and Linux (Centos). I have files with special characters in their filenames from many different languages. The zip archive is generated under Win7 ...
1
vote
1answer
47 views

Command output not working after changing codepage

I've found a rather weird issue while investigating the twisted world of character encoding. In Windows, if I type 'tree' the command works as expected but if I then type 'chcp 65001' (which is UTF-8) ...
0
votes
0answers
17 views

Can't open content records with Turkish characters

I am learning Turkish and I might want to have the capacity to open Turkish content documents. I am running Windows 7 and I have even introduced the Turkish show dialect for Windows. In any case, I ...
2
votes
1answer
253 views

How to convert this string to Japanese using GNU/Linux tools?

Here is a string from a text file: @™TdaŽ®Æ‚êƒ~ƒNƒXƒgƒŒ[ƒgEƒrƒLƒjver1.11d1.d2iƒrƒLƒjƒ‚ƒfƒ‹ver.1.1³Ž®”z•z”Åj It includes many nonprinting characters and is copied here: https://pastebin.com/...
0
votes
0answers
40 views

Strange/weird characters in file

I got a file where there are a lot of strange characters. I'm trying to find out how to "decode" or change it to plain text, but am stuck and can't find a way to do it. One line could look like this:...
1
vote
1answer
194 views

PDF displayed with Shifted Characters in Google Chrome

Observe this PDF located here, in Google Chrome specifically: http://www.saab.org/actuary/exam3/3-sample.pdf Notice how many of the characters appear to be shifted (this is most noticeable toward the ...
1
vote
1answer
618 views

Why does LibreOffice Calc change CSV file Character Set?

I have a .csv file export (actually printed to csv file) made on a Windows 10 machine. Working in Libreoffice Calc on Archlinux, Why can Calc open a file using settings Character Set: Western Europe (...
0
votes
1answer
28 views

Symbols are lost after conversion from UTF-8 to ISO8859-1 and back to UTF-8

I have a file with properties in French. I would like to convert it to ISO8859-1. But after conversion, some symbols are lost. What is wrong? > cat fr.properties VAR2="élément n’a" > cat fr....
0
votes
1answer
141 views

Resetting Unicode mappings in PDF text

PDFs often contain fonts without explicit mappings to Unicode, preventing us from extracting correct text from them -- curse you, Adobe! I need to process PDFs in a batch fashion on a Linux system. I ...
0
votes
1answer
465 views

MS Access won't display line breaks of existing data

I am creating an MS Access front end for data from somewhere else. In this case, it's a linked SQL Server table. The data is actually imported from a shared-hosting LAMP web server. It's the ...
0
votes
1answer
47 views

Keyboard layout and displayed characters

What does happen when one switch keyboard layout (alt+shift in Windows) for e.g. from en to pl. Does it change the charset from english to polish or maybe encoding? Why when for e.g. setting it to en ...
0
votes
1answer
333 views

Keyboard layout and charset encoding

How does keyboard layout (switched in Windows for e.g. by Alt+Shift) relates to character encoding? How to determine what charset encoding is currently used in Windows? How to know what charset ...
0
votes
1answer
365 views

Encoding problems when copy pasting from a remote desktop session

I connect to a distant machine to copy some arabic text to my machine. When connecting through teamviewer, I get the text nice and clean when pasted on my machine, like this : تدخر جهدا في سبيك ...
1
vote
0answers
358 views

Strange character render issue browsing Google/Youtube through Firefox/Chrome

I am having issues in both Firefox and Chrome browser on Windows 10. When you browse to www.google.com it just render something like this (I took that img from Google Image not an actual screenshot ...
1
vote
1answer
384 views

How to fix encoding - curly apostrophe appears as ‰Ûª

I have a text-file in which all the ASCII characters appear correctly but some others do not. In particular there is this word: don‰Ûªt In hex the bytes are 64 6f 6e 89 db aa 74. Obviously, it is ...
0
votes
0answers
113 views

character encoding - importing HTML into Microsoft Excel

I exported my Google Chrome (browser) bookmarks as an HTML file and then imported that file into Microsoft Excel. After importing, the text looks bad in Excel. E.g., the word "don't" in the HTML ...
2
votes
0answers
61 views

Find encoding that encodes a given character to a given sequence [closed]

I am trying to find out which encoding uses some text file I have received. I know that this encoding encodes é to 0x8E. Specific question: which encodings possibly correspond? General question: how ...
0
votes
2answers
489 views

Binary encoding formats

echo random text > text_file Saves text_file in text format with ASCII encoding. To check the encoding, I do chardetect text_file which tells me that the file is ASCII encoded. Now I have a jpg ...
-1
votes
1answer
98 views

Opening a 7z file on mac shows strange encodings

I am opening this file: https://dataverse.harvard.edu/file.xhtml?fileId=2901965&version=1.0 On TextEdit. When I do, the file looks like: 7zºØ'· t(ù‚]%N ò„,¢‡ò] ¬CRJ∫√ıìaIÄÅ≤íhYTÔ1/ıG3À=ôN’(...
1
vote
1answer
359 views

Keeping non-ASCII characters in folder and file names between Mac and Windows

When I create folders with en- or em-dashes on a Mac, I'd like for them to still be the same when I hand them to my friend who has a Windows 10 PC. When I create them, this is how they look on my ...
0
votes
2answers
88 views

Control I characters in my text file

I have no idea how did I suddenly get ^I characters in my source code file. Everywhere I had a space ( but only in the beginning of the line ) , it shows up as ^I. This is making my code very very ...
0
votes
0answers
282 views

How can I make `file -i` command return ISO-8859-9?

Let's say I have a text file with content şşş and in vi/vim I have set fileencoding as :set fileencoding=ISO-8859-9. Later when I check the encoding with file -i <myFile> it gives text/...
2
votes
0answers
45 views

Arduino IDE app menus are gibberish

The menus of the Arduino IDE are full of gibberish on this system. This only affects the menus at the top and all context menus. When I manage to weasel my way through and open, for example, the ...
1
vote
1answer
62 views

Why are smileys not displayed in webmail?

A friend of mine asked me why he cannot see smileys that people send him in a mail. He can see them in Mail on an iPhone, but on webmail on his MacBook, he sees codes like \uD83D\uDE1E that you can ...
111
votes
3answers
116k views

How do I change the character encoding for a webpage in Chrome?

Google Chrome, like almost every web browser in recent memory, used to have an option to change the character encoding for the webpage being viewed by going to Menu › Tools › Encoding (or some similar ...
6
votes
1answer
2k views

LSEP appearing on a specific web page, is there something I can do to Chrome to fix this?

When visiting This site in Chrome an LSEP symbol appears instead of the Line Separator (UTF-8) character: When I visit the site in Edge it looks like this: Is there a way to fix this for the Chrome ...
0
votes
1answer
5k views

Internet Explorer 11 not displaying UTF-8 encoded pages correctly

Some pages in IE 11, that seem to be UTF-8, are not displaying correctly. The same page displays correctly in Mozilla Firefox. I first noticed this in Amazon.com on the Orders page (pretty much any of ...
0
votes
1answer
2k views

FileZilla: preserve UTF-8 encoding when transferring to Linux

Is there a way to set FileZilla to preserve the existing encoding of my text files, which are UTF-8, when transferring from Windows to a Linux server? In the FileZilla Site Manager, I have selected ...
0
votes
0answers
25 views

Is there a way to pre-define a character set (like utf-8) for a .txt file you're opening w/ ooo calc, instead of having to select it?

I have a php script that spits out .txt files, and I was wondering if I could add something to define the character set that ooo calc would read/see when opening so that ooo calc will know ...
0
votes
0answers
399 views

Libreoffice special character appears as question marks

I'm trying to use a .csv sent to me by another individual. When I look inside at the data, special characters such as quotation marks, hyphens and apostrophes appear as question marks or special ...
0
votes
1answer
132 views

small omega in notepad/ notepad++ for autohotkey

I am trying to write an Autohotkey using notepad++ (also tried with notepad) but turned out it was not possible in default settings. I looked on internet and learned that ISO 8859-7:2003 and UNICODE ...
1
vote
1answer
734 views

OneNote backslash character codes?

Where do I find a reference page for oneNote which lists all the character codes for symbols you can type using back slash on one note? E.g. you can type \in and you get the set notation symbol ∈. ...
0
votes
0answers
92 views

The font 'AppleColorEmoji' contains bad / Widths

I'm trying to add an emoji in my email signature. I've copy/pasted if from a site listing all emojis. It displays fine in all the clients I've tried to open my emails with: But, when I try to save my ...