Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [character-encoding]

A character encoding system consists of a code that pairs each character from a given repertoire with something else — such as a bit pattern, sequence of natural numbers, octets, or electrical pulses — in order to facilitate the transmission of data (generally numbers or text) through telecommunication networks or for data storage.

0
votes
1answer
21 views

How does notepad.exe determine character encoding?

I have a .txt file saved in UTF-8 format without a BOM. It contains an 'é' character. How does notepad.exe determine that it is UTF-8 encoded? Other .txt files containing only < 0x80 characters ...
0
votes
0answers
23 views

How to copy non-ascii bytes to clipboard in Linux?

I want to copy some non-ascii bytes to the clipboard, such as 0xff. I will then paste them as input to a CLI program in bash (gnome-terminal). I couldn't achieve this using xclip and parcellite (they ...
0
votes
0answers
16 views

How do you create custom zalgo text?

I know what zalgo text is and that there are a few websites that can make it for you. But I'm looking for how I can make it however I want." HͥAͣQͫ" is an example, how can I make it so I can choose ...
0
votes
0answers
10 views

Sublime Text 3 showing u-strings instead of accented characters [duplicate]

I have written a JSON file with encoding utf-8. When I open the file with Sublime Text 3, I see this : "meals": [ "De\u0301jeuner", "Di\u0302ner" ], instead of "meals": ...
0
votes
0answers
5 views

AQS string parameter meaning how to read this code what do I reference?

Im seeing a lot of odd URL data for m.facebook.com like 2A%2B%2C%2D%......various letters I was wondering what it means?The last three years my phones & pc were endlessly hacked by slice accounts ...
1
vote
1answer
75 views

Windows 10 All alt codes and accentuated characters replaced by �

Since this morning, all alt codes I type in (like alt 256, alt 26, alt 144, ...), are replaced by the "�" character. It also happens sometimes in Windows with accentuated characters (like é à ï ...). ...
0
votes
0answers
39 views

How to type directly with keyboard U+200C character

How to type directly with keyboard U+200C character in Notepad or other places... in Windows 8.1? (without depending to a certain font) I don't want to copy and paste ways. I don't want any ...
0
votes
1answer
27 views

Which type of character encoding consumes the most memory?

I will store a long (45132 character) string in a Postrgres database whilst preserving every character (including really rare ones). Postgres can store strings up to 1GB (see here). In terms of the ...
2
votes
1answer
63 views

What text encoding is used in “Zip archive data, at least v?[0x314] to extract”?

I tried multiple search terms on the usual suspect search engines as well as on stack overflow and superuser, but can't find anything on this. Normally a Zip archive magic pattern is "at least v1.0" ...
1
vote
2answers
109 views

FTP File transfer With Chinese Characters

When I'm trying to send files with Chinese characters over ftp, I am getting: 500 ERROR (Internal Server ERROR) through my backend code; It's working fine for english characters but not for ...
0
votes
2answers
678 views

Exporting from Excel to CSV replaces Japanese characters with ??? even though Windows, Office locale is Japan/Japanese

I am exporting an excel file (Excel 2016) containing Japanese characters into CSV. (Note : I am not exporting to CSV UTF-8 provided). In the process, all Japanese characters are replaced with '?' My ...
1
vote
0answers
32 views

Windows 10 OpenSSH - Character Encoding

Is there any setting on client-side such that I get the German Umlaut display correctly when connecting from my Win10, 64b, English to Linux servers by using Windows 10 OpenSSH client? While I like ...
0
votes
1answer
48 views

PuTTY not showing character | and _ correctly in pstree and tree command output

On Linux, tree and pstree commands show tree-like hierarchy of directories and processes respectively. Their output usually includes the characters | and _ which are used to draw the tree structure. ...
0
votes
0answers
43 views

Interpret text file with some hex codes?

I have a file with contents looking like PK\u0003\u0004\u0014\u0000\u0006\u0000\b\u0000\u0000\u0000!\u0000À¸<91><91>¢\u0001. However, I have a different version of the same file looking ...
2
votes
0answers
44 views

Execute sh file with Russian or Chines Chars - Saved as UTF-8 or Unicode

I have a file which has some russian chars in. Below is the content of by sh file. #!/bin/sh sed -i "s/\bVAR1\b/Привет, как ты/g" file1.txt When i save this file i had to save this as UTF-8 or ...
0
votes
0answers
59 views

Docker container in Windows Bash ignores non-ASCII characters, work's in PowerShell

I'm running a Python Docker container on my Windows 10 computer. If I add Norwegian specific characters in my Python code, such as print("båt"), and run the container in Windows Bash, the output is bt....
0
votes
2answers
517 views

TXT files: how to switch from weird characters back to normal?

So, I have on a flash drive a txt file generated in Cyrillic (my own work, own pen drive), a few years old. Now I needed to open it, only to see this kind of mess. I wonder why is this happening and ...
0
votes
2answers
90 views

What is causing these two apparently identical files to have different hashes?

I am unable to figure out why the following two files are yielding different hashes (SHA1, CRC32, SHA384, whatever): https://cdn.jsdelivr.net/npm/jsonify-error@1.2.1/dist/jsonify-error.js https://...
0
votes
0answers
24 views

mlterm not showing Korean Hangul alphabet

On this Lubuntu 16.04, LXTerminal can print any Latin and Hangul symbol, but mlterm shows Latin characters only. How can this terminal (version 3.5.0) be parameterized to display also Hangul? (I mean, ...
0
votes
0answers
18 views

Issue with character set after installing two os

I had Windows 10, there I had text file with cyrillic characters. Recently I reinstalled Win 10 as well as installed Ubuntu 16.04. Now when I open that file on both systems, I see some unreadable ...
0
votes
0answers
44 views

What encoding is needed to read this HTML file containing Spanish characters?

I downloaded this HTML file from archive.org, which is a digitilised book, written in Spanish. As such, it has plenty of accents and other unique symbols (e.g. ñ). Now, when I open the html file in a ...
2
votes
2answers
685 views

Why when copying a non-English URL from address bar it's URL encoded and not as the text I see (decoded)?

When I copy a link from the address bar that has characters that needs to be encoded usually, the copied link will contain the encoded text and not the decoded (which is what's visually seen). But if ...
0
votes
0answers
31 views

Special characters in filenames were replaced by underscores CentOS

I've trouble with special characters in filenames of foreign languages. When I generate a filename with special characters, such as 'araç temizliği', the special character is replaced by an underscore ...
0
votes
0answers
363 views

Windows RDP accent and special characters

Windows 10 RDP behaviour seems to have changed post April update. Same hardware, same mouse, same OS, same everything. I'm connected to a remote session (Windows Server 2012 R2) and am experiencing ...
3
votes
1answer
2k views

Proper encoding for file names in zip archives created in Windows and unpacked in linux

I have problems with different charsets in Windows and Linux (Centos). I have files with special characters in their filenames from many different languages. The zip archive is generated under Win7 ...
1
vote
1answer
67 views

Command output not working after changing codepage

I've found a rather weird issue while investigating the twisted world of character encoding. In Windows, if I type 'tree' the command works as expected but if I then type 'chcp 65001' (which is UTF-8) ...
2
votes
1answer
320 views

How to convert this string to Japanese using GNU/Linux tools?

Here is a string from a text file: @™TdaŽ®Æ‚êƒ~ƒNƒXƒgƒŒ[ƒgEƒrƒLƒjver1.11d1.d2iƒrƒLƒjƒ‚ƒfƒ‹ver.1.1³Ž®”z•z”Åj It includes many nonprinting characters and is copied here: https://pastebin.com/...
0
votes
0answers
44 views

Strange/weird characters in file

I got a file where there are a lot of strange characters. I'm trying to find out how to "decode" or change it to plain text, but am stuck and can't find a way to do it. One line could look like this:...
1
vote
1answer
234 views

PDF displayed with Shifted Characters in Google Chrome

Observe this PDF located here, in Google Chrome specifically: http://www.saab.org/actuary/exam3/3-sample.pdf Notice how many of the characters appear to be shifted (this is most noticeable toward the ...
1
vote
1answer
736 views

Why does LibreOffice Calc change CSV file Character Set?

I have a .csv file export (actually printed to csv file) made on a Windows 10 machine. Working in Libreoffice Calc on Archlinux, Why can Calc open a file using settings Character Set: Western Europe (...
0
votes
1answer
31 views

Symbols are lost after conversion from UTF-8 to ISO8859-1 and back to UTF-8

I have a file with properties in French. I would like to convert it to ISO8859-1. But after conversion, some symbols are lost. What is wrong? > cat fr.properties VAR2="élément n’a" > cat fr....
0
votes
1answer
207 views

Resetting Unicode mappings in PDF text

PDFs often contain fonts without explicit mappings to Unicode, preventing us from extracting correct text from them -- curse you, Adobe! I need to process PDFs in a batch fashion on a Linux system. I ...
0
votes
1answer
567 views

MS Access won't display line breaks of existing data

I am creating an MS Access front end for data from somewhere else. In this case, it's a linked SQL Server table. The data is actually imported from a shared-hosting LAMP web server. It's the ...
0
votes
1answer
50 views

Keyboard layout and displayed characters

What does happen when one switch keyboard layout (alt+shift in Windows) for e.g. from en to pl. Does it change the charset from english to polish or maybe encoding? Why when for e.g. setting it to en ...
0
votes
1answer
362 views

Keyboard layout and charset encoding

How does keyboard layout (switched in Windows for e.g. by Alt+Shift) relates to character encoding? How to determine what charset encoding is currently used in Windows? How to know what charset ...
0
votes
1answer
412 views

Encoding problems when copy pasting from a remote desktop session

I connect to a distant machine to copy some arabic text to my machine. When connecting through teamviewer, I get the text nice and clean when pasted on my machine, like this : تدخر جهدا في سبيك ...
1
vote
0answers
391 views

Strange character render issue browsing Google/Youtube through Firefox/Chrome

I am having issues in both Firefox and Chrome browser on Windows 10. When you browse to www.google.com it just render something like this (I took that img from Google Image not an actual screenshot ...
1
vote
1answer
444 views

How to fix encoding - curly apostrophe appears as ‰Ûª

I have a text-file in which all the ASCII characters appear correctly but some others do not. In particular there is this word: don‰Ûªt In hex the bytes are 64 6f 6e 89 db aa 74. Obviously, it is ...
0
votes
0answers
126 views

character encoding - importing HTML into Microsoft Excel

I exported my Google Chrome (browser) bookmarks as an HTML file and then imported that file into Microsoft Excel. After importing, the text looks bad in Excel. E.g., the word "don't" in the HTML ...
2
votes
0answers
64 views

Find encoding that encodes a given character to a given sequence [closed]

I am trying to find out which encoding uses some text file I have received. I know that this encoding encodes é to 0x8E. Specific question: which encodings possibly correspond? General question: how ...
0
votes
2answers
522 views

Binary encoding formats

echo random text > text_file Saves text_file in text format with ASCII encoding. To check the encoding, I do chardetect text_file which tells me that the file is ASCII encoded. Now I have a jpg ...
-1
votes
1answer
118 views

Opening a 7z file on mac shows strange encodings

I am opening this file: https://dataverse.harvard.edu/file.xhtml?fileId=2901965&version=1.0 On TextEdit. When I do, the file looks like: 7zºØ'· t(ù‚]%N ò„,¢‡ò] ¬CRJ∫√ıìaIÄÅ≤íhYTÔ1/ıG3À=ôN’(...
1
vote
1answer
387 views

Keeping non-ASCII characters in folder and file names between Mac and Windows

When I create folders with en- or em-dashes on a Mac, I'd like for them to still be the same when I hand them to my friend who has a Windows 10 PC. When I create them, this is how they look on my ...
0
votes
2answers
124 views

Control I characters in my text file

I have no idea how did I suddenly get ^I characters in my source code file. Everywhere I had a space ( but only in the beginning of the line ) , it shows up as ^I. This is making my code very very ...
0
votes
0answers
317 views

How can I make `file -i` command return ISO-8859-9?

Let's say I have a text file with content şşş and in vi/vim I have set fileencoding as :set fileencoding=ISO-8859-9. Later when I check the encoding with file -i <myFile> it gives text/...
2
votes
0answers
45 views

Arduino IDE app menus are gibberish

The menus of the Arduino IDE are full of gibberish on this system. This only affects the menus at the top and all context menus. When I manage to weasel my way through and open, for example, the ...
1
vote
1answer
62 views

Why are smileys not displayed in webmail?

A friend of mine asked me why he cannot see smileys that people send him in a mail. He can see them in Mail on an iPhone, but on webmail on his MacBook, he sees codes like \uD83D\uDE1E that you can ...
113
votes
3answers
122k views

How do I change the character encoding for a webpage in Chrome?

Google Chrome, like almost every web browser in recent memory, used to have an option to change the character encoding for the webpage being viewed by going to Menu › Tools › Encoding (or some similar ...
6
votes
1answer
2k views

LSEP appearing on a specific web page, is there something I can do to Chrome to fix this?

When visiting This site in Chrome an LSEP symbol appears instead of the Line Separator (UTF-8) character: When I visit the site in Edge it looks like this: Is there a way to fix this for the Chrome ...
0
votes
1answer
5k views

Internet Explorer 11 not displaying UTF-8 encoded pages correctly

Some pages in IE 11, that seem to be UTF-8, are not displaying correctly. The same page displays correctly in Mozilla Firefox. I first noticed this in Amazon.com on the Orders page (pretty much any of ...