Questions tagged [utf-8]
For discussions about UCS Transformation Format with 8 bits (UTF8). It is an encoding table that represents all characters in the Unicode character set.
278
questions
0
votes
0
answers
71
views
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc when using pipenv
I have the following problem while trying to set up a virtual environment with pipenv using Python 3.12.5 (pipenv, version 2024.0.1) under Windows 11.
While this might sound like a question for Stack ...
- 1
1
vote
1
answer
142
views
How to programmatically disable or enable "Beta: Use Unicode UTF-8 for worldwide language support" in Windows 10 and Windows 11?
This Super User answer explains well what the feature is, but what matters is that it can be enabled using a checkbox in a subwindow of intl.cpl:
How can I accomplish this programmatically?
1
vote
1
answer
34
views
Why are UTF-8 paths (differently) escaped in outputs across core CLI utilities?
Let's say that I'm a psychopath who names my files like:
[-] `{title: "Non-Metadata", id: "s4a4ji"}`{.JSON5}.dir
That's my best attempt at making the most hideous plausible ...
0
votes
1
answer
83
views
Some UTF-8 characters not being displayed properly in Windows
In some of my applications in windows, I'm don't see some of the UTF-8 icons being rendered properly.
An example is this url in chrome browser which should display a color-palette icon like this: , ...
- 561
0
votes
1
answer
81
views
Batch command with quotes fails to move files and delete folder with special characters
I have a custom batch program that I update and run to negate having to constantly waste half an hour just to clean up the Windows start menu every time I'm forced to reinstall Windows. It saves me a ...
- 2,167
0
votes
0
answers
23
views
Cannot create Neovim keybinding with special character "ö" in Tmux
I have a Swedish keyboard, and I want to bind <C-ö> to a special action in Neovim.
This works perfect when I open Neovim outside of Tmux, but when I open it in Tmux, it doesn't do anything. It ...
- 101
0
votes
1
answer
319
views
UTF-8 Decoders fail to decode the encoded strings
I have some encoded values values which I believe is UTF-8. Now I dont really know if it is UTF-8 or not because other online tool and steps to decode UTF-8 is not working, BUT an open source tool ...
- 3
4
votes
2
answers
5k
views
How to convert UTF-8 to ANSI
(note that I posted a previous iteration of my question in stack overflow)
Hello there, I am trying to change multiple files from UTF-8 to ANSI in Notepad. My goal is change them to ANSI like my ...
- 41
0
votes
3
answers
810
views
Is it an ANSI or UTF8 file?
I edited the following batch file in notepad. At the bottom-right corner of notepad, it showed "UTF8". I saved the file in ANSI format.
Now, the bottom-right corner of notepad showed "...
- 297
0
votes
0
answers
288
views
Managing utf8 CSV on Mac
I use Excel on macOS and when I try to combine CSV files downloaded from SaaS platforms, it messes things up on the first lines. I don't have this problem when using Excel with Windows.
For now, I ...
- 1
3
votes
0
answers
313
views
ffmpeg printing unknown glyph between characters on utf-8 subtitle
FFmpeg is printing unknown glyphs between some two characters but the weird thing is it's not replacing a character that doesn't exist in the font, it's just printing a new unknown glygh and I can't ...
- 1,663
0
votes
1
answer
461
views
gnome-terminal does not display UTF-8 characters correctly (zsh)
I installed zsh v5.8.1 on Ubuntu 22.04.3. And my gnome-terminal looks like so:
the symboles do not seem to be right. Whats wrong?
- 113
0
votes
1
answer
422
views
cli program `less` interprets a file as binary when it is utf-8 text
I have a file that has non-ascii, UTF-8 characters in it. When I use less to view that file I get a warning saying may be a binary file. See it anyway? But the file is clearly not a binary. And when ...
- 205
1
vote
1
answer
119
views
How to define command line options in my sql systemctl service
I have a mysql install on Digital ocean cloud.
And by default it's working with system ctl service.
I need to run the server with those options :
That have taken from here.
So I've tried to edit this ...
- 121
1
vote
1
answer
280
views
Problems with Word link to Excel when Windows is set to use UTF-8
I've some Word documents that have fields that are automatically updated from an Excel file using Words built-in links to Excel (as in copy > paste special > paste as link).
After setting up a ...
8
votes
4
answers
14k
views
Convert UTF-16 LE to UTF-8 in windows via command line
(question re-written to be more useful)
I have a batch script which will interact with command line programs, take their output, and then perform decisions based on that output.
One of the programs I ...
- 103
2
votes
0
answers
418
views
create zip format whose AES password is UTF international letters
for reason beyond me we want to create zip format encrypted with AES-256, whose password can be International, on Linux.
If we use 7-zip, I found it works when creating 7z file, ie the following works:...
- 123
3
votes
2
answers
3k
views
rsync "File name too long (36)" error for filenames with UTF-8 characters
Trying to copy files from an NTFS-formatted HDD to an EXT4-formatted HDD, rsync fails to copy some of them having relatively long file names consisting of Asian (UTF-8) characters.
For example, there'...
- 33
1
vote
1
answer
278
views
Function to calculate length of a string in Google Spreadsheets when the string contains certain UTF-8 characters
This function returns 6 and it's expected to return 4:
=LEN("🥎A🥎B")
What function should I use to get the "real char count" expecting that one symbol in the UTF-8 icon map is 1 ...
- 643
3
votes
1
answer
12k
views
My text file is riddled with question marks. How can I make it readable?
When I open one of my text files in Visual Studio Code,
the text contains a lot of question marks where I had expected to see
Swedish letters, such as å, ä, ö :
^ click to enlarge
Down to the right (...
- 1,051
0
votes
1
answer
2k
views
Windows Server 2019 and File Encoding
I have run into an issue with Windows Server 2019 with saving files as UTF-8 without a BOM.
If you create a simple file in Notepad on Windows Server 2019, and save it as .xml with UTF-8 (there is no ...
0
votes
1
answer
137
views
How does thetext decoder knows which text encoder is used to encode?
When I create a text file on Windows operating system using Notepad and hit save, I get to choose the text encoding.
Suppose I saved this file using UTF-16 LE encoding and later sent this file to a ...
- 101
0
votes
1
answer
959
views
How to download a link that contains UTF-8 coding with WGET
I use wget (Windows) to download webpages. On a site all works well but some webpages return a zero file. Here are commands i execute The first loads well with WGET, the second returns an empty file
...
0
votes
0
answers
2k
views
"CocoaPods requires your terminal to be using UTF-8 encoding" in SSH session from Windows to macOs
I need to run pod install on macOS 12.4 through ssh remotely from a Windows 11 machine.
To do this, I ssh into the macOS machine through cmd. However, when I run pod install in the secure shell, I get ...
- 243
0
votes
2
answers
970
views
How to correctly decompress a file with ANSI coding so that the output file is with UTF-8 encoding? [duplicate]
I have a .zip file which is a compressed folder and has ANSI encoding (I think, because I opened it with Notepad and it said that it has ANSI encoding). The file (folder) contains other files which ...
- 1
0
votes
1
answer
4k
views
Locale Setting Issues on Arch Linux
I am running Arch Linux. My locale appears to be incorrectly set, but I don't know how to fix it. When I run locale, my system outputs the following:
locale: Cannot set LC_MESSAGES to default locale: ...
- 1
0
votes
1
answer
884
views
Does FTP server supports UTF-16 encoding?
I am checking my application for different encoding techniques. In one such use case related to the FTP server, I do not see anything related to UTF-16. This means I can able login to the FTP server ...
0
votes
0
answers
312
views
UTF8 characters not displayed in browsers both Firefox and Google Chrome
I use Linux Debian and everything used to work fine.
After upgrading to the newer 'bullseye' distribution I decided to compile my own kernel. All went well and I thought everything was working until ...
8
votes
0
answers
22k
views
Can I enable Unicode UTF-8 WorldWide Support in WIndows 11, but set another encoding for a specific app?
generally, I like to enable UTF-8 worldwide support in Windows 11, because most of apps I use support UTF-8 and I am using Linux OS with UTF-8 at the same time.
But the problem I had so far is one of ...
- 1,075
0
votes
0
answers
570
views
when running "LC_ALL=C.UTF-8 egrep -axv '.*'" to detect non-UTF8 characters how can I determine the exact character that caused the detection?
I frequently use this command:
LC_ALL=C.UTF-8 egrep -laxv '.*' filename
this tells me if the file contains any non-UTF8 characters (actually I usually use this in conjunction with find to scan many ...
- 165
1
vote
1
answer
5k
views
Open UTF-8 csv files by double-clicking in Excel
I want to open UTF-8 coded csv- or text-files directly in Excel 2016 or later.
Of course I am aware of the import option, but I want to do this with a simple double-click.
Problem is that this way ...
0
votes
1
answer
4k
views
Simplified Chinese characters are not displayed correctly in Windows command prompt
When I enter or paste a simplified Chinese character, it is displayed weirdly. It looks like a 'p' character enclosed in a rectangle. Please see the picture attached.
But I copy the character and ...
- 2,115
0
votes
1
answer
1k
views
How to convert this file into PNG
I have a file with no extension (as far as I can tell) that consists of special symbols. (File: https://drive.google.com/file/d/1h9MuUDfDtGEneEzZMNKLKWwY7JjZlBTS/view?usp=sharing)
Here're a couple of ...
- 3
1
vote
0
answers
333
views
Outlook password encoding for email services (IMAP, SMTP)
I’m trying to make sense of how different mail clients log onto IMAP and SMTP
services. There are the normal ones and then there is Outlook.
Outlook seems to unconditionally encode passwords as Latin-...
- 195
1
vote
0
answers
38
views
Bash how to printf UTF-8 [duplicate]
Here is a small script to reproduce the default:
printf "|%-32s|\n" "Here, winter and there, summer"
printf "|%-32s|\n" "Ici, l'hiver et la, l'ete"
printf "...
- 193
0
votes
1
answer
967
views
Printf in gawk with the correct encoding?
I'm wondering: can gawk printf in any format besides ASCII?
Currently, I'm using gawk match() to search through some UTF-8 text. When I go ahead and print out the matches gawk finds, it ends up like ...
- 1
2
votes
2
answers
15k
views
How can I perform an ISO-8859-1 to UTF-8 text file conversion while not changing any characters that are already valid UTF-8
Consider a text file that looks like this visually:
This’s ISO-8859-1
This’s UTF-8
Behind-the-scenes, the ’ curly quote character in the first line is encoded as ISO-8859-1, and the same ’ character ...
- 165
2
votes
0
answers
2k
views
Downloding files via SFTP on MobaXterm gets stuck when filenames are in Hebrew
When I try to download files whose names Hebrew characters, the download gets stuck and shows an error:
Error EInOutError: I/O error 123
How do I fix this? I don't want to use another FTP client, I ...
- 944
0
votes
0
answers
113
views
Unicode Error when trying to use Command Prompt with USB Flash drive or SD Card
I have problems when trying to use unicode characters like Russian characters (Cyrillic) on the Command Prompt (CMD). This problem only happens when I am on USB Flash drive or SD Card, which use FAT32....
- 1
2
votes
0
answers
897
views
Loading UTF8 CSV files generated on a Mac into Excel on Windows: does not recognize encoding
When I generate a UTF8 CSV file in Excel (or some other way) on Windows and then open it n Excel, I expect it to ask me to choose the right encoding.
I got a UTF8 CSV file that was generated on a Mac. ...
- 489
11
votes
7
answers
12k
views
Why does Unicode have big or little endian but UTF-8 doesn't?
UNICODE uses 2 bytes for one character, so it has big or little endian difference. For example, the character 哈 is 54 C8 in hex.
And its UTF-8 therefore is:
11100101 10010011 10001000
UTF-8 uses 3 ...
- 3,227
0
votes
1
answer
8k
views
MobaXterm ssh console not showing correct character in tree command
When in Linux SSH box, using MobaXterm, I enter tree command, the output is:
âââ jboss
âââ gc.log.0.current
âââ server.log
which should be:
└── jboss
├── gc.log.0.current
└── server.log
In ...
- 11
19
votes
1
answer
2k
views
Hyphen is not rendering in Notepad++
I'm sure this is an encoding issue, but I can't figure it out.
I exported a spreadsheet from Excel as a UTF-8 CSV. This produced a CSV in the UTF-8-BOM character encoding. Opening this file in Notepad+...
- 343
0
votes
1
answer
824
views
How to convert binary input into utf-16 characters on bash
I have bunch of bytes that has random bits in order and some of them 9 and 10. I can convert the 8 bit binary data into characters with a perl script like below.
perl -lape '$_=pack"(B8)*",@...
- 15
2
votes
1
answer
437
views
How to get a file list as utf8 encoded string into gnuplot?
I need the file list of a directory as utf8 encoded string in gnuplot.
In gnuplot you can make a system call via system(), which (to my limited understanding) just calls cmd.exe and will carry out the ...
- 155
2
votes
1
answer
3k
views
Converting excel files to UTF-8 .TSV
Every week I have to convert 25 or more Excel files to UTF-8 .tsv files. As of now I'm doing it using Editplus application manually.
I was searching Google and other sites where I'm getting UTF-8 ....
- 151
0
votes
2
answers
2k
views
Which text file encoding should i use to be compatible between Windows and Linux?
If i want text file (in this case .srt subtitle file) or just plain text file with various characters including ěščřžýáíéůú (WINDOWS-1250) in general to be working well both on Windows and Linux, ...
- 622
2
votes
1
answer
1k
views
Clipping special characters to clipboard using command prompt
I want to place a certain useful special character to the clipboard to use it. So I did this using a simple batch script, something like this:
@echo off
set /p char=<character.txt
echo %char%| ...
- 534
4
votes
1
answer
683
views
Regexp fails to match UTF-8 characters in Notepad++
Given the following UTF-8 text in Notepad++:
🤣 1
☺ ☺️ 2
😊 3
Both ^[^ ]+ and ^[\S]+ match only the second line.
The same expressions matches all 3 lines in Visual Studio 2019.
What am I missing?
- 647
2
votes
0
answers
301
views
Letting IMAP server know that Alpine is fine with UTF-8
I am using Alpine with a local IMAP server. For some messages, I get the following "advisory message" in Alpine:
***********************************************************
...
- 213