The fine art of converting letters into numbers. ASCII, ISO Latin-1, Unicode, EBCDIC… There are dozens of currently used encodings and converting between them is now a witch craft.

learn more… | top users | synonyms

78
votes
7answers
5k views

Why are special characters such as “carriage return” represented as “^M”?

Why is ^M used to represent a carriage return in VIM and other contexts? My guess is that M is the 13th letter of the Latin alphabet and a carriage return is \x0D or decimal 13. Is this the reason? ...
56
votes
9answers
30k views

OS X: Base64 encode via command line

Is there a terminal command in Mac OS X which will base64 encode a file or stdin?
29
votes
7answers
33k views

Batch-convert files for encoding

How can I batch-convert files in a directory for their encoding (e.g. ANSI->UTF-8) with a command or tool? For single files an editor helps, but how to do the mass files job?
26
votes
3answers
11k views

How can I change the default encoding type Thunderbird uses when composing a new email?

Whenever I compose a new email message in Thunderbird, it defaults to the Western (ISO-8859-1) encoding, and I have to manually change it to UTF-8 every time. How do I change the default?
25
votes
7answers
45k views

How to auto detect text file encoding?

There are many plain text files which were encoded in variant charsets. I want to convert them all to UTF-8, but before running iconv, I need to know its original encoding. Most browsers have an ...
23
votes
3answers
43k views

Sublime Text 2 - Encoding UTF-8

How do I change the file encoding in Sublime Text 2? How do I change character encoding of a text file in OpenOffice.org writer? http://www.sublimetext.com/forum/viewtopic.php?f=4&t=3541
23
votes
2answers
43k views

Change default code page of Windows console to UTF-8

Currently I'm running Windows 7 x64 and usually I want all console tools to work with UTF-8 rather than with default code page 850. Running chcp 65001 in the command prompt prior to use of any tools ...
18
votes
7answers
14k views

How can I create a zip / tgz in Linux such that Windows has proper filenames?

Currently, tar -zcf arch.tgz files/* encodes filenames in UTF, so Windows users see all characters spoiled in filenames which are not english, and can do nothing with it. zip -qq -r arch.zip files/* ...
14
votes
1answer
1k views

What is this '°͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌' strange character?

I saw this °͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌͌ strange character online. I also noted that: It needs 26 backspaces to delete but behaves like one character when selecting. It is drawn vertically covering ...
14
votes
1answer
384 views

EGit (Eclipse) wrongly interpreting file names with non-ASCII characters?

I recently switched to using a Git repository within Eclipse (Juno SR2), using EGit. In our project, some file names contains umlauts and other special non-ASCII-characters. On the command line, git ...
9
votes
4answers
20k views

What is a good way to batch re-encode mp3 files from a higher bitrate to a lower bitrate in Windows?

I have a few thousand MP3 files and they are all of mixed bitrate. I would like to run a program that can batch encode them all to 128 kbps. Are there any decent and free solutions for this for ...
9
votes
3answers
4k views

How can I convert from hex to base64?

Can anyone recommend a straightforward way/tool to convert hex to base64? I'm using Linux and OS X.
9
votes
2answers
274 views

General guidelines / workflow to convert or transfer video “professionally”?

I'm an IT "professional" who sometimes has to deal with small video conversion / video cutting projects, and I'd like to learn "the right way" to do this. Every time I search Google, there's always a ...
9
votes
1answer
3k views

Can GNU sed (for Windows) handle Unicode? If so, is it a code-page/locale issue, or a switch?

I've been using GNU SED on and off for a couple of years now. It spins me out a bit sometimes, but it does a good job... for single-byte char sets! I now and then notice references to GNU SED being ...
8
votes
2answers
1k views

Weird characters (´╗┐) at the start of a batch file

I am working with batch files in Windows, using both Notepad, and Notepad++. When I run the batch files, which all start with @echo off, I see the first line (when running on two separate machines) ...
8
votes
2answers
17k views

Is it possible to change the default encoding in notepad? [duplicate]

On windows 7 (x64), the default option, for saving text files in notepad is ANSI. One can select other encoding from the combo box, however, I'd like this option to be the default.
8
votes
2answers
3k views

What encoding to get Å Ä Ö to work

I'm writing a small application in VB.net that will enable me to easy create a user "Windows 7 account" with a password, instead of going though the control panel. The problem I'm having is that when ...
8
votes
1answer
4k views

Save UTF-8 without BOM Visual Studio 2012

It seems that Visual Studio 2012 hates UTF-8 without BOM and always adds it, resulting in an empty line in PHP pages. How can I change the encoding to UTF-8 without BOM (or at least to Unicode...)
7
votes
2answers
6k views

Which resize algorithm to choose for videos?

I'm using VirtualDub for encoding with those settings. However I record my stuff in 1920x1080 and resize it down to 1280x720. Now the question: which algorithm should I choose when making a balanced ...
7
votes
3answers
4k views

Finding out if a FLAC or WAVPACK audio file is NOT originally encoded from a lossy source

Is there a way of checking that the so-called FLAC or WAVPACK audio file was originally encoded from a lossless source (WAV, CDA, APE, etc.) instead of a lossy source (MP3, AAC, ATRAC, etc.)? Say I ...
7
votes
5answers
4k views

Ripping DVDs to 700 MB video files

When downloading movies on torrent websites, usually they come as 700 MB video files and with the .avi extension. In one single DVD I have 4~6 movies and they play very nice (with good quality) in my ...
7
votes
1answer
10k views

Determining the encoding of a file on Mac OS X?

How can determine the character encoding of plain text files (such as html) on Mac OS X?
6
votes
2answers
692 views

Why does VIM show the Unicode code point and not the UTF-8 code value?

Consider this supposed line of code that I found in a PHP blog, note the quotes: throw new Exception(“That's not a server name!”); Those quotes are RIGHT DOUBLE QUOTATION MARK (Unicode code point: ...
6
votes
2answers
4k views

How to encode audio with Opus codec?

The Opus audio codec looks like the best thing ever for compressing audio. It has recently become supported in the latest ffmpeg and VLC players. However, there is no documentation I can find on how ...
6
votes
1answer
2k views

How to switch back text encoding to UTF-8 with emacs?

Suddenly some of my text files, I'm editing with Emacs had changed and special characters were shown in a strange way, e. g. \344 instead of ä \374 instead of ü ... and so on. In another file I now ...
6
votes
1answer
258 views

Unadvisable to use UTF-8 for mails?

Thunderbird (in German) uses the charset ISO-8859-15 (or -1, can't check right now) by default for outgoing and incoming mails. (If you want to change the default, see Change Thunderbird default ...
6
votes
1answer
267 views

How to export html from emacs org-mode that can be opened in Microsoft Word

I created an HTML file using Emacs (org-mode export), which I want to open in Microsoft Word. However, Word says it cannot open the file. In the details of the error message, it says: Reference ...
5
votes
2answers
2k views

If I compress a file in a .7z archive with password, is it undecryptable without password?

I'm just curious to know whether or not an encrypted file is really save in terms of decrypting if I compress (or just save) a file into a .7z archive with a password. If I use a strong and ...
5
votes
5answers
2k views

Rip DVD on Linux

I want to rip a DVD and store it in a good and compressed format. What software to use for that and what is the best format; I want best quality and low file size. +1 if I don't have to install any ...
5
votes
1answer
508 views

Why do Cyrillic letter in URL domain appears so strange?

Short explanation: Why does russian characters in domain name in URL перезагрузкаопмо.рф get mapped to some randomness like this: http://xn--80aaigamcyttbbjfe2c.xn--p1ai. Please note, I am not ...
5
votes
1answer
2k views

ffmpeg - How to copy|extract encoding settings from existing media file?

I wished to be able do this multiple times now so I ask. If I have an existing video or audio file, ffmpeg, mplayer and other media players can detect at least some of it's "propreties" like ...
5
votes
1answer
4k views

Best Handbrake settings for encoding TV shows

Been using Handbrake since it's the fastest and easiest program for me to understand as I'm not a real A/V guy and I'm getting some under par results from it when it comes to TV shows. I see episodes ...
5
votes
1answer
286 views

One codec to rule them all

I am streaming videos in my house via Windows Media Player Streaming, which is basically DLNA. So theoretically any DLNA compliant device can pick up the stream. However, I've quickly found that ...
5
votes
3answers
3k views

SQL Server Management Studio 2005 - Set Default Encoding

Is anybody aware of a way to set the default encoding when saving a file to in SQL Server Management Studio to be ANSI instead of Unicode? Our source control system/diff tools barf when it runs ...
5
votes
1answer
15k views

MS Office 2010 Word change character encoding

I have a document that is not written in English but in other European language and when I open it in MS Word 2010 there are several characters displayed not properly. How can I change the encoding of ...
4
votes
3answers
3k views

́╗┐' is not recognized as an internal or external command

When executing certain files (mainly batch) using PsExec I get these weird ́╗┐' symbols before my command. My initial thought was, that I were using wrong encoding, but after checking, I realized that ...
4
votes
5answers
2k views

How do I find out which language some Unicode characters belong to?

On Facebook, there are currently messages floating around with these strange chars: ก็็็็็็็็็็็็็็็็็็็็ กิิิิิิิิิิิิิิิิิิิิ ก้้้้้้้้้้้้้้้้้้้ They are used to confuse the reader, because ...
4
votes
3answers
2k views

What's the best way to encode video for upload to YouTube?

Video is less than 5 minutes long, standard 4:3 aspect ratio (@ 640x480), imported from a digital camera, edited in Mac OS X with iMovie '09 and has a stereo, musical soundtrack. What encoding method ...
4
votes
2answers
736 views

Why does Firefox disable the 'Character encoding' menu for some sites?

For some websites I noticed the Firefox's 'View/character encoding' menu is disabled, why? Is it possible to enable it?
4
votes
2answers
4k views

How to bulk-rename files with invalid encoding or bulk-replace invalid encoded characters?

I have a debian server and I'm hosting music for an internet radio station. I have trouble with file names and paths because a lot of files got an invalid encoding, for example: ./music/Bändname - ...
4
votes
1answer
1k views

Firefox displays weird encoding icons on websites like GitHub

On Mozilla Firefox (on GNU/Linux) with default font and UTF-8 encoding, I am seeing some websites, such as GitHub, display weird characters across the page. Under a different browser, such as ...
4
votes
2answers
3k views

Why some unicode characters are missing in charmap?

In Windows, Character Map doesn't seem to display all of the unicode characters, even if the selected font supports them, and "Character set" selector is "Unicode". For example, the small envelope ...
4
votes
2answers
998 views

What is the video format to use for archiving videos for long term storage? [closed]

I want to store some videos for long term. But I'm not sure what is the most compatible video format. I don't want to wake up one day in the future when computers use some totally different ...
4
votes
1answer
615 views

7zip filename encoding problems

I have problems extracting files from 7zip archive with non-ascii symbols in filenames. I have identical versions of 7z on two computers, which differ only by the system locale. If I compress on one ...
4
votes
1answer
1k views

file-name encoding problems

I googled over this topic but couldn't find what I was looking for... the following "happend" to me: I had my files stored on a NTFS-USB Harddisk, because of space problems I moved them to an ext3 ...
4
votes
1answer
1k views

I'm trying to copy text from a PDF, but I get garbage

I'm trying to copy text from a PDF file, but I get garbage. I'm using Document Reader on Ubuntu to read the document. It's not like its not allowing me to copy, but just that the copied text looks ...
4
votes
1answer
6k views

Video format for both iPhone and Android?

Is there a format that plays both on iPhone and Android? I have been trying to encode videos in MP4 using ffmpeg - however, using libx264 or mpeg4 - the videos either only work in Android, or in ...
4
votes
3answers
870 views

Firefox Garbled Text

In FF 3.5.7 on Vista I am getting garbled text like below on some web pages. The obvious answer is to go to View > Character Encodings > then select UTF-8. However, it turns out I am already using ...
4
votes
0answers
412 views

Fix encoding of German umlauts in directories and filenames (ü = u╠ê and so on)

I have many zip-files where there are encoding errors for the German umlauts (äüöÄÜÖß). They show up in both the filename.zip as well as in the included directories and files like this: Fünf = ...
4
votes
1answer
148 views

What's wrong with `(add-to-list 'process-coding-system-alist '(“zsh” . utf-8))`?

In Emacs, I routinely use an inferior shell, invoked with M-x shell. I often need to specifically set its coding system, which I do with C-x RET p followed by utf-8 (twice). I want to do this once ...