Linux locale - how to disable smart quotes?

Question

On my Gentoo system, find . -regextype help outputs

find: Unknown regular expression type ‘help’; valid types are ‘findutils-default’, ‘awk’, ‘egrep’, ‘ed’, ‘emacs’, ‘gnu-awk’, ‘grep’, ‘posix-awk’, ‘posix-basic’, ‘posix-egrep’, ‘posix-extended’, ‘posix-minimal-basic’, ‘sed’.

I always thought smart quotes (or whatever they are called) are the scourge of Microsoft software, turns out nobody's protected from that ugliness. On my other system (Cygwin), LANG is set to en_US.UTF-8, and smart quotes are also displayed. If I unset LANG or set it to en.UTF-8, output changes to regular single quotes:

# unset LANG
# find -regextype help
find: Unknown regular expression type 'help'; valid types are 'findutils-default', 'awk', 'egrep', 'ed', 'emacs', 'gnu-awk', 'grep', 'posix-awk', 'posix-basic', 'posix-eg
rep', 'posix-extended', 'posix-minimal-basic', 'sed'.

But on the Gentoo system I mentioned earlier, LANG is unset. Whatever I try to set it (and other locale env vars) to, I see smart quotes and/or a bunch of error messages about incorrect locales.

How to get back my single quotes?

Answer 1

This is not auto-converted by a smart-quotes system. That text is the direct output of GNU find. A some system-wide smart-quotes interpreter would only affect input coming from the keyboard … and I've never heard of one for GNU/Linux/BSD (I found this post because I was looking for how to enable it!)

Here's a hex dump of your command output:

$ find . -regextype help 2>&1 |head -c99
find: Unknown regular expression type ‘help’; valid types are 
‘findutils-default’, ‘ed’
$ find . -regextype help 2>&1 |head -c99 |hd
00000000  66 69 6e 64 3a 20 55 6e  6b 6e 6f 77 6e 20 72 65  |find: Unknown re|
00000010  67 75 6c 61 72 20 65 78  70 72 65 73 73 69 6f 6e  |gular expression|
00000020  20 74 79 70 65 20 e2 80  98 68 65 6c 70 e2 80 99  | type ...help...|
00000030  3b 20 76 61 6c 69 64 20  74 79 70 65 73 20 61 72  |; valid types ar|
00000040  65 20 e2 80 98 66 69 6e  64 75 74 69 6c 73 2d 64  |e ...findutils-d|
00000050  65 66 61 75 6c 74 e2 80  99 2c 20 e2 80 98 65 64  |efault..., ...ed|
00000060  e2 80 99                                          |...|

This makes it easier to see that the output uses the explicit left and right single quotation march characters (U+2018 and U+2019 respectively, see this Unicode code point table). In UTF-8, these are represented as e2 80 98 and e2 80 99, which is why you see three dots (which denote non-ASCII characters) on either side of help (itself 68 56 5c 70) and the other quoted words.

You can therefore convert it back with sed:

$ find . -regextype help 2>&1 |head -c99 \
  |sed "s/\xe2\x80\x98/'/g; s/\xe2\x80\x99/'/g"
find: Unknown regular expression type 'help'; valid types are 
'findutils-default', 'ed'
$ find . -regextype help 2>&1 |head -c99 \
  |sed "s/\xe2\x80\x98/'/g; s/\xe2\x80\x99/'/g" |hd
00000000  66 69 6e 64 3a 20 55 6e  6b 6e 6f 77 6e 20 72 65  |find: Unknown re|
00000010  67 75 6c 61 72 20 65 78  70 72 65 73 73 69 6f 6e  |gular expression|
00000020  20 74 79 70 65 20 27 68  65 6c 70 27 3b 20 76 61  | type 'help'; va|
00000030  6c 69 64 20 74 79 70 65  73 20 61 72 65 20 27 66  |lid types are 'f|
00000040  69 6e 64 75 74 69 6c 73  2d 64 65 66 61 75 6c 74  |indutils-default|
00000050  27 2c 20 27 65 64 27                              |', 'ed'|