GNU sort cyrillic lines ignoring the case

Question

I have got a file in UTF-8 with lines of latin or cyrillic letters. I'm trying to sort it, ignoring the case.

I have tried

LC_ALL="ru" sort --ignore-case in.txt

but still got cyrillic lines in inpropper order, while lines in latin are sorted fine.

Example of sorted file:

Apple
apple
Banana
banana
...
// but with cyrillic letters i got this:
...
Арбуз
Банан
арбуз
банан

How can I manage this? Thank you in advance.

Answer 1

I suspect "ru" isn't a valid locale. Try the command lang=ru locale and see if there are any error messages. On my system, "ru" is an invalid locale and "ru_RU" is in iso8859-5; you need ru_RU.UTF-8 for it to work in utf-8.

$ LANG=ru locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory

To find the character set used for a locale, you can use the locale tool:

$ LANG=ru_RU locale -c -k charmap collate-codeset
LC_CTYPE
charmap="ISO-8859-5"
LC_COLLATE
collate-codeset="ISO-8859-5"

Note that locales are defined in categories. The LC_COLLATE category is the one used for sorting. Setting LC_ALL will override all of them, or you can set one individually by using the named environment variable, or you can set a default by setting LANG. Most people will want to set LANG for daily usage rather than LC_ALL.

In the output of the locale tool, categories which are set based on LC_ALL or LANG will show up in quotes, whereas categories set by their own variables (and LC_ALL itself if set, and LANG if LC_ALL is not set) will show up without quotes. For example:

$ LANG=en_US.UTF-8 LC_MESSAGES=ru_RU.UTF-8 locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
...
LC_MESSAGES=ru_RU.UTF-8
...
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Answer 2

$ cat input.txt 
banana
coffee
кофе
Банан
Apple
Coffee
арбуз
apple
Banana
Кофе
банан
Арбуз

$ export LC_ALL=en_US.UTF-8 && sort --ignore-case input.txt
apple
Apple
banana
Banana
coffee
Coffee
арбуз
Арбуз
банан
Банан
кофе
Кофе