Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is there a simple way to print all non-ASCII characters and the line numbers on which they occur in a file using a command line utility such as grep, awk, perl, etc?

I want to change the encoding of a text file from UTF-8 to ASCII, but before doing so, wish to manually replace all instances of non-ASCII characters to avoid unexpected character changes effected by the file conversion routine.

share|improve this question
$ perl -ne 'print "$. $_" if m/[\x80-\xFF]/'  utf8.txt
2 Pour être ou ne pas être
4 Byť či nebyť
5 是或不

or

$ grep -n -P '[\x80-\xFF]' utf8.txt
2:Pour être ou ne pas être
4:Byť či nebyť
5:是或不

where utf8.txt is

$ cat utf8.txt
To be or not to be.
Pour être ou ne pas être
Om of niet zijn
Byť či nebyť
是或不
share|improve this answer
    
Thanks. The perl snippet works directly, but the grep version doesn't work with GNU grep 2.16. I was able to make it work via: LC_ALL=C grep -n -P [$'\x80'-$'\xFF'], where the first bit turns off collation. – Joe Corneli Sep 18 '14 at 12:23

I want to change the encoding of a text file from UTF-8 to ASCII ...

... replace all instances of non-ASCII characters ...

Then tell your conversion tool to do so.

$ iconv -c -f UTF-8 -t ASCII <<< 'Look at 私.'
Look at .

$ iconv -c -f UTF-8 -t ASCII//translit <<< 'áēìöų'
aeiou
share|improve this answer
    
He said he wanted to do that replacement manually. Perhaps the most appropriate replacement is context-dependent. – mark4o Apr 27 '12 at 0:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .