Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have to deal with a file that has lot of invisible control characters, like "right to left" or "zero width non-joiner", different spaces than the normal space and so on, and I have troubles dealing with that.

Now, I would like to somehow view all letters in a given file, letter by letter (I would like to say "left to right", but I am unfortunately dealing with right-to-left language), as unicode codepoints, using only basic bash tools (like vi, less, cat...). Is it possible somehow?

I know I can display the file in hexadecimal by hexdump, but I would have to recompute the codepoints. I really want to see the actual unicode codepoints, so I can google them and find out what's happenning.

edit: I will add that I don't want to transcode it to different encoding (because that's what I am finding out online). I have the file in UTF8 and that is fine. I just want to know the exact codepoints of all the letters.

share|improve this question

1 Answer 1

up vote 1 down vote accepted

I wrote myself a perl one-liner, that do just that, and it also prints the original character. (It expects the file from STDIN)

perl -C7 -ne 'for(split(//)){print sprintf("U+%04X", ord)." ".$_."\n"}'

However, there should be a better way than this.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.