2

I am searching for the way to convert the hexadecimal value of Shift-JIS to the character in Unix/Linux command line.

A Shift-JIS code table can be found here.

For:

82 ae (0x82ae)

I would expect:

I know it’s somehow possible with ascii2uni and nkf but I’m kind of stuck.

NOTE: I could do this but it is not the expected result:

echo "0x82BE" | ascii2uni -a X | nkf -S
Result is: 1 token converted
闃セ

I found this thread that suggests using iconv but ended up with the same results using that method?

Can you folks help?

3
  • I found superuser.com/questions/313032/… but I found the same result with their method. Nov 24, 2015 at 7:09
  • That iconv recipe should have converted Shift-JIS to UTF-8. Did it display as 闃セ? What terminal program are you using?
    – Tom Zych
    Nov 24, 2015 at 8:04
  • Disregard, it's not the terminal encoding. See answer below.
    – Tom Zych
    Nov 24, 2015 at 9:07

2 Answers 2

1

Based on this answer on the Unix and Linux Stack Exchange site, recode works cleanly for me on Ubuntu 12.04.5 (LTS):

echo -n 0x82ae | recode SHIFT-JIS/x4..UTF-8

Of course this converts the output of the the hexadecimal code from Shift-JIS to UTF-8, but heck… UTF-8 is what all the kids are using nowadays. But you can just lop off the ..UTF-8 stuff like this and the output should be pure Shift-JIS:

echo -n 0x82ae | recode SHIFT-JIS/x4

To confirm it converted to UTF-8 correctly you can pipe it to xxd like this:

echo -n 0x82ae | recode SHIFT-JIS/x4..UTF-8 | xxd -p -u

And it checks out as being E38190 which matches the exact same character in UTF-8 as shown here. A full table of Shift-JIS to UTF-8 conversion mappings can be found here.

Or you can just run xxd from the command line like this to get the exact hexadecimal code for any character—or series of characters—you wish:

echo -n "ぐ" | xxd -p -u
2
  • 1
    The iconv should have converted to UFT-8 as well. I suspect his terminal is using another encoding.
    – Tom Zych
    Nov 24, 2015 at 8:24
  • 1
    Nope, I was wrong. Writing up the answer now.
    – Tom Zych
    Nov 24, 2015 at 8:54
2

Your file doesn't contain the binary you think it does. ascii2uni isn't encoding the way you expect it to; I'm not sure why.

echo "0x82BE" | ascii2uni -a X > test1
od -tx1 test1
0000000 e8 8a be 0a

(Note that echo -n does the same thing. The newline 0a is coming from ascii2uni, not echo.)

Converting this with iconv:

iconv -f SHIFT-JIS -t UTF-8 test1 > test2
od -tx1 test2
0000000 e9 97 83 ef bd be 0a
cat test2
闃セ

Which is what you got. (Note also that in your ascii2uni example, you had 82BE, which is だ in Shift-JIS, instead of your original 0x82ae, which is ぐ. I'll stick with 82BE.)

The problem is that the binary wasn't right to begin with. Do it this way:

echo -en '\x82\xbe' > test3
od -tx1 test3
0000000 82 be
iconv -f SHIFT-JIS -t UTF-8 test3 > test4
od -tx1 test4
0000000 e3 81 a0
cat test4
だ
1
  • Thanks for the explanation. Exactly what I needed. Much appreciated! Nov 24, 2015 at 9:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .