12

How to easily convert to/from plain machine-readable hexadecimal data (without any paddings/offsets/character view) with xdd or hexdump?

I'm tired of digging of some special format strings (and finding out that it suddenly starts wrapping lines after N characters or skip lines) or writing Perl one-liners every time.

Why is it not as simple as base64/base64 -d?

2
  • Because almost no one needs it. May 5, 2011 at 13:06
  • 1
    base64 does wrap the output by default.
    – user1686
    May 5, 2011 at 13:09

3 Answers 3

20
hexdump -ve '1/1 "%02x"'
xxd -p | tr -d '\n'

If you get tired of writing this every time, create an alias.

3
  • 1
    I'm tired of looking at man page and gettings confused either by necessity to compose hexdump's formt strings or by xxd's "postscript" thing. (Is this "postscript" something about printers or about letters?)
    – Vi.
    May 5, 2011 at 13:59
  • 2
    @Vi, think of -p as "plain". I surmise the Postscript comment is because the Postscript language allows for in-line data (typically images) encoded exactly this way. So Postscript programmers may use xxd to convert binary images into a convenient form for embedding in a Postscript file. May 5, 2011 at 14:05
  • For uppercase hexadecimal output with xxd, use -u. Jun 11, 2017 at 12:43
10

How to easily convert to/from plain machine-readable hexadecimal data

In brief.

$ xxd -plain test.txt > test.hex
$ xxd -plain -revert test.hex test2.txt
$ diff test.txt test2.txt
$

Explanation:

$ xxd -plain test.txt > test.hex

This writes a hex encoding of the data in test.txt into new file test.hex. The -p or -plain option makes xxd use "plain" hex format with no spaces between pairs of hex digits (i.e. no spaces between byte values). This converts "abc ABC" to "61626320414243". Without the -p it would convert the text to a 16-bit word oriented traditional hexdump format, which is arguably easier to read but less compact and therefore less suitable as a transmission format and slightly harder to reverse.

$ xxd -plain -revert text.hex test2.txt

This uses the -r or -revert option for reverse operation. The -plain option is used again to indicate that the input hex file is in plain format.

I make the output filename different from the original filename so we can later compare the results with the original file.

$ diff test.txt test2.txt
$ 

The diff command outputs nothing - this means there is no difference between the original and reconstituted file contents.


I'm tired of digging of some special format strings

Use alias or declare functions in your .profile to create mnemonics so you don't have to remember or dig about in man pages.

or just remember -plain and -revert.


Wrapped output

Yes, there are new-line characters in the output. You want to avoid that. You can use the -c or -cols option to specify how long you want the output lines to be to attempt to avoid line-wrapping of the output. -c 0 gives the default length and the man page suggests 256 is the limit but it seems to work beyond that.

$ xxd -plain -cols 9999 test.txt > test.hex
$ wc test.txt test.hex
  121   880  4603 test.txt
    1     1  9207 test.hex

The wc wordcount command tells us how many lines, words and characters are in each file.

So 121 lines (880 words, 4603 bytes) of ASCII text were encoded as 1 line of hex digits.

7
  • (was skipping "-p" option while reading man because of mysterious "postscript" thing)
    – Vi.
    May 5, 2011 at 13:57
  • Wrapped output.
    – Vi.
    May 5, 2011 at 13:59
  • 1
    @Vi: I understand. Out of curiosity, what is the problem with wrapped output (i.e. a small percent of arguably superfluous LF chars)? May 5, 2011 at 14:01
  • It can be a problem if including hex in config files, like INIT=01303031313102 CMD1=41414141FF000000.
    – Vi.
    May 5, 2011 at 15:02
  • Can you elaborate a little bit on what the 3 command lines are doing (by editing the answer)? Jun 11, 2017 at 12:17
2

Here is the version using od utility (part of coreutils package):

od -An < input | tr -dc '[:alnum:]'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.