12

I have a file instructions.txt with the contents:

00000000000000000000000000010011
00000010110100010010000010000011
00000000011100110000001010110011
00000000011100110000010000110011
00000000011100110110010010110011
00000000000000000000000000010011

How can I create a binary file instructions.bin of the same data as instructions.txt. In other words the .bin file should be the same 192 bits that are in the .txt file, with 32 bits per line. I am using bash on Ubuntu Linux. I was trying to use xxd -b instructions.txt but the output is way longer than 192 bits.

6

oneliner to convert 32-bit strings of ones and zeros into corresponding binary:

$ perl -ne 'print pack("B32", $_)' < instructions.txt > instructions.bin

what it does:

  • perl -ne will iterate through each line of input file provided on STDIN (instructions.txt)
  • pack("B32", $_) will take a string list of 32 bits ($_ which we just read from STDIN), and convert it to binary value (you could alternatively use "b32" if you wanted ascending bit order inside each byte instead of descending bit order; see perldoc -f pack for more details)
  • print would then output that converted value to STDOUT, which we then redirect to our binary file instructions.bin

verify:

$ hexdump -Cv instructions.bin
00000000  00 00 00 13 02 d1 20 83  00 73 02 b3 00 73 04 33  |...... ..s...s.3|
00000010  00 73 64 b3 00 00 00 13                           |.sd.....|
00000018

$ xxd -b -c4 instructions.bin
00000000: 00000000 00000000 00000000 00010011  ....
00000004: 00000010 11010001 00100000 10000011  .. .
00000008: 00000000 01110011 00000010 10110011  .s..
0000000c: 00000000 01110011 00000100 00110011  .s.3
00000010: 00000000 01110011 01100100 10110011  .sd.
00000014: 00000000 00000000 00000000 00010011  ....
|improve this answer|||||
8

Adding the -r option (reverse mode) to xxd -b does not actually work as intended, because xxd simply does not support combining these two flags (it ignores -b if both are given). Instead, you have to convert the bits to hex yourself first. For example like this:

( echo 'obase=16;ibase=2'; sed -Ee 's/[01]{4}/;\0/g' instructions.txt ) | bc | xxd -r -p > instructions.bin

Full explanation:

  • The part inside the parentheses creates a bc script. It first sets the input base to binary (2) and the output base to hexadecimal (16). After that, the sed command prints the contents of instructions.txt with a semicolon between each group of 4 bits, which corresponds to 1 hex digit. The result is piped into bc.
  • The semicolon is a command separator in bc, so all the script does is print every input integer back out (after base conversion).
  • The output of bc is a sequence of hex digits, which can be converted to a file with the usual xxd -r -p.

Output:

$ hexdump -Cv instructions.bin
00000000  00 00 00 13 02 d1 20 83  00 73 02 b3 00 73 04 33  |...... ..s...s.3|
00000010  00 73 64 b3 00 00 00 13                           |.sd.....|
00000018
$ xxd -b -c4 instructions.bin
00000000: 00000000 00000000 00000000 00010011  ....
00000004: 00000010 11010001 00100000 10000011  .. .
00000008: 00000000 01110011 00000010 10110011  .s..
0000000c: 00000000 01110011 00000100 00110011  .s.3
00000010: 00000000 01110011 01100100 10110011  .sd.
00000014: 00000000 00000000 00000000 00010011  ....
|improve this answer|||||
  • Sorry, there's still an endianness bug in this. Working on fixing it! – nomadictype Oct 10 '18 at 0:49
  • 1
    Actually, it's fine. I was confused earlier by using the wrong output width in the last xxd command. – nomadictype Oct 10 '18 at 0:58
  • 1
    I've tested the script and it works but outputs: (standard_in) 1: syntax error. Can you explain what syntax error it is referring to or why this occurs? Does this happen on your machine too? – dopamane Oct 10 '18 at 12:00
2

My original answer was incorrect - xxd cannot accept either -p or -r with -b...

Given that the other answers are workable, and in the interest of "another way", how about the following:

Input

$ cat instructions.txt
00000000000000000000000000010011
00000010110100010010000010000011
00000000011100110000001010110011
00000000011100110000010000110011
00000000011100110110010010110011
00000000000000000000000000010011

Output

$ hexdump -Cv < instructions.bin
00000000  00 00 00 13 02 d1 20 83  00 73 02 b3 00 73 04 33  |...... ..s...s.3|
00000010  00 73 64 b3 00 00 00 13                           |.sd.....|
00000018

Bash pipeline:

cat instructions.txt \
    | tr -d $'\n' \
    | while read -N 4 nibble; do 
        printf '%x' "$((2#${nibble}))"; \
      done \
    | xxd -r -p \
    > instructions.bin
  • cat - unnecessary, but used for clarity
  • tr -d $'\n' - remove all newlines from the input
  • read -N 4 nibble - read exactly 4× characters into the nibble variable
  • printf '%x' "$((2#${nibble}))" convert the nibble from binary to 1× hex character
    • $((2#...)) - convert the given value from base 2 (binary) to base 10 (decimal)
    • printf '%x' - format the given value from base 10 (decimal) to base 16 (hexadecimal)
  • xxd -r -p - reverse (-r) a plain dump (-p) - from hexadecimal to raw binary

Python:

python << EOF > instructions.bin
d = '$(cat instructions.txt | tr -d $'\n')'
print(''.join([chr(int(d[i:i+8],2)) for i in range(0, len(d), 8)]))
EOF
  • An unquoted heredoc (<< EOF) is used to get content into the Python code
    • This is not efficient if the input becomes large
  • cat and tr - used to get a clean (one-line) input
  • range(0, len(d), 8) - get a list of numbers from 0 to the end of the string d, stepping 8× characters at a time.
  • chr(int(d[i:i+8],2)) - convert the current slice (d[i:i+8]) from binary to decimal (int(..., 2)), and then to a raw character (chr(...))
  • [ x for y in z] - list comprehension
  • ''.join(...) - convert the list of characters into a single string
  • print(...) - print it
|improve this answer|||||
  • 1
    Note: in many shells | at the end of a line works like a backslash: the command continues to the next line. This way you can get rid of few backslashes. I'm not sure if using pipe symbols after LFs was your informed decision. I'm mentioning the other way in case you didn't know. – Kamil Maciorowski Oct 10 '18 at 10:03
  • 1
    I didn't know, thanks! I do like breaking the pipeline into logical lines, and having the pipes | (or redirects >, boolean operators &&, etc...) explicitly at the front for visibility / clarity... perhaps a stylistic / preference thing. – Attie Oct 10 '18 at 11:36
  • 1
    After some thoughts I may start using this style because one can tell the two lines are connected, by examining any of them. If | is at the end, the next line may look like a standalone command, it may be confusing. This is why I thought the style might be your informed decision. – Kamil Maciorowski Oct 10 '18 at 11:45
  • Awesome, let me know how it goes :-) – Attie Oct 10 '18 at 12:43
  • 1
    It's going well. :) – Kamil Maciorowski Oct 16 '18 at 12:00
1

You may as well try posting this to CodeGolf SE site, but here's my alternative Python version (just for kick challenge):

python -c "import sys,struct;[sys.stdout.buffer.write(struct.pack('!i',int(x,2)))for x in sys.stdin]" \
< input.txt > output.bin

Assuming input.txt contains your data, and it is formatted to 32 characters per line.

This uses Python 3 struct package and writing / reading to stdin / out. (In Python 2 it would've been shorter).

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.