I am using the bash shell and would like to pipe the out of the command openssl rand -base64 1000 to the command dd such as dd if={output of openssl} of="sample.txt bs=1G count=1.
I think I can use variables but I am however unsure how best to do so. The reason I would like to create the file is because I would like a 1GB file with random text.
Add a comment
|
if= is not required, you can pipe something into dd instead:
something... | dd of=sample.txt bs=1G count=1
It wouldn't be useful here since openssl rand requires specifying the number of bytes anyway. So you don't actually need dd – this would work:
openssl rand -out sample.txt -base64 $(( 2**30 * 3/4 ))
1 gigabyte is usually 230 bytes (though you can use 10**9 for 109 bytes instead). The * 3/4 part accounts for Base64 overhead, making the encoded output 1 GB.
Alternatively, you could use /dev/urandom, but it would be a little slower than OpenSSL:
dd if=/dev/urandom of=sample.txt bs=1G count=1
Personally, I would use bs=64M count=16 or similar:
dd if=/dev/urandom of=sample.txt bs=64M count=16
-
2I posted a question regarding compressing large files at superuser.com/questions/467697/… and was advised that using
/dev/urandomgenerates a binary file and not a true text file. – PeanutsMonkey Sep 6 '12 at 19:10 -
2@PeanutsMonkey: Right; you would need something like
dd if=/dev/urandom bs=750M count=1 | uuencode my_sample > sample.txt. – Scott Sep 6 '12 at 19:33 -
3@PeanutsMonkey: There's no single "real world scenario", some scenarios might be dealing with gigabytes of text, others – with gigabytes of JPEGs, or gigabytes of compiled software... If you want a lot of text, download a Wikipedia dump for example. – user1686 Sep 6 '12 at 20:06
-
2@PeanutsMonkey: The
ddreads 750,000,000 bytes from/dev/urandomand pipes them intouuencode.uuencodeencodes its input into a form of base64 encoding (not necessarily consistent with other programs). In other words, this converts binary data to text. I used 750M because I trusted grawity's statement that base64 encoding expands data by 33⅓%, so you need to ask for ¾ as much binary data as you want in your text file. – Scott Sep 6 '12 at 20:07 -
4Note if it says
dd: warning: partial read (33554431 bytes); suggest iflag=fullblockit will create a truncated file so add theiflag=fullblockflag, then it works. – rogerdpack Sep 27 '18 at 20:21
Create a 1GB.bin random content file:
dd if=/dev/urandom of=1GB.bin bs=64M count=16 iflag=fullblock
-
6For me,
iflag=fullblockwas the necessary addition compare to other answers. – dojuba Sep 18 '18 at 14:55
If you want EXACTLY 1GB, then you can use the following:
openssl rand -out $testfile -base64 792917038; truncate -s-1 $testfile
The openssl command makes a file exactly 1 byte too big. The truncate command trims that byte off.
-
That extra byte is probably because of the
-base64. Removing it will result in a file with the correct size. – Daniel Oct 10 '19 at 11:34
Try this script.
#!/bin/bash
openssl rand -base64 1000 | dd of=sample.txt bs=1G count=1
This script might work as long as you don't mind using /dev/random.
#!/bin/bash
dd if=/dev/random of="sample.txt bs=1G count=1"
-
8I wouldn't recommend wasting
/dev/randomon this unless there's a very good reason to do so./dev/urandomis much cheaper. – Ansgar Wiechers Sep 6 '12 at 18:22 -
1
-
-
-
3@grawity, @PeanutsMonkey: He made a typo; he meant
random=$(openssl rand -base64 1000). Although I would question whetherbashwould let you assign a gigabyte-long value to a variable. And even if you do sayrandom=$(openssl rand -base64 1000), the subsequentif=$randomdoesn't make sense. – Scott Sep 6 '12 at 19:28
