127

I am using the bash shell and would like to pipe the out of the command openssl rand -base64 1000 to the command dd such as dd if={output of openssl} of="sample.txt bs=1G count=1. I think I can use variables but I am however unsure how best to do so. The reason I would like to create the file is because I would like a 1GB file with random text.

Improve this question
1
  • What do you want to do with that file? To check e.g. compression algorithms, use the type of data they are designed for (natural language text you can get boatloads at project Gutenberg; source code, grab e.g. the GNU, BSD, Sourceforge packages or sample github). "Real world" data is not random.
    – vonbrand
    May 24, 2021 at 1:24

6 Answers 6

Reset to default
190

if= is not required, you can pipe something into dd instead:

something... | dd of=sample.txt bs=1G count=1

something... | head -c 1G > sample.txt

It wouldn't be useful here since openssl rand requires specifying the number of bytes anyway. So you don't actually need dd – this would work:

openssl rand -out sample.txt -base64 $(( 2**30 * 3/4 ))

1 gigabyte is usually 230 bytes (though you can use 10**9 for 109 bytes instead). The * 3/4 part accounts for Base64 overhead, making the encoded output 1 GB.

Alternatively, you could use /dev/urandom, but it would be a little slower than OpenSSL:

dd if=/dev/urandom of=sample.txt bs=1G count=1

I would use bs=64M count=16 or similar, so that 'dd' won't try to use the entire 1 GB of RAM at once:

dd if=/dev/urandom of=sample.txt bs=64M count=16

or even the simpler head tool – you don't really need dd here:

head -c 1G /dev/urandom > sample.txt
Improve this answer
15
  • 2
    I posted a question regarding compressing large files at superuser.com/questions/467697/… and was advised that using /dev/urandom generates a binary file and not a true text file.
    – PeanutsMonkey
    Sep 6, 2012 at 19:10
  • 2
    @PeanutsMonkey: Right; you would need something like dd if=/dev/urandom bs=750M count=1 | uuencode my_sample > sample.txt.
    – Scott - Слава Україні
    Sep 6, 2012 at 19:33
  • 4
    @PeanutsMonkey: There's no single "real world scenario", some scenarios might be dealing with gigabytes of text, others – with gigabytes of JPEGs, or gigabytes of compiled software... If you want a lot of text, download a Wikipedia dump for example.
    – user1686
    Sep 6, 2012 at 20:06
  • 2
    @PeanutsMonkey: The dd reads 750,000,000 bytes from /dev/urandom and pipes them into uuencode. uuencode encodes its input into a form of base64 encoding (not necessarily consistent with other programs). In other words, this converts binary data to text. I used 750M because I trusted grawity's statement that base64 encoding expands data by 33⅓%, so you need to ask for ¾ as much binary data as you want in your text file.
    – Scott - Слава Україні
    Sep 6, 2012 at 20:07
  • 7
    Note if it says dd: warning: partial read (33554431 bytes); suggest iflag=fullblock it will create a truncated file so add the iflag=fullblock flag, then it works.
    – rogerdpack
    Sep 27, 2018 at 20:21
46

Create a 1GB.bin random content file:

 dd if=/dev/urandom of=1GB.bin bs=64M count=16 iflag=fullblock
Improve this answer
1
  • 8
    For me, iflag=fullblock was the necessary addition compare to other answers.
    – dojuba
    Sep 18, 2018 at 14:55
4

Since, your goal is to create a 1GB file with random content, you could also use yes command instead of dd:

yes [text or string] | head -c [size of file] > [name of file]

Sample usage:

yes this is test file | head -c 100KB > test.file
Improve this answer
1
  • In my mac os 100KB is not working, need to use this: head -c 102400
    – Deqing
    Jul 1, 2022 at 5:45
4

If you want EXACTLY 1GB, then you can use the following:

openssl rand -out $testfile -base64 792917038; truncate -s-1 $testfile

The openssl command makes a file exactly 1 byte too big. The truncate command trims that byte off.

Improve this answer
1
  • That extra byte is probably because of the -base64. Removing it will result in a file with the correct size.
    – Daniel
    Oct 10, 2019 at 11:34
3

If you just need a somewhat random file which is not used for security related things, like benchmarking something, then the following will be significantly faster:

truncate --size 1G foo
shred --iterations 1 foo

It's also more convenient because you can simply specify the size directly.

Improve this answer
-2

Try this script.

#!/bin/bash
openssl rand -base64 1000 | dd of=sample.txt bs=1G count=1

This script might work as long as you don't mind using /dev/random.

#!/bin/bash
dd if=/dev/random of="sample.txt bs=1G count=1"
Improve this answer
6
  • 8
    I wouldn't recommend wasting /dev/random on this unless there's a very good reason to do so. /dev/urandom is much cheaper.
    – Ansgar Wiechers
    Sep 6, 2012 at 18:22
  • 1
    Also, $var=(command) isn't valid syntax in this context.
    – user1686
    Sep 6, 2012 at 18:58
  • @grawity - When you say it isn't valid, what do you mean?
    – PeanutsMonkey
    Sep 6, 2012 at 19:08
  • I mean exactly that – it's incorrect.
    – user1686
    Sep 6, 2012 at 19:22
  • 3
    @grawity, @PeanutsMonkey: He made a typo; he meant random=$(openssl rand -base64 1000). Although I would question whether bash would let you assign a gigabyte-long value to a variable. And even if you do say random=$(openssl rand -base64 1000), the subsequent if=$random doesn't make sense.
    – Scott - Слава Україні
    Sep 6, 2012 at 19:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.