30

I'm doing some testing of a piece of code and I want to fill a hard drive with data. I found that dd can make huge files in an instant, but df disagrees. Here's what I tried:

dd if=/dev/zero of=filename bs=1 count=1 seek=$((10*1024*1024*1024))

ls -lh shows a 10G file. However, df -h shows that the partition did not shrink. So what do I need to do to make df recognize the data is now taken? I'm hoping for something fast that I code up in a unit test.

2
  • You will have more lucky if you ask at serverfault.com
    – joksnet
    Dec 17, 2010 at 15:27
  • BTW, 10*1024*1024*1024 is the same as 10 * 2**30, A bit simpler.
    – deltaray
    Nov 15, 2017 at 15:25

7 Answers 7

28

The trouble with the seek=<big number> trick is that the filesystem is (usually) clever: if part of a file has never been written to (and is therefore all zeros), it doesn't bother to allocate any space for it - so, as you've seen, you can have a 10GB file that takes up no space (this is known as a "sparse file", and can very useful in some instances, e.g. certain database implementations).

You can force the space to be allocated with (for example):

dd if=/dev/zero of=filename bs=$((1024*1024)) count=$((10*1024))

which will take much longer, but will actually fill the disk. I recommend making the block size much higher than one, because this will determine how many system calls the dd process makes - the smaller the blocksize, the more syscalls, and therefore the slower it will run. (Though beyond 1MB or so it probably won't make much difference and may even slow things down...)

1
  • I usually use bs=(whatever half the cache size is on the drive I'm writing). This has worked well to date when imaging partitions onto larger hard drives for later resizing. ETA: the critical word HALF.
    – atroon
    Apr 5, 2011 at 20:34
32

As another option to this, you can use the yes along with a single string and its about 10 times faster than running dd if=/dev/urandom of=largefile. Like this

yes abcdefghijklmnopqrstuvwxyz0123456789 > largefile
3
  • This is, surprisingly, about as fast as using dd if=/dev/zero of=filename bs=1M.
    – bhinesley
    Feb 1, 2012 at 3:45
  • The syntax is easy to remember too.
    – AWT
    Apr 21, 2015 at 18:48
  • Nice. yes `dd if=/dev/urandom count=1 bs=1M|base64` > random works too. Feb 8, 2022 at 18:44
7

You have created what is known as a "sparse file" - a file that, because most of it is empty (i.e. reads back as \0), doesn't take space on the disk besides what is actually written (1B, after 10GB of gap).

I don't believe you could make huge files, taking actual disk space in an instant - taking physical space means the filesystem needs to allocate disk blocks to your file.

I think you're stuck with the oldfashioned "dd if=/dev/zero of=filename bs=100M count=100" which is limited by your drive sequential write speed.

0
7

If you're just testing for cases with filled file systems, maybe fallocate is good enough. And faster too!

fallocate -l 150G $filename
1
  • 3
    This needs a filename to be specified: fallocate -l 150G foo. Nevertheless +1 for mentioning the tool. Jun 21, 2016 at 6:26
3

Stop using seek and use a very large bs and/or count. As it is you're making a sparse file, and obviously you need to not do that.

3

If you want to literally fill the hard drive, do this:

dd if=/dev/zero of=zeros bs=1M

You can optionally specify count if you want to limit the size, but if you omit the count, it will just run until you're out of disk space.

dd if=/dev/zero of=zeros bs=1M count=10240

As psmears mentioned, you'll get better performance if you set the block size to 1 MB (bs=1M) instead of 1 B (bs=1). This will still take a while, but if you want to check on the progress of your command, open a separate console and run these commands:

ps aux | grep dd

Use the PID of dd in this command (replace PID with dd's PID):

kill -USR1 PID

Then go look at your dd terminal. Of course, this is of limited use when you're just trying to fill up the drive (you could just use df or du to check free disk space or file size, respectively). However, there are other times when it's handy to make dd output its progress.

Extra credit: one practical use for zeroing the free space is that, afterward, you can then delete the "zero" files and dd the entire partition (or disk, if you've zeroed all the partitions) to a disk image file (say, disk-backup.dd), then compress the file. The free space is now highly compressible, so your compressed dd image will be a lot smaller than the original block device whose contents it contains.

Shenanigans: zip up a large zeros file and e-mail it to all your friends. Tell them it's something really cool.

2

You create a non-sparse 1 TB file with the following command:

dd if=/dev/zero of=bigfile1 bs=10000000000 count=512

This was useful in testing what happens when quotas are over-reached, or a file system fills up.

df -h shows the available space getting smaller.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .