Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm doing some testing of a piece of code and I want to fill a hard drive with data. I found that dd can make huge files in an instant, but df disagrees. Here's what I tried:

dd if=/dev/zero of=filename bs=1 count=1 seek=$((10*1024*1024*1024))

ls -lh shows a 10G file. However, df -h shows that the partition did not shrink. So what do I need to do to make df recognize the data is now taken? I'm hoping for something fast that I code up in a unit test.

share|improve this question

migrated from stackoverflow.com Dec 18 '10 at 3:10

This question came from our site for professional and enthusiast programmers.

    
You will have more lucky if you ask at serverfault.com –  joksnet Dec 17 '10 at 15:27
add comment

7 Answers

The trouble with the seek=<big number> trick is that the filesystem is (usually) clever: if part of a file has never been written to (and is therefore all zeros), it doesn't bother to allocate any space for it - so, as you've seen, you can have a 10GB file that takes up no space (this is known as a "sparse file", and can very useful in some instances, e.g. certain database implementations).

You can force the space to be allocated with (for example):

dd if=/dev/zero of=filename bs=$((1024*1024)) count=$((10*1024))

which will take much longer, but will actually fill the disk. I recommend making the block size much higher than one, because this will determine how many system calls the dd process makes - the smaller the blocksize, the more syscalls, and therefore the slower it will run. (Though beyond 1MB or so it probably won't make much difference and may even slow things down...)

share|improve this answer
    
I usually use bs=(whatever half the cache size is on the drive I'm writing). This has worked well to date when imaging partitions onto larger hard drives for later resizing. ETA: the critical word HALF. –  atroon Apr 5 '11 at 20:34
add comment

You have created what is known as a "sparse file" - a file that, because most of it is empty (i.e. reads back as \0), doesn't take space on the disk besides what is actually written (1B, after 10GB of gap).

I don't believe you could make huge files, taking actual disk space in an instant - taking physical space means the filesystem needs to allocate disk blocks to your file.

I think you're stuck with the oldfashioned "dd if=/dev/zero of=filename bs=100M count=100" which is limited by your drive sequential write speed.

share|improve this answer
add comment

As another option to this, you can use the yes along with a single string and its about 10 tiems faster than running dd if=/dev/urandom of=largefile. Like this

yes abcdefghijklmnopqrstuvwxyz0123456789 > largefile
share|improve this answer
    
This is, surprisingly, about as fast as using dd if=/dev/zero of=filename bs=1M. –  bhinesley Feb 1 '12 at 3:45
add comment

Stop using seek and use a very large bs and/or count. As it is you're making a sparse file, and obviously you need to not do that.

share|improve this answer
add comment

Just to supplement. If you want to use some data rather than pure zero, /dev/urandom may as well be a good idea.

share|improve this answer
add comment

You create a non-sparse 1 TB file with the following command:

dd if=/dev/zero of=bigfile1 bs=10000000000 count=512

This was useful in testing what happens when quotas are over-reached, or a file system fills up.

df -h shows the available space getting smaller.

share|improve this answer
add comment

If you want to literally fill the hard drive, do this:

dd if=/dev/zero of=zeros bs=1M

You can optionally specify count if you want to limit the size, but if you omit the count, it will just run until you're out of disk space.

dd if=/dev/zero of=zeros bs=1M count=10240

As psmears mentioned, you'll get better performance if you set the block size to 1 MB (bs=1M) instead of 1 B (bs=1). This will still take a while, but if you want to check on the progress of your command, open a separate console and run these commands:

ps aux | grep dd

Use the PID of dd in this command (replace PID with dd's PID):

kill -USR1 PID

Then go look at your dd terminal. Of course, this is of limited use when you're just trying to fill up the drive (you could just use df or du to check free disk space or file size, respectively). However, there are other times when it's handy to make dd output its progress.

Extra credit: one practical use for zeroing the free space is that, afterward, you can then delete the "zero" files and dd the entire partition (or disk, if you've zeroed all the partitions) to a disk image file (say, disk-backup.dd), then compress the file. The free space is now highly compressible, so your compressed dd image will be a lot smaller than the original block device whose contents it contains.

Shenanigans: zip up a large zeros file and e-mail it to all your friends. Tell them it's something really cool.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.