Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I use dd in its simplest form to clone a hard drive

dd if=INPUT of=OUTPUT

However, I read in the manpage that dd knows a blocksize parameter. I was wondering whether there is an optimal value for the blocksize parameter that will speed up the cloning procedure?

Cheers!

Stefan

share|improve this question
add comment

5 Answers

up vote 2 down vote accepted

64k seems to be a good pick:

Results:

  no bs=        78s     144584+0 records
  bs=512        78s     144584+0 records
  bs=1k         38s     72292+0 records
  bs=2k         38s     36146+0 records
  bs=4k         38s     18073+0 records
  bs=5k         39s     14458+1 records
  bs=50k        38s     1445+1 records
  bs=500k       39s     144+1 records
  bs=512k       39s     144+1 records
  bs=1M         39s     72+1 records
  bs=5M         39s     14+1 records
  bs=10M        39s     7+1 records

(taken from here).

this matches with my own findings regarding read/write buffering for speeding up an io-heavy converter-program i was once pimping @work.

share|improve this answer
add comment

dd will happily copy using the BS of whatever you want, and will copy of partial block.

Basically, the block size (bs) parameter seems to set the amount of memory thats used to read in a lump from one disk before trying to write that lump to the other.

If you have lots of RAM, then making the BS large (but entirely contained in RAM) means that the I/O sub-system is utilised as much as possible by doing massively large reads and writes - exploiting the RAM. Making the BS small means that the I/O overhead as a proportion of total activity goes up.

Of course in this there is a law of diminishing returns. My rough approximation is that a block size in the range about 128K to 32M is probably going to give performance such that the overheads are small compared to the plain I/O, and going larger won't make a lot of difference. The reason for the lower bound being 128K to 32M is - it depends on your OS, hardware, and so on.

If it were me, I'd do a few experiments timing a copy/clone using a BS of 128K and again using (say) 16M. If one is appreciably faster, use it. If not, then use the smaller BS of the two.

share|improve this answer
add comment

Yes, but you won't find it without lots of testing. I've found that 32M is a good value to use though.

share|improve this answer
add comment

For those that end up here via Google, even if this discussion is a bit old...

Keep in mind that dd is dumb for a reason: the simpler it is, the fewer ways it can screw up.

Complex partitioning schemes (consider a dual-boot hard drive that additionally uses LVM for its Linux system) will start pulling bugs out of the woodwork in programs like Clonezilla. Badly-unmounted filesystems can blow ntfsclone sky-high.

A corrupt filesystem cloned sector-by-sector is no worse than the original. A corrupt filesystem after a failed "smart copy" may be in REALLY sorry shape.

When in doubt, use dd and go forensic. Forensic imaging requires sector-by-sector copies (in fact, it can require more sectors than you're going to be able to pull off with dd, but that's a long story). It is slow and tedious but it will get the job done correctly.

Also, get to know the "conv=noerror,sync" options, so that you can clone drives that are starting to fail-- or make ISOs from scratched (cough) CDs-- without it taking months.

share|improve this answer
add comment

First, there are much better tools to clone a hd than dd. A simple file copy ( cp -a ) is a perfectly good way. If you want to keep the same fs UUID and the boot loader intact ( as well as any fragmentation present ), then clonezilla or ghost4linux are good candidates.

Because dd is a dumb animal, it wastes time copying free space, and requires that the destination be at least as large as the source, and any additional size is unused, requiring you to manually expand the fs later.

As for what block size is best, it depends on a number of factors. The largest is whether the source and destination are different partitions on the same disk, or different disks ( on different controllers ). Whether you use the direct flag is important as well. If you do not enable direct and you are copying between two disks, then very large block sizes ( above 1 MB ) actually slow things down, since the block cache will make dd wait a bit after such a large write before it can read from the source again so you don't keep both disks busy. If you are copying between two partitions on the same disk, then this delay can actually be a good thing and larger block sizes are better because there are fewer seeks.

I usually find using the direct flag with 32 or so MB block size works best going between partitions on the same disk, and 128-512k either without the direct flag, or with direct but piping one input dd to another output dd for drive to drive.

share|improve this answer
    
Thanks for the answer. Sorry for the late reply. Thanks for pointing out alternatives to dd. I was working in a sort of restricted lab environment and falling back to dd was really necessary, but good to know about the alternatives for future work. –  Phi Mar 29 '11 at 12:09
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.