Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm trying to figure out the optimal size for a large copy from my hard drive using dd. I'm trying to figure out what the best blocksize to use it, which I would assume is the hardware block size for that drive.

share|improve this question

5 Answers 5

Linux exposes the physical sector size in files /sys/block/sdX/queue/physical_block_size. Although, to get the best performance you should probably do a little testing with different sizes and meassure. I could not find a clear answer in that using exactly the physical block size would get the optimal result (although I assume it cannot be a bad choice).

share|improve this answer
1  
i've got a Debian Lenny system (2.6.26 kernel) that only exposes a hw_sector_size in that location, and a newer Ubuntu Karmic system (2.6.31 kernel) that provides both. so this is somewhat dependent on the kernel in use. –  quack quixote Mar 18 '10 at 10:46

The lsblk command is great for this:

lsblk -o NAME,PHY-SeC

The results:

NAME   PHY-SEC 
sda        512 
├─sda1     512 
├─sda2     512 
└─sda5     512 
share|improve this answer
    
Does it differentiate between logical size and physical size? –  CMCDragonkai Jul 4 at 6:33
$ sudo hdparm -I /dev/sda | grep -i physical
Physical Sector size:                  4096 bytes

http://nxadm.wordpress.com/2010/04/30/4096-physical-block-size-drives/

share|improve this answer

Mine isn't intended to be a complete answer, but I hope it also helps.

Here is a little something from http://mark.koli.ch/2009/05/howto-whole-disk-backups-with-dd-gzip-and-p7zip.html


3 - Determine the Appropriate Block Size

For a quicker backup, it can help to nail down the optimal block size of the disk device you are going to backup. Assuming you are going to backup /dev/sda, here's how you can use the fdisk command to determine the best block size:

rescuecd#/> /sbin/fdisk -l /dev/sda | grep Units

Units = cylinders of 16065 * 512 = 8225280 bytes

Note the fdisk output says "cylinders of 16065 * 512". This means that there are 512 bytes per block on the disk. You can significantly improve the speed of the backup by increasing the block size by a multiple of 2 to 4. In this case, an optimal block size might be 1k (512*2) or 2k (512*4). BTW, getting greedy and using a block size of 5k (512*10) or something excessive won't help; eventually the system will bottleneck at the device itself and you won't be able to squeeze out any additional performance from the backup process.(emphasis added)


I suspect the difference in performance between a near-optimal and optimal block size for a given configuration is negligible unless the data set is enormous. Indeed, a user at FixUnix (post from 2007) claimed his optimal times were only 5% faster than the sub-optimal ones. Maybe you can squeeze a little more efficiency out by using a multiple of the "cluster" size or filesystem block size.

Of course, if you move too far away to either side of the optimal block size you'll run into trouble.

The bottom line is you will likely gain only around 5% in performance (i.e. 3 minutes per hour) with the absolute optimal block size, so consider whether it is worth your time and effort to research further. As long as you stay away from extreme values, you should not suffer.

share|improve this answer
1  
some reason for using echo "p" | /sbin/fdisk /dev/sda... instead of /sbin/fdisk -l /dev/sda... ? the second will be cleaner and won't attempt to make any changes. –  quack quixote Mar 18 '10 at 15:30
    
You would be best asking Mark Kolich (linked). He was creating a backup, and I only quoted a section of his article. –  Mark C Mar 18 '10 at 15:35
1  
@MarkC The linked article uses /sbin/fdisk -l /dev/sda | grep Units. It might have been changed in the last two years. In any case, I've updated your answer. –  Bob Nov 15 '12 at 11:10
    
Thanks, @Bob. | –  Mark C Nov 17 '12 at 10:33

Each disk transfer generates an interrupt that processor must handle. Typical 50Mb/s disk will want to generate 100000 of them each second at 512b block size Normal processor would handle 10s of thousands of those, thus bigger (2^x) block size would be more handy (4k as default FS block size in most systems up to 64k ISA DMA size) would be more practical...

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.