1

We have a sixty terabyte block device which we want to zero using dd. Is it possible to parallelize this operation in order to reduce the amount of time it takes?

I've tried running the following command, but this is not quite what I'm looking for:

seq 1 5 | parallel dd if=/dev/zero of=/dev/mapper/mydevice{} bs=10M count=10
3
  • Is it a single physical device? HDD or SDD?
    – Paul
    Jul 28, 2015 at 1:57
  • If it is a single HDD (which I guess it isn't, given the size), it won't be any faster, as the head will then have to move between the two write-positions. If it's an MD-array, you could destroy the array and run dd on the various disks in parallel. If it's a SAN disk or something like that, you are out of luck I guess, unless you use tooling from the SAN vendor.
    – Sig-IO
    Jul 28, 2015 at 9:39
  • It's an MD-array but we don't want to destroy the array. We've got LVM and LUKS on top of the linux software RAID array. We want to allocate block data with zeros to ensure that the outside world will see this as random data i.e. protect against disclosure of usage patterns. Jul 29, 2015 at 2:39

3 Answers 3

1
seq 0 600000 | parallel dd if=/dev/zero of=/dev/mapper/my60device bs=10M count=10 seek={}0

You already use if, of, bs, and count so I assume you know what they do.

seek seeks n blocks into the output file before doing the dd.

You do not write how the 60 TB device is physically arranged on spindles. That can have a huge impact. So this might be faster:

seq 0 60000 | parallel --shuf dd if=/dev/zero of=/dev/mapper/my60device bs=10M count=100 seek={}00

It writes 1 GB blocks and shuffles the execution, which may speed up or slow down depending on the physical layout.

4
  • Can you expand your answer to explain the parameters? Thanks.
    – fixer1234
    Jul 28, 2015 at 20:55
  • Thanks! I was trying to use --skip instead of --seek . The former acting on the input, the latter acting on the output. D'hoi! Jul 29, 2015 at 2:37
  • As far as how the 60 terabyte block device is physically arranged on spindles: it's 12 SATA drives in a 6 Gbps SAS backplane with software linux RAID5 on top. Jul 29, 2015 at 2:38
  • Then parallelizing output to the device will most likely not help: When you write block 1 and block n-1 at the same time, they will both cause all drives to be active = no good. So if you can, break the raid, dd on each device separately (parallel -j0 dd if=/dev/zero of={} bs=1M ::: /dev/sd... ) and after that re-create the raid.
    – Ole Tange
    Jul 29, 2015 at 12:37
1

Or you could try my home made parallel data write tool pdd It works pretty well on NVMe drives.

1
  • Welcome to Super User. Can you expand your answer a little with the "how" so it is more self-contained? Thanks.
    – fixer1234
    Apr 25, 2019 at 3:34
0

Unless the underlying device driver supports scheduling two I/O operations at once through some bizzare ioctl stuff, you can't issue explicitly parallel I/O operations to a device.

dd is the wrong tool for that anyway since it works by writing data to a file or block device sequentially - the best you can do is tell it to write a large block size which lets the operating system optimize for sequential access better - which does improve the speed of things like writing to a tape.

Spinning two dd's on a single block device won't change the fact that most I/O devices, especially things that look like disk drives, can really only do a single thing or small queue of things at once - and any optimization (i.e. leveraging NCQ) is done at the device driver level.

1
  • Yeah we've done performance tests with large arrays with/without NCQ in the past, but with a different hardware RAID controller and no LUKS encryption. The last time we zero'd a 22 terabyte LUKS array it took about 8 hours. I'm going to try @OleTange's suggestion and see if we can do better than 21 hours on a 60 terabyte array. Jul 29, 2015 at 2:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .