26

I'm trying to copy an iso on a usb stick with this command:

sudo dd bs=4M if=lubuntu-17.04-desktop-amd64.iso of=/dev/sdc status=progress 

What is quite straight forward. The iso is 912M big in size. Why does the output text freeze here instead of ending the program?

956301312 bytes (956 MB, 912 MiB) copied, 11.216 s, 85.3 MB/s

I can't even stop the program from running with ctr-c.

5
  • 1
    Can you confirm the data is not still being transferred? (tools: iotop, iostat -d 1). Is 85.3 MB/s a realistic transfer rate for your USB stick and USB port? It looks as if dd waited for some buffer to flush, but I would expect it to exit anyway like in this question. Did it exit eventually? – Kamil Maciorowski Jun 19 '17 at 21:44
  • It exits eventually, but minutes later. The transfer rate changes to a way lower number than. Basically the total time by the total size, although everything was transferred in the first seconds according to the output. The time is also not updated anymore after those 11 seconds. – Natjo Jun 19 '17 at 21:57
  • 1
    It looks to me your dd behaves as if it had conv=fdatasync option enabled by default. I think if it didn't, it would exit like it happens in my previous link; then you would sync and have to wait anyway. Try to force synchronous writing with oflag=dsync. The overall average speed may be lower but the progress report should be accurate. Don't use very small obs in this case because dd will sync the target device after every obs of data. – Kamil Maciorowski Jun 19 '17 at 22:22
  • Probably because it can't read the last block - the iso is unlikely to be a multiple of 4 megabytes in size. Use a smaller blocksize, or just use cp - no need to use dd. – dirkt Jun 19 '17 at 23:10
  • 2
    @dirkt It's a myth. I have never lost this last block nor had any problems with it. Run dd if=/etc/fstab bs=32M and it will read this small file just fine unless you're short of memory. – Kamil Maciorowski Jun 19 '17 at 23:46
40

The behavior you're seeing is only dd syncing the data on-disk after the operation. In order to optimize IO operations, Linux often reads data in larger chunks than requested (read-ahead) and delay writes so they can be combined (dirty cache). At the end of the operation, either dd syncs the file on-disk or the kernel does it implicitly, and the process remains active until all the writes are finished.

If there is no other massive IO operations on the machine you should be able to estimate how much data there is left to write by looking at the "Dirty:" value in /proc/meminfo - this is the total amount of data pending to be written to disk.

The amount of data the system can leave unwritten in memory at any one time can be controlled with the following sysctl tuning knobs. By default only the ratios are used. You can define the value in percent (ratio) or bytes.

vm.dirty_background_ratio
vm.dirty_ratio
vm.dirty_background_bytes
vm.dirty_bytes

You will find the official documentation for these parameters here:

https://www.kernel.org/doc/Documentation/sysctl/vm.txt

2
  • Does this mean that the 85.3 MB/s is the reading-speed? That might explain why mine is super fast initially, and then drops over time. – Jeppe Nov 3 '20 at 20:47
  • Correct... If you send USR1 signals to print progress you will see what dd thinks it wrote. IIRC the final status also prints before dd blocks so it's not real figures. It may be different with fsync/fdatasync as conv= option for the final status. Using sync/dsync in oflags= will avoid the sync delay but likely slow down the whole process too by avoiding write caches entirely. – Thomas Guyot-Sionnest Nov 3 '20 at 22:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.