49

I'm trying to copy an iso on a usb stick with this command:

sudo dd bs=4M if=lubuntu-17.04-desktop-amd64.iso of=/dev/sdc status=progress

What is quite straight forward. The iso is 912M big in size. Why does the output text freeze here instead of ending the program?

956301312 bytes (956 MB, 912 MiB) copied, 11.216 s, 85.3 MB/s

I can't even stop the program from running with ctr-c.

Improve this question
5
  • 1
    Can you confirm the data is not still being transferred? (tools: iotop, iostat -d 1). Is 85.3 MB/s a realistic transfer rate for your USB stick and USB port? It looks as if dd waited for some buffer to flush, but I would expect it to exit anyway like in this question. Did it exit eventually?
    – Kamil Maciorowski
    Jun 19, 2017 at 21:44
  • It exits eventually, but minutes later. The transfer rate changes to a way lower number than. Basically the total time by the total size, although everything was transferred in the first seconds according to the output. The time is also not updated anymore after those 11 seconds.
    – Natjo
    Jun 19, 2017 at 21:57
  • 1
    It looks to me your dd behaves as if it had conv=fdatasync option enabled by default. I think if it didn't, it would exit like it happens in my previous link; then you would sync and have to wait anyway. Try to force synchronous writing with oflag=dsync. The overall average speed may be lower but the progress report should be accurate. Don't use very small obs in this case because dd will sync the target device after every obs of data.
    – Kamil Maciorowski
    Jun 19, 2017 at 22:22
  • Probably because it can't read the last block - the iso is unlikely to be a multiple of 4 megabytes in size. Use a smaller blocksize, or just use cp - no need to use dd.
    – dirkt
    Jun 19, 2017 at 23:10
  • 2
    @dirkt It's a myth. I have never lost this last block nor had any problems with it. Run dd if=/etc/fstab bs=32M and it will read this small file just fine unless you're short of memory.
    – Kamil Maciorowski
    Jun 19, 2017 at 23:46

1 Answer 1

Reset to default
75

The behavior you're seeing is only dd syncing the data on-disk after the operation. In order to optimize IO operations, Linux often reads data in larger chunks than requested (read-ahead) and delay writes so they can be combined (dirty cache). At the end of the operation, either dd syncs the file on-disk or the kernel does it implicitly, and the process remains active until all the writes are finished.

If there is no other massive IO operations on the machine you should be able to estimate how much data there is left to write by looking at the "Dirty:" value in /proc/meminfo - this is the total amount of data pending to be written to disk.

The amount of data the system can leave unwritten in memory at any one time can be controlled with the following sysctl tuning knobs. By default only the ratios are used. You can define the value in percent (ratio) or bytes.

vm.dirty_background_ratio
vm.dirty_ratio
vm.dirty_background_bytes
vm.dirty_bytes

You will find the official documentation for these parameters here:

https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Improve this answer
3
  • 1
    Does this mean that the 85.3 MB/s is the reading-speed? That might explain why mine is super fast initially, and then drops over time.
    – Jeppe
    Nov 3, 2020 at 20:47
  • 1
    Correct... If you send USR1 signals to print progress you will see what dd thinks it wrote. IIRC the final status also prints before dd blocks so it's not real figures. It may be different with fsync/fdatasync as conv= option for the final status. Using sync/dsync in oflags= will avoid the sync delay but likely slow down the whole process too by avoiding write caches entirely.
    – Thomas Guyot-Sionnest
    Nov 3, 2020 at 22:14
  • 1
    I assume the answer is to just wait it out ?
    – Mike Q
    Oct 4, 2022 at 22:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .