21

Sometimes, when I write images to a flash drive, this happens:

$ sudo dd if=install57.fs of=/dev/sdc
573440+0 records in
573440+0 records out
293601280 bytes (294 MB) copied, 0.549231 s, 535 MB/s

Basically, Linux caches everything, writes nothing, and dd exits. After I type sync, it starts writing the data (flash drive LED starts blinking).

Why does this happen?

1
  • 3
    Are you sure /dev/sdc is an actual device on your system and you aren't writing to a file /dev/sdc? Do an ls --color /dev - /dev/sdc should be yellow if it's a device.
    – LawrenceC
    Aug 21, 2015 at 14:13

3 Answers 3

21

Use this instead:

sudo dd if=install57.fs of=/dev/sdc conv=fsync

This calls the fsync() after every write() system call. This forces dd not to cache anything. See this part of the manpage of fsync (man 2 fsync):

fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache 
pages for) the file referred to by the file descriptor fd to the disk device (or other 
permanent storage device) where that file resides. The call blocks until the device reports 
that the transfer has completed. It also flushes metadata information associated with the 
file (see stat(2)).

This is the kernels default behavior. Linux kernels manage write and read caches like this: When the write() syscall is issued, the data is quickly written to the cache and a write-completed status is sent to the process. When the buffer is needed or when there is free time on the bus, the data is written from the cache to the hard disk.

4
  • 1
    I like that your answer and mine cover almost entirely different approaches. Nice, +1. Aug 21, 2015 at 14:15
  • 1
    @ChrisInEdmonton dito +1
    – chaos
    Aug 21, 2015 at 14:17
  • all answers are good in this question. Aug 21, 2015 at 20:58
  • @chaos Just to clarify: This is the kernels default behavior.-Do you mean that conv=fsync is default when writing to block devices such that dd does not cache anything? Looking for a good answer to this: unix.stackexchange.com/questions/312687/… Sep 28, 2016 at 11:55
10

This happens because Linux, and most other operating systems, cache both reads and writes. In most cases, this makes your operating system more responsive.

If you want to ensure the cached data has been written, you want to use sync, as you know. Linux exposes a significant number of settings you can tune as well. This article gives a good overview of some settings. You could set the vm.dirty_background_bytes to 0, for example, to ensure the kernel kicks off the flusher threads immediately.

7

sync(8) - Linux man page :

The kernel keeps data in memory to avoid doing (relatively slow) disk reads and writes. This improves performance, but if the computer crashes, data may be lost or the file system corrupted as a result. sync ensures that everything in memory is written to disk.

Note: unmount (or eject) automatically calls sync which "hides" this in normal filesystem usage.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .