6

I use dd to measure write perfomance and observed something strange: writing to /data/emzed2 is faster than writing to /data. This is the way I measured the performance:

$ dd if=/dev/zero of=/data/emzed2/testfile bs=32

^C4921834+0 records in

4921834+0 records out 157498688 bytes (157 MB) copied, 2,87329 s, 54,8 MB/s

$ dd if=/dev/zero of=/data/testfile bs=32

^C2487991+0 records in

2487991+0 records out 79615712 bytes (80 MB) copied, 2,6501 s, 30,0 MB/s

both folders are on the same partition on a SSD drive. I use Ubuntu 14.04 and repeated the experiment several times with similar results.

Any idea whats going on ?

3
  • 1
    I have similar measurements on 13.10. Roughly a 50% speed increase. Jul 7, 2014 at 15:49
  • 1
    I guess this belongs to SU. You didn't mention the file system, it's probably ext4. My guess would be that /data is a partition, and that the transfer at the root of this partition is faster as it benefits from ext4 optimizations, including "The fourth trick is that all the inodes in a directory are placed in the same block group as the directory, when feasible." (ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout)
    – Paul Guyot
    Jul 7, 2014 at 20:57
  • It is ext4 and /data is a partition and this effect only occurs in partition roots ! Can you write a regular answer so that I can accept it ? Jul 10, 2014 at 7:18

1 Answer 1

4

It seems that you are hit by ext4 optimizations. Some of these are described in Ext4 Disk Layout documentation.

Every time you grow a file, for example with dd, some place is allocated for the new data. Eventually, the file could be fragmented, i.e. data could be spread in several chunks through the disk. Fragmentation is a well known source of slowness (when reading and writing) and therefore file system implementations often try to avoid it. Fragmentation is way more expensive with spinning disks (the head needs to travel to successive chunks) than with SSD, yet most file system implementations use spinning-disk optimized strategies with SSD.

A combination of third, fourth and fifth "tricks" described in the documentation mentioned above might explain why writing to the subdirectory is faster in your case.

The fifth trick is that the disk volume is cut up into 128MB block groups. […] When a directory is created in the root directory, the inode allocator scans the block groups and puts that directory into the least heavily loaded block group that it can find.

As a result, emzed2 was probably created in an empty block of 128MB.

The fourth trick is that all the inodes in a directory are placed in the same block group as the directory, when feasible.

Consequently, /data/testfile is created in (probably heavily loaded) root block group, while /data/emzed2/testfile is created in (probably empty) emzed2 block group.

The third trick […] is that it tries to keep a file's data blocks in the same block group as its inode.

For /data/emzed2/testfile, the file system will first allocate all data blocks in the emzed2 block group. If this block was initially empty, for the first 128MB, this means no fragmentation at all. For /data/testfile, the file system will first fill the root block group if it was not filled yet, and then look for other places to store data.

Also, you are growing the file 32 bytes at a time. Fortunately, file systems such as ext4 allocate more data than what you request (by chunks of 8kb in the case of ext4) and tries to delay allocation. You probably would see a similar pattern with bs=8196, but the speed difference might decrease with larger block sizes.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .