How do I test to see if all writes to my hard drive are aligned to its 4k sectors?

Question

I'm using Linux with 4 hard drives that use 4k sectors. There are several layers between my filesystem and the raw devices: Disks > Linux Raid 5 > dm-crypt > LVM.

Every resource I have found has explained how to set up each layer to ensure that writes on top of that layer will be aligned to the 4k sector boundary. However, I have not found anything that explains how to verify that the writes made to the hard drives are actually happening at 4k boundaries.

I'm not interested in re-examining my set up to use logic to determine if it is aligned correctly. I want to examine what is actually happening when writes are made to the disk.

How can I log or view the addresses and size of the writes that are being made to my hard drives, so I can verify that they are correctly aligned?

Answer 1

Asked myself the same question some time ago and simply did the following:

Wrote with the shell a couple of times a rather unusual string to a file (something like "WackaWacka") Then simply searched with a hex dump (used od) the actual content of the disk and checked wether the first occurence of the string was stored exactly on the beginning of a 4k block.

Hint: Do not use an editor - it may create temporary files you don't know of which may contain the strings, too. Do it this way:

 $ for i in 1 2 3 4 5 ...
 >  do
 >   echo "WackaWacka!"
 >  done > mytestfile

So .sh_history may contain the search string, but not 5 times in a row ;-)

And then, just search:

 # sync
 # od -c /dev/sda | grep 'W   a   c   k   a'

Well, it's best done on a rather empty disk to avoid serching through Gigabytes of data ;-)

Answer 2

Write a 4k block and watch how much data is read/written with iostat (The 'Blk_read' 'Blk_wrtn' columns). If the data is not aligned, a write will trigger reads first and will trigger more than 4k of writes.

You'll need to be careful to not measure any metadata updates, though... or just drown them out by making 1000s of 4k writes.... So make sure nothing else is scanning disks or holding open files (I think lsof would be sufficient?), then open a new file, wait, run iostat, write 4k to the file, sync the write (or just wait a while?) then check iostat again.

This seems to give a reasonable output for me:

iostat  -d /dev/hdb3
dd if=/dev/urandom of=/mount/path/ofhdb3/tmptest bs=4k count=10000 conv=fdatasync
iostat  -d /dev/hdb3

Note iostat's man page claims to report in 512 byte blocks, and I see just over 80000 additional blocks were written, and no blocks read. If your alignment is off, you'll see an similar number of reads (since to write a mis-aligned 4k, requires reading the two blocks impacted, mutating them, and writing them back). In fact, the only reason alignment is important is to avoid such reads (so that's really what you want to look for: does a write workload trigger reads?)