iostat is still the king of detailed I/O information, the kind of info that can confirm if your SSD is living up to expectations, for example.
$ iostat -xht 5 nvme0n1
Linux 4.15.0-43-generic (myhost) 06/02/2021 _x86_64_ (8 CPU)
06/02/2021 08:59:20 AM
avg-cpu: %user %nice %system %iowait %steal %idle
2.6% 0.1% 0.8% 0.1% 0.0% 96.4%
Device r/s w/s rkB/s wkB/s rrqm/s wrqm/s %rrqm %wrqm r_await w_await aqu-sz rareq-sz wareq-sz svctm %util
95.03 20.80 1.8M 300.1k 0.00 13.15 0.0% 38.7% 0.24 0.57 0.03 19.2k 14.4k 0.02 0.3%
Every five seconds, this will print detailed IO stats for the NVMe drive. The most important ones are usually not actually read/write bandwidth --
wKB/s -- but rather the reads and writes per second (
w/s, aka IOPS) and, perhaps most important, the average time a process waited for each read and write --
w_await (in milliseconds).
All of these values feed into each other using the surprisingly useful Little's law formula, translated a bit to
latency = queue_size / IOPS, or
await = aqu_sz / (r/s + w/s).