High latency when writing big files. How to prevent more-than-second page faults when quickly filling up FS?

Question

When I do cat /dev/zero > /some/file all processes tends to become slow (even non-IO related). latencytop shows "Page Faults" are more than 2 seconds. Swap is not used.

Adjusting vm.dirty* does not resolve the issue. The kernel is 3.0.4-zen (happens in various versions e.g. in 2.6.35 too). Already tried "writeback/dirty-throttling-v*" thing - no effect. Filesystem is reiserfs.

1. Is it 12309? Why IO affects page faults even in absence of swap?
2. How to battle it? ionice -c3 for IO-heavy thing seems to have no/low effect. Even explicitly limiting the speed (e.g. to 50% of possible speed) seems to leave high latency issues.

Update: Seems like in Linux 3.3 it behaves better.

Answer 1

I am wondering if filesystem modules can use page faults in the moving of data from user processes to the kernel side or in moving from the filesystem module to underlying block devices. You are creating a massive inrush of data to reiserfs and maybe it has something to do with reiserfs's memory management.

I'm not a kernel hacker by any means but I would try the same thing with an ext2 filesystem and see if you get different results.