I'm working on a bit of software for which it would be useful for us to write betweeen 100 and 200 files a second to disk. The files vary in size from 1k to 3k. Ubuntu is the operating system, using ext4 file system.
So far I have seen some odd behaviour. At first the inodes ran out, causing us to get "out of disk space" errors. But to combat this we've moved to a large partition and massively increased the number of inodes.
However, I am still getting "No space left on device" errors. On different machines. With
df -i and
df -h reporting plenty of free space and plenty of free inodes.
From speaking to people I've had answers varying from "it should be fine" to "that's far too many files to be writing to disk a second". I've experienced this behaviour on 3 different Ubuntu machines.
Is there a definitive answer to this? For this many writes should I just be using a database or should the operating system be able to handle this?