write-through RAM disk, or massive caching of file system?

Question

I have a program that is very heavily hitting the file system, reading and writing randomly to a set of working files. The files total several gigabytes in size, but I can spare the RAM to keep them all mostly in memory. The machines this program runs on are typically Ubuntu Linux boxes.

Is there a way to configure the file system to have a very very large cache, and even to cache writes so they hit the disk later? I understand the issues with power loss or such, and am prepared to accept that. Crashing aside, in normal operation the writes should eventually reach the disk!

Or is there a way to create a RAM disk that writes-through to real disk?

Answer 1

Consider using an ext4 filesystem using fast-and-loose mount options:

noatime,data=writeback,nobh,barrier=0,commit=300

to postpone writing data from cache back to physical disk.

Other than that, you could use aufs to union-mount a tmpfs-filesystem on top of your regular filesystem, do all the writing and then merge the tmpfs back down to the real filesystem afterwards.

Answer 2

Are you seeing high numbers of IO waits, indicating that the read and write requests are not being satisfied via existing buffers? As others have noted, Linux is very good about giving spare RAM to buffers, so you should check this first.

If you're not seeing IO waits, then it's possible that your performance problems (do you even have problems? your question doesn't say) are due to kernel context switches from lots of small program-initiated IO operations. In this case you can gain a significant performance boost by rewriting your application to use memory-mapped files. But that's more of a question for StackOverflow.

Answer 3

Linux by default uses any spare RAM as a file cache, so no configuration is necessary for that.

You may want to consider using ext4 as the filesystem. It uses quite a number of techniques to speed up disk access, including delayed allocation which:

This has the effect of batching together allocations into larger runs. Such delayed processing reduces CPU usage, and tends to reduce disk fragmentation, especially for files which grow slowly. It can also help in keeping allocations contiguous when there are several files growing at the same time.

Data loss is pretty rare due to the use of journaling.

Ext4 is now the default filesystem in recent releases of Linux, though you will probably want to make sure the kernel you use is at least 2.6.30

Answer 4

You can see my answer here: Reserve RAM for cache and buffer. If you want to reserve memory for cache and buffer: echo 10 > /proc/sys/vm/vfs_cache_pressure when 100 is the default value. Then you can limit the max ram used by each app: echo 8192 > /proc/sys/vm/max_map_count.

Answer 5

Use mmap on those files - it uses the kernel's superior caching, making it much faster.

Answer 6

Linux uses your RAM very efficiently.
I don't think you really need to work on getting things cached with large files.
To check this out for yourself, try these things.

The Linux disk cache is very unobtrusive. It uses spare memory to greatly increase disk access speeds, and without taking any memory away from applications.

Check how much RAM your system has and if you can scale it up.
There is no need for a RAM-disk for this purpose,
the disk cache will work well with all available RAM space.

Answer 7

RapidCache provides a RAM disk with write-through, but is currently "in beta".