using one disk as cache for others

Question

Given a PC with several hard drives: Is it possible to use one fast disk as a giant file cache?

I.e. automatically copying frequently accessed data to that one disk, and transparently redirecting reads and writes to that disk, so that other drives would only have be accessed occassionally.

(writes would have to be forwarded to the other disks after a while of course)

Advantages:

• the other drives could be powered down most of the time; reducing power, heat, noise
• speed of the other drives would not matter much.
• cache disk could be solid state.

How can I set such a system up?

What OS supports these options? Is this possible at all using Windows or Linux?

p.s.

Example:

There are 3 disks with 1 Tb each. most of the files are only accessed very rarely, but about 5% of each disk is used frequently. Which files are used frequently may change over time. A solid state disk with 150GB should cache the currently frequently accessed files, so that access time is faster and the drives can be put into power saving mode.

Answer 1

It's possible for both Linux and Windows and, in fact, is a good idea. For Windows you can move it in the Advanced settings under performance of your system. Linux depends on the distro but you normally establish it during installation.

Putting the paging on another disk is a good possible boost. In fact, Microsoft has a temporary paging option called "Readyboost" which does this with a thumb drive!

As far as if it's going to be smart about WHAT is cashed is dependent on the OS and really relies more on the Physical RAM of the device. Late Linux versions have excellent cashing as well as Microsoft's SuperFetch and Readyboost on Windows Vista and 7. Older OSes are not so smart.

Answer 2

You're thinking of something like bcache. It doesn't quite give you all the things you're looking for though:

the other drives could be powered down most of the time; reducing power, heat, noise

Not really. If the drive is a cache, writes have to be synchronized back to the original disk. Unless you've got a largely read-only workflow (most people don't), you're just not going to see this.

Also, for a cache like this, you'd really only want to be caching things that the cache does better than the underlying storage. If you're talking SSDs and HDDs -- small random I/O. HDs are already good at large sequential I/O.

Answer 3

Actually, something just like that would be quite easy, provided you dont mind using ZFS filesystem for all disks. Then you can put cache (L2ARC cache) and write log (ZIL) onto SSD ... ergo: fast acces to your often used files without need to move them around! ... and never worrying that you get out of space on SSD and having to shuffle files/programs around in worst possible moment (that is, when you neeed to do something asap)

only trouble is ZFS Fuse implementaion aint that fast ... zfs.kqinfotech.com might go to the rescue sometime later ...