Filter squid cached files to multiple cache dirs

Question

I am currently setting up a small home network - I've chosen to go with squid proxy and I am wondering if it is possible to setup a single squid instance with multiple cache_dirs so that different files (more precisely, files with different sizes) end up on different cache_dirs?

The reason for this is that I'll be running squid as a VM on a machine with semi-cannibalized hardware. I have a small SSD (which I'm hoping to use for caching small, dynamic, very transient files such as web pages, small pics and such) and a large, slow HDD (which I'd like to use for caching larger, static content such as windows updates, youtube transfers, large pics and the like).

This is my first foray into squid and it's only a small network so I would like to avoid introducing complexities such as multiple squid instances or hierarchical caches. Ideally, there would be a way to specify some simple set of criteria that would place the given file either in cache dir A or cache dir B.

I have sifted through SquidFAQ but I found nothing similar mentioned there so I may have missed something really elemental.

How could this be done? Is there a pattern people use to cater for such cases? Is it even necessary/advisable?

Answer 1

Hey i came across your question earlier, trying to setup a similar thing.

Your post is old, but not unseen. So maybe if there is somebody out there, like i was yesterday, this makes sense:

fast SSD, but expensive per MB and slow HDD, but cheap per MB. We just need a folder on any filesystem on both of them. Dont use the same physical device twice of course. Your answer: you can tier them using the special shared option min-size and max-size:

cache_dir rock /mnt/ssd1/squid 1024 min-size=0 max-size=65536

and

cache_dir aufs /mnt/hdd1/squid 250000 16 256 min-size=65536

This will spread the cache over 2 cache_dirs, the first for the chunks < 64kb in a "Rock" database (this will become a file /mnt/ssd1/squid/rock/rock) on the fast SSD that has a maximum size of 1 GB*.

And the latter for the chunks that are larger than that, in a folder structure with the "aufs" store type /mnt/hdd1/squid/16 folders/256 folders (still experimenting myself, might try diskd) with a maximum size of 250GB *=until squid will start its retention scheme. On that, i chose cache_replacement_policy heap LFUDA, why is beyond the scope here. But what is relevant is to mention is that you need to be carefull with this setup if you overlap the min-max boundaries for the different cache_folders.

If for instance if had a filesystem on a second HDD, i could add that like cache_dir aufs /mnt/hdd2/ 250000 16 256 min-size=65536. This would completely overlap the other cache_dir on hdd1.

Or, you could have a 3 tier setup, make sure you read up on the squid.conf parts TAG: cache_dir (note your answer in the end part there, under "COMMON OPTIONS") and especially the last part of TAG: store_dir_select_algorithm. If you have overlapping min-max boundaries.