I have a task receiving daemon that runs on many machines. Arbitrary tasks get sent by clients that are then executed on the server, sometimes taking a little memory and sometimes pushing up to the limit that I set in the daemon's config (tasks will never exceed the limit, even with linux's overcommit behavior). I want to keep the limit as high as possible, so as to allow tasks to run and complete, but not so high that pressing up against the limit the box becomes entirely unresponsive, and not so high that the tasks are actually slowing themselves down by bogging down the box. I want to always be able to ssh in.
Is there a formula or technique that I could use to determine how big to set the limit? The best idea I have now is to do a fresh boot of the box without any tasks running and check memory usage (not counting cache/buffers) and assume that's what's needed to keep things running. Then I would subtract that from how much physical memory the box has to get the limit.