I am running ubuntu 13.04 and currently if any user accidently writes some code that uses too much RAM it brings the whole system down so badly I have to switch the power off and on again. Is there some way to configure users so that by default they are limited to using 4GB of RAM, say? Or is there another way to make the out of memory killer more aggressive than it currently is?
You can use ulimit to do this (not an expert but the manpage is generally helpful) or edit the configuration file directly.
The only part that really needs explanation is "hard". Users can set their own limits but according to TFM those are "soft" limits which can only be set up to the hard limit. Hopefully your users can't sudo to root.