Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I try to tweak my development system to maximal reliability. I disabled swap, because for GUI usage it mostly renders the machine unresponsive in such a way not useable anymore. Nevertheless, if agressive appications eat up the memory, some mechanisms seem to kick in that making the most out of it on cost of speed. There is no harddrive swap operation, but the system is getting unresponsive likewise. So I want to let the OOM killer kick in before the system make any special efforts on memory gain. Is it possible to configure the OOM killer to act if there is less than 100 MB free physical memory for example?

share|improve this question
2  
I think the real issue here is, there's not enough ram to start with. You won't use swap unless there's no ram. By turning off swap... you run out of ram and have no where to page it to. Which causes ugly things to happen. Your system seems to be set up badly, and no amount of tweaking will fix that. –  Journeyman Geek Mar 29 '12 at 8:51
2  
I don't agree. Development and 'power use' often involves experimental usage. For example, when using a command line image processing tool, there are no specs how much memory it's operation take in relation to the image size. So I just give it a run. And I don't expect it to render my whole machine useless. For a single experiment, I could use ulimit to keep it secured, but for whole system operation with sometimes plenty of operations, the containment of one process is not so usefull but a 'life insurance' for the whole machine definetly is. –  dronus Mar 29 '12 at 11:14
1  
The fact that your system grids to a halt when using swap is suspect. Your computer is using swap cause its out of memory. Swap is slowing down cause disk access is slow. Disk access is slow due to ???. Its problems all the way down. Its not just that you're low on ram. Its that you can't use the one way to mitgate that due to something else. –  Journeyman Geek Mar 29 '12 at 13:35
2  
@JourneymanGeek, you are off in left field. Disks are slow compared to ram, period, hence heavy swapping always grinds the system to a halt. Of course he is out of memory because he tried running a program that uses a lot of memory. The question is what to do when out of memory? Kill the hog, or slow down due to having no memory left for the disk cache. –  psusi Mar 29 '12 at 17:38
1  
@psusi: You don't understand how swap space works, with slow down being an assumption. We are all running page files and swap spaces world wide; and then you come with the suggestion to disable it, you have to explain a little bit more than that... –  Tom Wijsman Mar 29 '12 at 17:57
show 4 more comments

3 Answers 3

The default policy of the kernel is to allow applications to keep allocating virtual memory as long as there is free physical memory. The physical memory isn't actually used until the applications touch the virtual memory they allocated, so an application can allocate much more memory than the system has, then start touching it later, causing the kernel to run out of memory, and trigger the out of memory (OOM) killer. Before the hogging process is killed though, it has caused the disk cache to be emptied, which makes the system slow to respond for a while until the cache refills.

You can change the default policy to disallow memory overcommit by writing a value of 2 to /proc/sys/vm/overcommit_memory. The default value of /proc/sys/vm/overcommit_ratio is 50, so the kernel will not allow applications to allocate more than 50% of ram+swap. If you have no swap, then the kernel will not allow applications to allocate more than 50% of your ram, leaving the other 50% free for the cache. That may be a bit excessive, so you may want to increase this value to say, 85% or so, so applications can allocate up to 85% of your ram, leaving 15% for the cache.

share|improve this answer
    
@TomWijsman, how do you figure? –  psusi Mar 29 '12 at 18:52
    
Changing these values from there defaults without theoretical background is not going to reach in a more reliable system, you can only justify that change with proper statistics. Just because you can change it doesn't mean you should. If you constantly in low memory conditions that means that you are using more memory than you have and should buy more memory, it doesn't mean you should fiddle with your settings and kill random applications. Interrupting with your daily working or introducing corruption, that's really not the way to go... –  Tom Wijsman Mar 29 '12 at 18:56
    
@TomWijsman, the question makes it clear that he isn't constantly in low memory conditions; he just sometimes runs a command that takes an unexpectedly large amount of memory. Buying more memory is not the the only solution when you run out. Other potential solutions include finding better ways to make use of the memory you have, or just not doing whatever needs that much memory. The question makes it clear that the latter is more acceptable than going out and buying more ram. –  psusi Mar 29 '12 at 19:02
    
Which line in the question makes this clear? I see the opposite given in I disabled swap, because for GUI usage it mostly renders the machine unresponsive in such a way not useable anymore.. He mentioned GUI, while you are assuming he runs a command. Buying more memory is the first solution, using less memory yourself is the second solution, making your system unstable by fiddling with the stable defaults is the last solution. The question doesn't have to be answered literally, so I don't see what's your problem that you have to bother both of us in the comments. Rant doesn't help... –  Tom Wijsman Mar 29 '12 at 19:06
    
Hey, this answer sounded quite cool. Unfortunately, the 'commit' refers to virtual memory demand it seems, which is quite bad estimated by application programmers. For example with my (no swap) desktop running, there is about 400 of 2000mb physical memory used, but 1600mb 'commit'ted as /proc/meminfo's Committed_AS states. With some applications running, this value easily exceeds the physical memory so it's hard to set a feasable limit by this. –  dronus Mar 29 '12 at 22:17
show 1 more comment

I also struggled with that issue. I just want my system to stay responsive, no matter what, and I prefer losing processes to waiting a few minutes. There seems to be no way to achieve this using the kernel oom killer.

However, in the user space, we can do whatever we want. So i wrote the Early OOM Daemon ( https://github.com/rfjakob/earlyoom ) that will kill the largest process (by RSS) once the available RAM goes below 10%.

Without earlyoom, it has been easy to lock up my machine (8GB RAM) by starting http://www.unrealengine.com/html5/ a few times. Now, the guilty browser tabs get killed before things get out of hand.

share|improve this answer
add comment

Reliability isn't reached by low memory conditions and an OOM killer.

It is wrong to organize a party in a closet and place "cleaning out my closet" on your small playlist.

Is it possible to make the OOM killer intervent earlier?

Doing this will have unintended side results, because you have no control over what is killed.

I try to tweak my development system to maximal reliability.

Maximal reliability involves testing your system and improving your system based on these tests.

Just tweaking random things won't get you anywhere...

I disabled swap, because for GUI usage it mostly renders the machine unresponsive in such a way not useable anymore. Nevertheless, if agressive appications eat up the memory, some mechanisms seem to kick in that making the most out of it on cost of speed.

Due to low memory conditions, disabling the swap won't improve the behavior, it does the opposite.

To increase reliability in this situation, add more memory such that your system is more responsive and there are no random processes being killed without the user's intention. You shouldn't resort on low memory conditions and a mechanism like this, especially not in a development environment...

There is no harddrive swap operation, but the system is getting unresponsive likewise.

Low memory conditions indeed result in unresponiveness, whether you have a swap or not.

So I want to let the OOM killer kick in before the system make any special efforts on memory gain.

Special efforts that will do more harm than good, as I explained above. Instead, you could kill processes you don't need yourself, but I guess you can't do that so the OOM will kill processes that you need.

Is it possible to configure the OOM killer to act if there is less than 100 MB free physical memory for example?

Might be, but you get a higher return on investment if you just buy some extra memory which doesn't really cost much these days. Consider that you're going to hit yourself in the foot on the long run if you continue to work on low memory conditions. OOM is like a bailiff, it doesn't assist you, it assists the OS...

share|improve this answer
1  
Of course disabling swap improves the behavior because instead of thrashing the disk, the OOM kicks in and kills the memory hog. Running out of ram isn't the problem ( and adding more just means you have to try harder to run out ). The problem is what to do when you DO run out. You want the OOM to kill the hog, and thus relieve the low memory condition. –  psusi Mar 29 '12 at 17:43
1  
Because killing an application that is trying to use more memory than you have is preferable to bringing the entire system to its knees. In a perfect world you would have unlimited memory and never run out, but in reality, sometimes you run out by accident and would rather be told "not enough memory" than have the system grind to a halt. –  psusi Mar 29 '12 at 18:25
1  
Buying some extra memory might solve some problems, depending on the amount bought. But it doesn't change the fact that there may be inexpected usages by orders of magnitude. So I want the application to fail, but NOT the system under those conditions. Some examples: Process a folder full of compressed images, most of them "normal" size, but some of them really large. A small mistake could make a dead loop with memory runaway eating 1GB/s. Accidentally open a video file in a text editor. Usually this ends with symptoms like jerky mouse and almost dead UI until the OOM kicks in. –  dronus Mar 29 '12 at 22:03
1  
I have no problems with killing applications that would run dead either. Consider a system with 2GB physical + 2GB swap. An application that quickly runs out the physical memory can easily eat the swap too. It would just die later, after rendering the system unresponsive for minutes to hours. So why not kill it quickly before GUI operation get flaky? Much processes do all their work with 10mb, some take 1gb, and some rare would need 10gb, that's life. –  dronus Mar 29 '12 at 22:43
1  
@TomWijsman: I don't like introducing anything into the application because this is a general problem that can be caused by several applications. That's why I still see the OOM as the apropriate tool. ulimit -v is unrealiable because it can only limit one application and doesn't care about the total available ressources. –  dronus Apr 5 '12 at 8:22
show 10 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.