Sign up ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

First question on overflow =)... +100 bounty. Couldn't think of something I really cared about until now:

I'm really fed up with the state of Linux desktop responsiveness, e.g. -- in situations with low free-RAM, or situations with high disk throughput, the system slows to a crawl; this is absolutely terrible for applications which require decent performace. Additionally, the UI is completely unresponsive. Compare this for example with OS X, where if an application is hogging resources, one can always option-click to Force Quit it, whereas in Linux I cannot even alt-tab or switch desktop, or even ctrl-alt-f1 to get a terminal -- well I can, it just takes about 1-2 minutes per operation.

I use gkrellm so I can see the situation as it unfolds. Typically the memory utilization becomes pretty high, or the disk throughput jumps dramatically.

It's not bad hardware, with a 2.6GHz quad-core and 4GB of 800MHz DDR2 RAM (would have had 6GB, but due to a hardware incompatibility couldn't mix-and-match with old set). This problem may go away when I inevitable get more RAM, but I don't feel that's the heart of the problem. I even have two swap partitions on different disks.

I feel the problem is threefold:

  • runaway programs that hog up massive amounts of memory -- the law must be laid down for these programs, with limits on their
    • (e.g. tabs on Chrome, each of which is 20-50MB, some of which can use hundreds of MB)
    • (e.g. other programs like update-db and indexers which I've had to disable and remove from cron because they were slowing the system to a crawl whenever they ran, etc.)
  • something terrible going in the kernel or bus contention of some sort, such that high-disk-throughput situations slow the entire system to a crawl (perhaps by paging out important programs)
  • the kernel is not prioritizing UI or important programs in terms of resources, such as memory, paging, even processor utilization

Upvotes go to:

I am thus looking for a solution where all such programs go away. In particular, I am looking for a solution such that the processes will slow down proportionally, while the system and other programs remains entirely unaffected and responsive long enough to manually kill something. Also the window manager process (and anything else that might affect UI responsiveness) should be responsive under all circumstances.

In particular I am intrigued by /etc/security/limits.conf (man limits.conf), but am worried this only gives per-user control, and the commented examples in the file seem rather opaque in terms of description or where to begin. I'm hoping that a limits.conf works, but would not be surprised if it didn't even work, or if it was not an appropriate solution for my problem, or as granular as I'm trying to achieve. A per-process-name limits.conf would be ideal, assuming again that limits.conf works. I'd be happy to try out a limits.conf that people provide, to test if it works, though I'm open to all solutions at this point.

It might also be useful to have any insights on how OS X manages to keep up such good UI responsiveness.

I have already tweaked my /tmp and cache folders to be on tmpfs, and in general disk utilization is near-zero.

Vaguely-related topics:

  • memory overcommit

Answers I do not think will work:

  • swapoff (this still lets memory hog programs get away with murder, and the system permanently freezing if memory is really bad -- upvotes to anyone who can suggest a tweak that invoked the OOM-killer earlier before swapping and targets specific programs)
  • echo ?? > /sys/.../swappiness (no discernable effect)
  • nice (has never worked)
  • ionice (never noticed a difference)
  • selinux (program incompatibility seems to be a nightmare)
  • realtime linux, i.e. can interrupt kernel (don't want to deal with compiling and updating custom kernel; might be okay if it has migrated into repositories)
  • *
share|improve this question
hmm, I seem unable to place a bounty; I guess the link doesn't show up for 48hr?... well I'll be posting the bounty with all reputation I've acquired then –  user76871 May 15 '11 at 15:41
+1, this is the single biggest issue I have with the Linux desktop on a day-to-day basis. I have occasional freezes, perhaps once every couple weeks, but those are not often enough to get particularly annoying. However, it only seems to be an issue with applications that have, as you said, heavy IO utilization: Applications that have heavy CPU utilization have little no no effect on general system performance. Didn't know about ionice, it seems that it would be the correct solution to this problem if it were to work properly. –  crazy2be May 16 '11 at 2:24
3 years later and this is still an issue on Linux. @crazy2be or user76871, I don't suppose you have found a solution in the meantime? –  Glutanimate May 18 '14 at 4:24
@Glutanimate: yes, 32GB of physical RAM and no less (well, maybe 16GB... but that's pushing it), also maaaaaybe large amounts of video RAM. This doesn't fix unresponsiveness due to high CPU or interrupts or whatnot, but it does prevent unresponsiveness in low-memory situations. –  user76871 Jun 12 '14 at 2:23

4 Answers 4

Sounds like your system goes into heavy swapping. Using vmstat 1 may reveal some details - just let it run in a terminal window and switch to it when the slowdown kicks in.

Rather than putting /tmp and "cache" into tmpfs, I would use a normal disk filesystem mounted with the noatime option. Often used data will stay in the caches anyway, and older data can be written to disk to free some RAM for applications. If /tmp and/or cache grows bigger, this might help a lot.

share|improve this answer
+1 for mentioning noatime. –  LawrenceC May 15 '11 at 18:49
Thank you for mentioning noatime, unfortunately I used to use that mount option, and I don't think it helped much to ensure responsiveness (though it helps a ton to ensure the disk isn't overworked); just to be sure I've re-enabled noatime on my current setup. Having a non-tmpfs with noatime seems a bit strange though, since I'd still imagine massive writes must happen. –  user76871 May 16 '11 at 0:58
+1, tried vmstat 1 -- extremely useful in clinching diagnosis that swapping is, in fact, a large part of the issue main issue –  user76871 May 25 '11 at 18:24
Ouch. Never seen a linux system that needed such heavy swapping. Have you checked with df -m how much memory is used up in tmpfs filesystems? Something is eating your RAM relatively fast. –  Turbo J May 26 '11 at 5:25
thanks for the suggestion and teaching me about the -m option. Unfortunately df -h -m seems to indicate a mere 100MB of my memory is in tmpfs, so I doubt it is related any if at all to using memory for tmpfs and caches. This also doesn't seem that uncommon; I have had it happen on multiple distributions when their RAM is pushed to near the limit. –  user76871 May 27 '11 at 3:07

Putting all your temporary and cache files on a tmpfs is lowering the amount of free RAM you have, so you might be causing the system to go to swap sooner than it would need to without this.

It sounds like you have some applications that are relying on some sort of kernel facility or driver that is getting overloaded. You don't go into too much detail about what types of applications other than you are using browsers and indexers, and that you've disabled the indexers.

You might try switching to a desktop environment or window manager that consumes less resources, such as LXDE or IceWM. At work I use a Linux system with LXDE installed and ROX-Filer for a very minimal desktop environment. The purpose of this Linux system is to run VMWare Player so that I can run Windows XP and Windows 7 simultaneously. It's similar hardware specs to what you say and I don't have too many responsiveness issues under this heavy load I'm putting the hardware through. I don't have any responsiveness issues with Linux itself (it's usually the VMs that sometimes make me wait a second, and sharing 1 disk between 2 VMs + 1 OS this is expected) and have always been able to suspend or shutdown the VMs whenever I want to. This includes having Firefox running on Linux often in the background.

So to me it points to some issue with specific applications you are running.

Is DMA enabled for your disk drives? (use hdparm) If you are using full-disk encryption, that requires all disk traffic to go through the CPU which negates much of the benefit of DMA. The effect of that would be that high disk traffic causes CPU to spike which would then slow the entire system down. (EDIT: to clarify, having DMA disabled OR using dm-crypt will cause high CPU during high disk traffic)

share|improve this answer
The point of the question is not that the WM is bloated and causing the system to become slow (it is likely perfectly responsive under normal usage), but that the kernel is not properly prioritizing applications when it runs out of memory and has to go into heavy swapping. I've had this issue on every desktop Linux i've ever used, and while using lighter programs or adding more ram might help, it doesn't address the root of the problem. –  crazy2be May 15 '11 at 20:27
In my previous post I said the following: "It sounds like you have some applications that are relying on some sort of kernel facility or driver that is getting overloaded." So maybe the bottleneck is in a specific kernel module. I'm not a kernel expert, but I'm sure that memory allocation from the kernel side, especially module side, is working differently than userland side. CPU utilization in kernel side is also likely handled differently (don't know if you can "nice" kernel processes). I can't comment further without knowing the specific applications involved. –  LawrenceC May 15 '11 at 21:50
Also if you are using FUSE NTFS that can cause slowness. –  LawrenceC May 15 '11 at 21:50
I am aware that a RAM-based filesystem such as tmpfs (obviously) causes RAM to run out quicker, and that a lightweight WM can slightly reduce symptoms of the underlying issue. I felt pressured to use tmpfs due to the poor responsiveness writing to disk can cause. Nevertheless thank you for your suggestion, especially the part about DMA, which I've added to the list of possibly-related topics. For the record, I believe DMA is enabled, and I am not using a cryptographic filesystem. –  user76871 May 16 '11 at 0:54

This is a common problem with Linux's scheduler. The system slows down to a crawl whenever IO heavy activities occur. There aren't really many things you could do to improve the situation unless you're into kernel hacking :)

Maybe these can help:

share|improve this answer
As I recall, those kernel patches are really only relevant if you are compiling a program or doing something else that is very CPU (and IO?) heavy in a terminal, whilst trying to interact with GUI applications. It does not help in the more common situation where one GUI application is doing some heavy work and you are trying to work with another GUI application, unfortunately. –  crazy2be May 16 '11 at 2:16

I'm no kernel developer but I spent years philosophizing on this issue because I ran into this soooo many times. I actually came up with a metaphor for the whole situation so let me tell you that. I'll assume in my story that things like "swap" don't exist. Swap doesn't make much sense with 32 GB RAMs these days anyways.

Imagine a neighborhood of yours where water is connected to each building through pipes and the towns needs to manage the capacity. Let's assume that you only have a production of 100 units of water per second (and all unused capacity goes to waste because you don't have reservoir tanks). Each home (home = a little app, a terminal, the clock widget, etc.) requires one 1 unit of water per second. This is all nice and good because your population is like 90 so everybody gets enough water.

Now the mayor (=you) decide that you want to open a large restaurant (=browser). This restaurant will house multiple cooks (=browser tabs). Each cook needs 1 unit of water per second. You start with 10 cooks, so the total water consumption for the whole neighborhood is 100 units of water which is still all good.

Now the fun stuff begins: you hire another cook into your restaurant which makes the total water requirements 101 which obviously you don't have. You need to do something.

The water management (=kernel) has 3 options.

1. The first option is just disconnect the service for the homes who didn't use the water recently. This is fine but if the disconnected home wants to use the water again, they will need to go through the lengthy registration process again. The management can disconnect multiple homes to free up more water resources. Actually, they will disconnect all homes which didn't use water recently thus keeping some amount of free water always available.

Although your town keeps functioning, the downside is that progress grinds to a halt. Most of your time is spent on waiting on the water management to reinstate your service.

This is what the kernel does with the file-backed pages. If you run a large executable (like chrome), its file is copied the memory. When low on memory or if there are parts which haven't been accessed recently, the kernel will can drop those parts because it can reload them from the disk anyways. If this is done excessively, this grinds your desktop to a halt because everything will be just waiting for disk IO. Note that the kernel will also drop a lot of least recently used pages when you start doing a lot of IO. This is why it takes ages to switch to a background app after you copied several large files like DVD images.

This is the most annoying behavior for me because I hate hickups and you don't have any control over it. It would be nice to be able to switch it off. I'm thinking of something along the lines of

sed -i 's/may_unmap = 1/may_unmap = (vm_swappiness >= 0)/' mm/vmscan.c

and then you could set vm_swappiness to -1 to disable this. This worked quite well in my little tests but alas I'm no kernel developer so I didn't send it to anyone (and obviously the little modification above is not complete).

2. The management could deny the new cook's request for water. This initially sounds like a good idea. However there are two downsides. First, there are companies which request a lot of water subscriptions even though they don't use them. One possible reason to do this is to avoid all the overhead of talking to the water management whenever they need some extra water. Their water usage goes up and down depending on the time of the day. E.g. in the restaurant's case, the company needs much more water during noon compared to midnight. So they request all possible water they might use but that wastes water allocations during midnight. The problem is that not all companies can foresee their peak usage correctly so they request much-much more in the hopes they will never need to worry about requesting more. Even though this makes capacity planning for the water management hard but in exchange the companies can simplify and speed up their internal processes because they will never need to work with the water management again.

This is what Java's virtual machine does: it allocates a bunch of memory on startup and then works from that. By default the kernel will only allocate the memory when your Java app actually starts using it. However if you disable overcommit, the kernel will take the reservation seriously. It will only allow the allocation to succeed if it actually has the resources for it.

However, there is one other, more serious problem with this approach. Let's say one company starts requesting a single unit of water every day (rather than in steps of 10). Eventually you will reach a state where you have 0 free units. Now this company will not be able to allocate more. That's fine, who cares about the big companies anyways. But the problem is that the small homes will not be able to request more water either! You will not be able the build small public bathrooms to deal with the sudden influx of tourists. You won't be able provide emergency water for the fire in the nearby forest.

In computer terms: In very low memory situations without overcommit you won't be able to open a new xterm, you won't be able to ssh into your machine, you won't be able to open a new tab to search for possible fixes. In other words disabling overcommit also makes your desktop useless when low on memory.

3. Now here's an interesting way of handling the problem when a company starts using too much water. The water management blows it up! Literally: it goes to the restaurant's site, throws dynamites into it and waits until it explodes. This will instantly reduce the town's water requirements by a lot so new people can move in, you can create public bathrooms, etc. You, as the mayor, can rebuild restaurant in the hopes that this time it will require less water. For example you will tell people to not go into the restaurants if there already too many people inside (e.g. you will open less browser tabs).

This is actually what the kernel does when it runs out of all options and it needs memory: it calls the OOM killer. It picks a large application (based on many heuristics) and kills it, freeing up bunch of memory but maintaining a responsive desktop. Actually the Android kernel does this even more aggressively: it kills the least recently used app when the memory is low (compared to the stock kernel which does it only as a last resort). This is called the Viking Killer in Android.

I think this is one of the simplest solution to the problem: it's not like you have more options than this so why not get over it sooner than later, right? The problem is that the kernel sometimes does quite a lot of work to avoid invoking the OOM killer. That's why you see that your desktop is very slow and the kernel is not doing anything about it. But fortunately there is an option to invoke the OOM killer yourself! First, make sure the magic sysrq key is enabled (e.g. echo 1 | sudo tee /proc/sys/kernel/sysrq) then whenever you feel that the kernel is running low on memory, just press Alt+SysRQ,Alt+f.

OK so all that is nice but you want to try it out? The low memory situation is very simple to reproduce. I have a very simple app for that. You will need to run it twice. The first run will determine how much free RAM you have, the second run will create the low memory situation. Note that this method assumes that you have swap disabled (e.g. do a sudo swapoff -a ). Code and usage follows:

// gcc -std=c99 -Wall -Wextra -Werror -g -o eatmem eatmem.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

int main(int argc, char** argv)
    int limit = 123456789;
    if (argc >= 2) {
        limit = atoi(argv[1]);
    setbuf(stdout, NULL);
    for (int i = 1; i <= limit; i++) {
        memset(malloc(1 << 20), 1, 1 << 20);
        printf("\rAllocated %5d MiB.", i);
    return 0;

And here is how you use it:

$ gcc -std=c99 -Wall -Wextra -Werror -g -o eatmem eatmem.c
$ ./eatmem
Allocated 31118 MiB.Killed
$ ./eatmem 31110
Allocated 31110 MiB.Killed

The first invocation detected that we have 31,118 MiB of free RAM. So I told the application to allocate 31,110 MiB RAM so that kernel will not kill it but eat up almost all my memory. My system froze: even the mouse pointer did not budge. I've pressed Alt+SysRQ,Alt+f and it killed my eatmem process and the system got restored.

Even though we covered our options what do in a low memory situation, the best approach (just like any other dangerous situation) is to avoid it in the first place. There are many ways to do this. One common way I've seen is to put the misbehaving applications (like browsers) into different containers than the rest of the system. In that case the browser won't be able to affect your desktop. But prevention itself is outside of the question's scope so I won't write about it.

TL;DR: Although currently there is no way to fully avoid paging, you can mitigate full system-halt by disabling overcommit. But your system will still be unusable during low-memory situation but in a different way. Regardless of above, in a low-memory situation press Alt+SysRQ,Alt+f to kill a large process of the kernel's choosing. Your system should restore its responsiveness after a few seconds. This assumes you have the magic sysrq key is enabled (it is not by default).

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.