2

When I run a piece of python code that uses too much RAM, my computer would start to swap so badly that I had to do a hard reboot. To fix this I turned off swap assuming that the OOM killer would step in. However, I still get exactly the same symptoms! That is I can hear the hard drive thrashing once all the RAM is used, the computer freezes and I have to do a hard reboot.

What could be going on and how can I fix it?

My system is running Ubuntu 14.04.2 LTS, 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux with 8GB of RAM.

My /etc/fstab shows

# / was on /dev/sda1 during installation
UUID=80919e96-e83b-4c88-b30e-8673e1faa3b4 /               ext4    errors=remount-ro 0       1
# swap was on /dev/sda5 during installation
#UUID=68064725-c1bb-4293-b8d1-45b2a42fb7b4 none            swap    sw              0       0

Further details

tail -n+1 /proc/sys/vm/overcommit_*
==> /proc/sys/vm/overcommit_memory <==
0

==> /proc/sys/vm/overcommit_ratio <==
50

cat /proc/sys/vm/oom_kill_allocating_task
0

Example code

I am sorry to say I don't know how to make a minimalist python example that shows this problem. However this is non-minimalist code that definitely does http://paste.ubuntu.com/11621823/. Simply change the value of "path" in the code and run it. On my system it uses up all the RAM and then completely freezes the system.

2
  • 1
    Well, is the OOM killer enabled on your system?
    – arkascha
    Jun 7, 2015 at 8:01
  • @arkascha I am not sure how to check but I added some more info to the question. Is there something else I can usefully add?
    – Simd
    Jun 7, 2015 at 8:44

1 Answer 1

2

Your problem is your python code and it has nothing to do with swap.

Swap space is used by operating system when load is increasing on memory. Then those processes which are not used currently are moved to swap space which is a partition. In this way more memory can be free for process which is currently running. When processes moved to swap are used then it is moved from swap area back to memory.

This way swapping is helpful to tackle situations where more memory is required to run something than the amount of free memory available.

Since you turn off swap this is not going to solve your problem but only increase it. Only solution for your problem which i can think of is to either optimize your python code to use less memory or increase memory in your machine.

Also it would be better if you can put your code here so that people can help in optimizing it.

6
  • 1
    Thanks for this but I can't help but feel that a user shouldn't be able to completely kill a system by just using too much RAM.
    – Simd
    Jun 7, 2015 at 8:42
  • 1
    @felipa If you run something that starts taking too much RAM normally operating system will start killing processes which is called OOM killer. This post might give you more insight on problem. unix.stackexchange.com/questions/92525/can-linux-run-out-of-ram
    – shivams
    Jun 7, 2015 at 8:56
  • 1
    Thanks. That's the problem. The OOM killer doesn't seem to do that on my system and I would like to fix that. Also, I am not sure why the disk is thrashing if I have no swap.
    – Simd
    Jun 7, 2015 at 8:58
  • Did you see I added the code? I still have no idea how to fix this.
    – Simd
    Jun 14, 2015 at 14:30
  • Bullshit. Even Windows deals with memory overload gracefully: it issues a warning and suggests a kill and never lags in the process.
    – Yurii
    Nov 9, 2017 at 10:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .