Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I run a Python program with a memory leak, I would normally expect the program to eventually die with MemoryError. But instead, what happens is that all the virtual memory is used until my disk runs out of space. I am running Mac OS X 10.8 on a retina MacBook Pro. My computer generally has between 10GB to 20GB free. Mac OS X is smart enough to not die completely when the disk runs out of space (rather, it gives me a dialog letting me force quit my GUI programs).

Is there a way to make Python just die when it runs out of real memory, or some reasonable amount of virtual memory? This is what happens on Linux, as far as I can tell. I guess Mac OS X is more generous than Linux with virtual memory (the fact that I have an SSD might be part of this; I don't know just how smart OS X is with this stuff). Maybe there's a way to tell the Mac OS X kernel to never use so much virtual memory that leaves less than, say, 5 GB free on the hard drive?

share|improve this question
    
"Maybe there's a way to tell the Mac OS X kernel to never use so much virtual memory that leaves less than, say, 5 GB free on the hard drive?" Your suggested fix wouldn't help. If you make it run out of memory sooner, the problem will just occur sooner. – David Schwartz Nov 21 '12 at 0:19
    
Oh, yeah, you're right. Duh :) I guess I want just Python to do that, not the whole system. – asmeurer Nov 21 '12 at 0:30

Python Level

According to this post, resource.setrlimit() maybe what you need.

Example

#!/usr/bin/python

import resource
import sys
import signal
import time

import os

soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
print 'Soft limit starts as  :', soft

# Use env MY_PY_SET_LIMIT to control limit value
# If MY_PY_SET_LIMIT is not set, RLIMIT_STACK will not change
MY_PY_SET_LIMIT = os.getenv('MY_PY_SET_LIMIT')

if MY_PY_SET_LIMIT != None :
  resource.setrlimit(resource.RLIMIT_STACK, (int(MY_PY_SET_LIMIT), int(MY_PY_SET_LIMIT)))

soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
print 'Soft limit changed to :', soft

TMP = ""

for i in range(10240):
  TMP += "0123456789"
  print len(TMP)

System Level

For Linux, it is actually answer multiple times before on various "board" of stackexchange and other sites too. The best answer I found is here which contain an example.

The answer is use ulimit -v < kByte >. For example, limiting the vm to 10M:

ulimit -v 10240

However, on OS X there are indication (here & here) that ulimit maybe ignore. Those links are very old. I am not sure if situation change in more recent OS X releases.

There is this post for OS X to use launchd conf. It suggest using a Stack section in a plist config

<key>SoftResourceLimits</key>
<dict>
    <key>Stack</key>
    <integer>10000000000</integer>
</dict>

Or with /etc/launchd.conf

launchd.conf

umask 002
limit stack 67104768 67104768
limit maxproc 3400 4500
limit maxfiles 256 unlimited
setenv PATH /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

PS: according to Mountain Lion man launchd.conf(5) per user launchd.conf is not support

 $HOME/.launchd.conf  Your launchd configuration file (currently unsupported).
share|improve this answer
    
ulimit indeed doesn't seem to work. – asmeurer Dec 15 '12 at 2:01
    
that left us no choice but go with launchd – John Siu Dec 15 '12 at 2:08
    
It's not clear where that should go. It sounds maybe like it should be part of some info.plist in an .app package, but Python doesn't work like that. – asmeurer Dec 15 '12 at 2:12
    
updated answer to clarified – John Siu Dec 15 '12 at 2:21
    
update with a potential Python solution. – John Siu Dec 15 '12 at 3:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .