Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

If I run a Python program with a memory leak, I would normally expect the program to eventually die with MemoryError. But instead, what happens is that all the virtual memory is used until my disk runs out of space. I am running Mac OS X 10.8 on a retina MacBook Pro. My computer generally has between 10GB to 20GB free. Mac OS X is smart enough to not die completely when the disk runs out of space (rather, it gives me a dialog letting me force quit my GUI programs).

Is there a way to make Python just die when it runs out of real memory, or some reasonable amount of virtual memory? This is what happens on Linux, as far as I can tell. I guess Mac OS X is more generous than Linux with virtual memory (the fact that I have an SSD might be part of this; I don't know just how smart OS X is with this stuff). Maybe there's a way to tell the Mac OS X kernel to never use so much virtual memory that leaves less than, say, 5 GB free on the hard drive?

share|improve this question
    
"Maybe there's a way to tell the Mac OS X kernel to never use so much virtual memory that leaves less than, say, 5 GB free on the hard drive?" Your suggested fix wouldn't help. If you make it run out of memory sooner, the problem will just occur sooner. –  David Schwartz Nov 21 '12 at 0:19
    
Oh, yeah, you're right. Duh :) I guess I want just Python to do that, not the whole system. –  asmeurer Nov 21 '12 at 0:30

1 Answer 1

up vote 9 down vote accepted
+50

Python Level

According to this post, resource.setrlimit() maybe what you need.

Example

#!/usr/bin/python

import resource
import sys
import signal
import time

import os

soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
print 'Soft limit starts as  :', soft

# Use env MY_PY_SET_LIMIT to control limit value
# If MY_PY_SET_LIMIT is not set, RLIMIT_STACK will not change
MY_PY_SET_LIMIT = os.getenv('MY_PY_SET_LIMIT')

if MY_PY_SET_LIMIT != None :
  resource.setrlimit(resource.RLIMIT_STACK, (int(MY_PY_SET_LIMIT), int(MY_PY_SET_LIMIT)))

soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
print 'Soft limit changed to :', soft

TMP = ""

for i in range(10240):
  TMP += "0123456789"
  print len(TMP)

System Level

For Linux, it is actually answer multiple times before on various "board" of stackexchange and other sites too. The best answer I found is here which contain an example.

The answer is use ulimit -v < kByte >. For example, limiting the vm to 10M:

ulimit -v 10240

However, on OS X there are indication (here & here) that ulimit maybe ignore. Those links are very old. I am not sure if situation change in more recent OS X releases.

There is this post for OS X to use launchd conf. It suggest using a Stack section in a plist config

<key>SoftResourceLimits</key>
<dict>
    <key>Stack</key>
    <integer>10000000000</integer>
</dict>

Or with /etc/launchd.conf

launchd.conf

umask 002
limit stack 67104768 67104768
limit maxproc 3400 4500
limit maxfiles 256 unlimited
setenv PATH /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin

PS: according to Mountain Lion man launchd.conf(5) per user launchd.conf is not support

 $HOME/.launchd.conf  Your launchd configuration file (currently unsupported).
share|improve this answer
    
ulimit indeed doesn't seem to work. –  asmeurer Dec 15 '12 at 2:01
    
that left us no choice but go with launchd –  John Siu Dec 15 '12 at 2:08
    
It's not clear where that should go. It sounds maybe like it should be part of some info.plist in an .app package, but Python doesn't work like that. –  asmeurer Dec 15 '12 at 2:12
    
updated answer to clarified –  John Siu Dec 15 '12 at 2:21
    
update with a potential Python solution. –  John Siu Dec 15 '12 at 3:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.