Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

When I compile stuff I usually want to do this fast, so on my workstation I issue

make -j16

And (gnu) make starts compiling with 16 cores. However when I am back on my laptop I don't have 16 cores there. So when I there issue the same command my machine freezes to death. I can't switch to alternative terminals via CTRL+ALT+F1 etc., nor remote login will succeed. CTRL+C, CTRL+4 none of them will be regarded. (BTW: no automatic kill will succeed, e.g. automatic out of memory) I have to power off my machine then. (I use Ubuntu 11.10 with a kernel 3.0.x)

One solution is to get in advance the number of cores available on current machine with a small make target where any other "parallel" targets depend on and don't use "make -j $NUMCORES" directly. (I've already done that and realized with a small c++ program using boost threads). But this won't protect me from accidentally specifying "make -j16" again.

Also "too many" cores must not be the same number of cores (including threading cores) as there are available, as +1 or +2 threads wouldn't still kill the machine.

Can I employ ulimits to moderate the problem? I though about specifying to set swap-space to 0. Then I should have a change on aborting I guess.

share|improve this question
1  
For the first solution, export MAKEFLAGS="-j$(nproc)" can be used; no need for special makefiles. For the second, alias make="nice ionice make" perhaps? – grawity Mar 21 '12 at 11:58
    
Why didn't you posted this as an answer? Thanks! – math Mar 21 '12 at 12:01
    
Because it doesn't prevent you from running make -j16, it only gives a slight chance of recovering. – grawity Mar 21 '12 at 12:03
    
Yes you are right ;), However this is much better than the special make target I employed so far! – math Mar 21 '12 at 12:06
up vote 5 down vote accepted

You don't need to write any special Makefiles for determining the number of cores; the default flags can be specified in environment and Linux coreutils come with a tool called nproc:

export MAKEFLAGS="-j$(nproc)"

If nproc does not exist in your system, an alternative (also only for Linux) is getconf:

export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"

Partial protection against complete freezes can be done by running make (along with the entire build process) at low CPU and IO priority:

alias make="nice ionice make"

Note, however, that this will slow down compilation if other processes are using disk IO or the CPU heavily at the same time.


You could also write a wrapper script (or a shell function) that checks all arguments given to it:

make() {
    local arg
    for arg; do
        [[ $arg == -j* ]] && {
            echo "Rejecting '$arg' in make args. Use 'command make ...' to bypass."
            return 1
        }
    done
    command make "$@"
}
share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .