Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

When I compile stuff I usually want to do this fast, so on my workstation I issue

make -j16

And (gnu) make starts compiling with 16 cores. However when I am back on my laptop I don't have 16 cores there. So when I there issue the same command my machine freezes to death. I can't switch to alternative terminals via CTRL+ALT+F1 etc., nor remote login will succeed. CTRL+C, CTRL+4 none of them will be regarded. (BTW: no automatic kill will succeed, e.g. automatic out of memory) I have to power off my machine then. (I use Ubuntu 11.10 with a kernel 3.0.x)

One solution is to get in advance the number of cores available on current machine with a small make target where any other "parallel" targets depend on and don't use "make -j $NUMCORES" directly. (I've already done that and realized with a small c++ program using boost threads). But this won't protect me from accidentally specifying "make -j16" again.

Also "too many" cores must not be the same number of cores (including threading cores) as there are available, as +1 or +2 threads wouldn't still kill the machine.

Can I employ ulimits to moderate the problem? I though about specifying to set swap-space to 0. Then I should have a change on aborting I guess.

share|improve this question
1  
For the first solution, export MAKEFLAGS="-j$(nproc)" can be used; no need for special makefiles. For the second, alias make="nice ionice make" perhaps? –  grawity Mar 21 '12 at 11:58
    
Why didn't you posted this as an answer? Thanks! –  math Mar 21 '12 at 12:01
    
Because it doesn't prevent you from running make -j16, it only gives a slight chance of recovering. –  grawity Mar 21 '12 at 12:03
    
Yes you are right ;), However this is much better than the special make target I employed so far! –  math Mar 21 '12 at 12:06
add comment

1 Answer

up vote 5 down vote accepted

You don't need to write any special Makefiles for determining the number of cores; the default flags can be specified in environment and Linux coreutils come with a tool called nproc:

export MAKEFLAGS="-j$(nproc)"

If nproc does not exist in your system, an alternative (also only for Linux) is getconf:

export MAKEFLAGS="-j$(getconf _NPROCESSORS_ONLN)"

Partial protection against complete freezes can be done by running make (along with the entire build process) at low CPU and IO priority:

alias make="nice ionice make"

Note, however, that this will slow down compilation if other processes are using disk IO or the CPU heavily at the same time.


You could also write a wrapper script (or a shell function) that checks all arguments given to it:

make() {
    local arg
    for arg; do
        [[ $arg == -j* ]] && {
            echo "Rejecting '$arg' in make args. Use 'command make ...' to bypass."
            return 1
        }
    done
    command make "$@"
}
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.