Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am working on numerical simulations, and I run them with parameters so that they finish in a couple minutes. I'd like to let them run longer, maybe days.

Currently, I have the parameters compiled into the C programs, but I guess I should make all the parameters command line arguments, so that I can change them on startup.

However, I am not sure how to distribute “tasks” onto several Linux computers. They have multiple cores each, but I may not be able to install extra software, since they are university computers that allow SSH, but no root access. ssh, nohup and screen might be able to get me somewhere, but I'd like to drop in a list with commands, like

run="100-200" mkdir "$run" && cd "$run" && ./main 100 200 && ./plot
run="100-300" mkdir "$run" && cd "$run" && ./main 100 300 && ./plot

It would be great if I could just supply ./main 100 200 && ./plot and it would run it into a new working directory.

Is there some scheduling software for that? I need something for Linux that is free. Bonus points if I can install it into my home directory.


So far, I have found TORQUE, and some people at my university apprently run a cluster with HTCondor.

share|improve this question

1 Answer 1

Install on your servers Sun Grid Engine (I think last free from Sun was 6.2; since Oracle bought it there were some problems to download newer version. However, the old one is more than enough for the ~100 nodes that we use.). This is a proper queueing system. It will take care of scheduling for you. Believe me this work that will pay back.

share|improve this answer
    
I just checked it. Oracle let the SGE go. You can find it on gridscheduler.sourceforge.net –  Kris_R Jun 26 '13 at 14:32
    
SGE does a really good job, but IMHO you cannot install it as non-root. It creates dedicated users, needs an execution daemon and so on... –  mpy Jun 26 '13 at 14:57
    
actually it's even better to not install sge with root account. i use on both cluster that I have sgeadmin. the deamon - there you are probably right. –  Kris_R Jun 26 '13 at 15:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.