Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm setting up a small cluster of 20 machines that will have to run a batch command. The batch will have some instructions then run a Java program that will crunch data for about 2h then return the results.

My experience with clusters is zero.

Could you please advice on general guidelines to follow and available software?

Thank you in advance!

share|improve this question
1  
What operating system do you plan to run on your clustered machines? –  John T May 11 '11 at 13:17
    
@John T Linux, CentOS. Thks. –  Frankie May 11 '11 at 13:48

2 Answers 2

up vote 3 down vote accepted

CentOS, being a derivative of Red Hat, comes with the Red Hat Cluster Suite. The Red Had Cluster Suite is more geared towards high availability (service failover) and load balancing of IP services (such as web servers). Unless your Java program listens for commands over the network, RHCS is probably not what you're looking for.

By description, I believe you're after a distributed computing approach. GNU Queue or Gearman would be more suitable for your needs. GNU Queue's development has been halted for some time now but older releases are still available and usable.

share|improve this answer
    
thanks for the great advice. Eventually I went with Condor but you set me up on the tracks. Gnu Queue, if weren't abandoned, could have been right on target. –  Frankie May 17 '11 at 20:45

If you're just getting started with HPC, you might enjoy listening to http://www.rce-cast.com

Episode 7 is about cluster planning: http://www.rce-cast.com/Podcast/rce-07-cluster-planning.html

As I mentioned in my answer to your other question, we use Condor. We're happy with it.

share|improve this answer
    
ended up using Condor. Have gave it some test runs and so far seams great! Thanks! –  Frankie May 17 '11 at 20:46

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.