up vote 1 down vote favorite
share [g+] share [fb]

I'm setting up a small cluster of 20 machines that will have to run a batch command. The batch will have some instructions then run a Java program that will crunch data for about 2h then return the results.

My experience with clusters is zero.

Could you please advice on general guidelines to follow and available software?

Thank you in advance!

link|improve this question
40% accept rate
1  
What operating system do you plan to run on your clustered machines? – John T May 11 '11 at 13:17
@John T Linux, CentOS. Thks. – Frankie May 11 '11 at 13:48
feedback

2 Answers

up vote 3 down vote accepted

CentOS, being a derivative of Red Hat, comes with the Red Hat Cluster Suite. The Red Had Cluster Suite is more geared towards high availability (service failover) and load balancing of IP services (such as web servers). Unless your Java program listens for commands over the network, RHCS is probably not what you're looking for.

By description, I believe you're after a distributed computing approach. GNU Queue or Gearman would be more suitable for your needs. GNU Queue's development has been halted for some time now but older releases are still available and usable.

link|improve this answer
thanks for the great advice. Eventually I went with Condor but you set me up on the tracks. Gnu Queue, if weren't abandoned, could have been right on target. – Frankie May 17 '11 at 20:45
feedback
up vote 1 down vote

If you're just getting started with HPC, you might enjoy listening to http://www.rce-cast.com

Episode 7 is about cluster planning: http://www.rce-cast.com/Podcast/rce-07-cluster-planning.html

As I mentioned in my answer to your other question, we use Condor. We're happy with it.

link|improve this answer
ended up using Condor. Have gave it some test runs and so far seams great! Thanks! – Frankie May 17 '11 at 20:46
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.