2

I'm setting up a small cluster of 20 machines that will have to run a batch command. The batch will have some instructions then run a Java program that will crunch data for about 2h then return the results.

My experience with clusters is zero.

Could you please advice on general guidelines to follow and available software?

Thank you in advance!

Improve this question
2
  • 1
    What operating system do you plan to run on your clustered machines?
    – user1931
    May 11, 2011 at 13:17
  • @John T Linux, CentOS. Thks.
    – Frankie
    May 11, 2011 at 13:48

2 Answers 2

Reset to default
3

CentOS, being a derivative of Red Hat, comes with the Red Hat Cluster Suite. The Red Had Cluster Suite is more geared towards high availability (service failover) and load balancing of IP services (such as web servers). Unless your Java program listens for commands over the network, RHCS is probably not what you're looking for.

By description, I believe you're after a distributed computing approach. GNU Queue or Gearman would be more suitable for your needs. GNU Queue's development has been halted for some time now but older releases are still available and usable.

Improve this answer
1
  • thanks for the great advice. Eventually I went with Condor but you set me up on the tracks. Gnu Queue, if weren't abandoned, could have been right on target.
    – Frankie
    May 17, 2011 at 20:45
1

If you're just getting started with HPC, you might enjoy listening to http://www.rce-cast.com

Episode 7 is about cluster planning: http://www.rce-cast.com/Podcast/rce-07-cluster-planning.html

As I mentioned in my answer to your other question, we use Condor. We're happy with it.

Improve this answer
1
  • ended up using Condor. Have gave it some test runs and so far seams great! Thanks!
    – Frankie
    May 17, 2011 at 20:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .