Question

I have been considering building a simple beowulf cluster with cheap, old parts. Would it be more cost effective to simply build a computer with a single multi-core processor? I would be using the cluster to run repeated regressions on fairly large datasets.

Difficulty is no issue here, I would enjoy building a cluster, but I'm a college student, so cost is definitely an issue. Any advice would be appreciated.

Answer 1

There's always the new usage of the GPU for scientific computation.

The article Scientific computing on the GPU, which contains some interesting links, says:

The scientific distributed computing project, Folding@Home is seriously investing into this approach and claims to be able to obtain a performance of about 100 gigaflops from a standard desktop PC/Mac. Recall, that today a high-end desktop can deliver a few tens of gigaflops at most, using all the cores of its CPU. So, it's an order of magnitude improvement in performance!

See also this article : From CPU to GPU.

And remember that one PC can contain several GPUs, and that way be the equivalent of a large cluster.

Answer 2

I was in a similar situation as you few years ago, and after searching for a way to do it with more hardware I settled on something a bit different. Instead of buying more hardware and making a mess of my apartment and cranking up my electricity bill, I wrote a server and a client that would connect to the server and get some data to work on. I distributed the client to about 20-30 of my friends and in no time I got a pretty respectable "cluster" computer.

Now, my problem was embarrassingly parallel and it was rather trivial to write both client and the server (I used Python, fwiw) and a bunch of my friends had powerful computers mostly running idle, so this may or may not work in your case. Still, it my be a viable alternative.

Answer 3

If electricity bill isn't an issue, I'd searched for Case + CPU + motherboard with build-in LAN and video.

Many people have old PCs or parts that they can't sell, and would give them away for nothing or a bottle of beer. If you're a student, maybe it's worth putting announcements in the dorms that you buy for old parts.

But if you pay for electricity, calculate it's cost carefully.

Answer 4

Use Amazon EC2. http://aws.amazon.com/ec2/

You are able to startup as many servers as you need, run your computations, then shut them down and only pay for the time that they are running by the hour. Very cost effective versus building your own cluster.