I have recently seen Fit-PC small form factor Linux machines on sale which have a very low power usage. Would it be feasible to create a cluster of these and just plug and play to expand the cluster?
Is it feasible? Certainly. Is it plug-n-play? Well, aside from software loading, yes (and even that can be done via network booting).
I bring this up because, if you just want to create an ultra-low-power cluster, the Fit-PC2 might be reasonable (though the product that Molly's answer links to might be a better choice).
Personally I think the Atom Z530 processor will be a bit underpowered for most applications you'd want to cluster, and the dual-core Atom330 would be a better foundation for an Atom-based cluster. But again it depends on your application and budget.
The Fit-PC is certainly a cool product, but building your cluster around a cool component is not good design practice. The first step in cluster design is laying out the requirements -- what will the cluster actually do? Once you know that, then you can worry about what components will help you achieve that goal.
Interesting idea, but this may not be the most cost efficient solution, as the price of the fit-PC is quite steep.
if you're thinking of building an Atom-based cluster computer, consider some Toradex Robin modules instead, they're cheaper (and even smaller, credit card size):
Price starts at $160, up to 20% discount if you buy in bulk.