Background: For some friends in a local academic institution, I wrote a Java program that simulates the growth of the crystal in a microcellular matrix as a concurrent program that implements 100,000 MonteCarlo iterations, each iteration comprises of 14,240^2 steps. First implemented in 2-dimensions, its able to support up to 14,240x14,240 size matrix, each cell is a byte, and associated data structures to support algorithms to compute areas of the grains etc.
The program has been written as 4 concurrent threads that use the quad core CPU (100%) with each thread executing 1/4th the number of steps, and the threads "recombine" at end of the 14,240^2/4 trials, and start the next iteration, where I fork new threads. The access to the common matrix etc. is controlled by using
synchronize keyword, and 4 threads are controlled to close by using a semaphore.
Also, I built the hardware for them – Intel Core i5 2500K processor, with 16GB RAM, dual partitioned into Windows (for their use) and Linux – for my testing.
Problem: These friends now want me to try and help write a 3-Dimensional version and are willing to concede to a size of 10,000x10,000x10,000 matrix.
My idea: Clearly one single machine will not be able to handle this. Is it possible to use a cluster of two-three such machines? If so, can anyone please guide me to a open source, free, metaware or a Java implementation that can execute JVM(s) on these nodes, and then run this program ?
On one single machine I am able to touch up to a 400x400x400 matrix only. I am willing to rewrite the code so that each quadrant of the matrix is processed on different cluster, with each quadrant trials but need to know how to execute the program concurrently in different machines and how do I synch the computations.
I remain grateful for any suggestion or idea.