Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I've been given access to a recently built cluster that uses Kerrighed. I'm the first user of this cluster, so setup and configuration problems may be a possibility. I'm also totally new to Kerrighed (and mostly new to cluster computing). I'm having trouble performing some basic tasks and want to check that I'm appropriately conceiving of how the cluster should function.

In particular, I've created a simple OpenMP program as a test case. Based on the way I've setup, compiled and run the program, I expect the program to utilize 24 cores of the cluster. However, this is not the case despite changing the environment variable OMP_NUM_THREADS and other OMP environment variables.

I'm using this program as a test case:

#include <omp.h>

#define NRA 4800                
#define NCA 1000        
#define NCB 10000

int main (int argc, char *argv[])
{
int     tid, nthreads, chunk, i, j, k;

chunk = 100;                    /* set loop iteration chunk size */
double temp;

/*** Spawn a parallel region explicitly scoping all variables ***/
#pragma omp parallel shared(nthreads,chunk) private(temp,tid,i,j,k)
  {
  tid = omp_get_thread_num();
  if (tid == 0)
    {
    nthreads = omp_get_num_threads();
    }

  /*** Put work inside inner loops, parallelize over the outer loop**/
  #pragma omp for schedule (static, chunk)
  for (i=0; i<NRA; i++)
    {
    for(j=0; j<NCB; j++)
      for (k=0; k<NCA; k++)
        temp = (i+k) * (j+k);
    }
  }   /*** End of parallel region ***/

}

I then compile the test program with:

gcc-4.7 -fopenmp test.c -o omp

I set the number of threads as an environment variable

$ OMP_NUM_THREADS=24 ; export OMP_NUM_THREADS

Then run the program:

$ ./omp

When looking at the cluster usage, the command omp only uses 12 cores to execute (there are 84 cores listed in /proc/cpuinfo ). It's my expectation that the steps I've laid out would result in the use of 24 cores since I've run this code as a test on a true SMP machine with those results.

Have I done something wrong in compiling this file for the Kerrighed cluster? Do I have some other major misunderstanding? If not, then what would the next step be in deducing the problem with the cluster?

Thank you for the help, Andrew

share|improve this question

1 Answer 1

Suggest you try here --

https://listes.irisa.fr/sympa/arc/kerrighed.users

http://dir.gmane.org/gmane.linux.cluster.kerrighed.user

Low traffic mailing list, but some kerrighed devs answer tech questions there.

Since your kerrighed cluster is fresh, will probably help to give info on your hardware (how many cores per node? what exact cpu model? all nodes identical?) and on your software versions for openmp/kerrighed/linux.

p.s. AFAIK kerrighed only supports process-migration, as opposed to thread-migration, which could be confusing your core-counting?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.