1

I'm using GNU parallel to run multiple jobs in parallel like this: parallel python3 main.py arg ::: args*. With single-threaded jobs, I used -j 100% to ensure that 1 job is mapped to 1 core.

Now, each job has four threads and I have 32 cores. I want to distribute the jobs such that they do not overlap, i.e., running 8 jobs at a time each on four different cores. I don't get how to do this. I tried -j8, which I thought would limit the number of parallel jobs. But it still starts all jobs at once - and limits them to 8 cores? -j 25% didn't work either...

Improve this question

1 Answer 1

Reset to default
0

Both -j 8 and -j 25% will do the same on a 32 core machine, so if it does not work, then you have found a bug. Make an MCVE stackoverflow.com/help/mcve.

But first: Get the newest version to make sure you are not being a hit by a bug already fixed.

Improve this answer
2
  • Just to clarify: Is -j supposed to limit the number of jobs running on however many cores or the number of cores processing however many jobs?
    – stefanbschneider
    Jul 11, 2017 at 14:46
  • -j limits the number of jobs running in parallel at any given time. In GNU Parallel terminology it is called job slots.
    – Ole Tange
    Jul 12, 2017 at 1:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .