I am running a command (pngquant to be precise: https://github.com/pornel/pngquant) in a terminal window. I noticed, that if I open 4 terminal windows, and run pngquant command in each of them, I get 4x speed increase, effectively compression 4 times as many images in the same time as before.

So I used this approach and assigned each pngqunat process a portion of images I want to compress, effectively creating multiple processes on multiple threads

Can you run command on multiple threads without doing this tricks that I did? I would like to just say "run pngquant compression on all this images and use all threads available."

  • 4
    You could employ parallel. – simlev May 10 '17 at 16:33

You are looking for the exact same system this recent question required. The answer given (python concurrent-based script) will perfectly fit your needs once adapted a small bit for your simpler case.

| improve this answer | |

Expanding my previous comment into an answer.

Gnu parallel is the command line tool conceived to run jobs simultaneusly thus making use of multiple threads, cores, CPUs, PCs. xargs now offers similar functionality with the addition of the -P switch.

I ran a test for you on a two core VPS, using pngquant to convert a bunch of .png files (126 images averaging 9MB in size).

pngquant *.png took 26 minutes.

find . -iname "*.png" | parallel pngquant {} took 14 minutes.

top shows the parallel execution of two pngquant commands at a time:

 5808 user    20   0   70948  59164   2140 R 100.0  2.9   0:03.49 pngquant
 5811 user    20   0   70952  61000   2024 R  99.3  3.0   0:05.67 pngquant
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.