1

I am running a Cromwell pipeline executor on Ubuntu 18.04, which by definition runs other programs in an orchestrated manner. I am trying to find a way to impose TOTAL CPU usage limits on individual executions, especially on those with scattering. Although this can be attempted on Docker containers (if a task is executed inside one), I am wondering if there is a good top-level solution.

When I run Cromwell, it is owned by my user and it is single threaded. The processes it further spawns are owned by root, they usually take up to 100% but can be very numerous.

I tried using cpulimit with --include-children flag to give the app a total of two cores (out of 40):

sudo java -jar /nas/files/cromwell-35.jar run test.wdl --inputs test_inputs.json &
sudo cpulimit --include-children --limit=200 --pid=$!

but it imposes limits on individual executions not their TOTAL CPU usage (summed together):

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                 
26581 root      20   0 1946776 239476  34984 R 100.0  0.1   0:10.03 suppa.py                                                                
26770 root      20   0 1939040 231784  34892 R 100.0  0.1   0:09.79 suppa.py                                                                
26811 root      20   0 1920448 213176  34636 R 100.0  0.1   0:09.36 suppa.py                                                                
27338 root      20   0 1937880 230936  35048 R 100.0  0.1   0:08.81 suppa.py                                                                
27662 root      20   0 1928868 221892  34896 R 100.0  0.1   0:08.64 suppa.py                                                                
27751 root      20   0 1920044 212844  34744 R 100.0  0.1   0:08.49 suppa.py                                                                
27786 root      20   0 1922524 215464  35064 R 100.0  0.1   0:08.03 suppa.py                                                                
27920 root      20   0 1921360 214376  35044 R 100.0  0.1   0:08.72 suppa.py                                                                
27983 root      20   0 1921224 213944  34880 R 100.0  0.1   0:08.10 suppa.py                                                                
28064 root      20   0 1899252 191972  34836 R 100.0  0.1   0:08.00 suppa.py                                                                
28482 root      20   0 1861976 154748  34844 R 100.0  0.1   0:04.89 suppa.py                                                                
28586 root      20   0 1856140 148976  34736 R 100.0  0.1   0:04.44 suppa.py                                                                
28587 root      20   0 1846516 139324  34900 R 100.0  0.1   0:04.45 suppa.py                                                                
26096 root      20   0 1950144 242780  34908 R 100.0  0.1   0:11.42 suppa.py                                                                
26442 root      20   0 1936560 229028  34692 R 100.0  0.1   0:11.50 suppa.py                                                                
27000 root      20   0 1931020 223848  34988 R 100.0  0.1   0:10.02 suppa.py                                                                
27108 root      20   0 1933736 226632  34988 R 100.0  0.1   0:09.93 suppa.py                                                                
27321 root      20   0 1923300 216092  34852 R 100.0  0.1   0:08.76 suppa.py                                                                
27565 root      20   0 1923028 215848  34780 R  99.7  0.1   0:09.07 suppa.py                                                                
19245 root       9 -11    4532    840    776 S  21.5  0.0   0:26.33 cpulimit                                                                
17119 root      20   0 50.406g 1.147g  33940 S  11.6  0.5   1:15.74 java          

I also read about using groups but I am not sure how to do that since I cannot readily differentiate these child processes from other root processes.

Is there a dummy-proof solution for limiting the total CPU time of a command + children to a total percentage of CPU?

0

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.