2

I am currently building a stress testing tool, and as such I need a pretty massive number of threads. I already went through all settings to raise the limit, but there is one last setting, the upper system-wide limit in /proc/sys/kernel/threads-max, that I can't seem to be able to change.

I tried

sysctl -w kernel.threads-max=200000

Editing manually with nano or echo

echo 200000 > /proc/sys/kernel/threads-max

Editing /etc/sysctl.conf and running

sysctl -f

If I run those as sudo, I have no error displayed (the new value is even displayed), but when checking again, the value hasn't changed either. When trying to edit the value with gedit, it spat an

invalid argument"

whatever the value I try, even the original one. I had no problem changing the pid_max value.

I really have no clue why it refuses my edits, and I haven't been able to find anybody with a similar problem, so I would be very grateful if someone could explain what is happening.

  • The whole problem is precisely that the simple&obvious method does not work... – Anarkopsykotik Jun 16 '17 at 15:08
  • The files in /proc/ are virtual, and can as such not be edited. ( Please try $ du /proc/ to know there isn't anything! ) .... The setting "kernel.threads-max=200000" is elsewhere ... – Knud Larsen Jun 16 '17 at 17:51
  • They can be edited, I have done so for other settings, and also modified that one on other machines (test servers running fedora) – Anarkopsykotik Aug 23 '17 at 12:43
4

The repsonse lies in man proc(5), here is the interesting part:

 /proc/sys/kernel/threads-max (since Linux 2.3.11)
                     This file specifies the system-wide limit on the number
                     of threads (tasks) that can be created on the system.

                     Since Linux 4.1, the value that can be written to
                     threads-max is bounded.  The minimum value that can be
                     written is 20.  The maximum value that can be written
                     is given by the constant FUTEX_TID_MASK (0x3fffffff).
                     If a value outside of this range is written to threads-
                     max, the error EINVAL occurs.

                     The value written is checked against the available RAM
                     pages.  If the thread structures would occupy too much
                     (more than 1/8th) of the available RAM pages, threads-
                     max is reduced accordingly.

I assume your kernel version is > 4.1, so since 200000 (the number you try) is less than 0x3fffffff, the problem looks like the available RAM in not sufficient.

  • Isn't 0x3fffffff more than 2.000.000.000 ? Also, even the original numbers or lower did not work, although it could be indeed related to the limit on available ram. – Anarkopsykotik Aug 23 '17 at 12:41
  • 200,000 is far less than 0x3fffffff, like it is written – pim Aug 23 '17 at 12:44
  • Well google is lying to me or its not hexa then ;p But anyway, I am pretty sure it's related to available ram, as tweaking the number seem to work now (did not work last time, so must depend on actually used RAM) but as soon as I get too high, it revert to a default of 96062 (which must be calculated from my physical ram) – Anarkopsykotik Aug 23 '17 at 12:51
  • 1
    Untested, but Wolfram Alpha says 0x3fffffff = 1073741823, so the following should be: echo kernel.threads-max = 1073741823 >> /etc/sysctl.conf echo 1073741823 > /proc/sys/kernel/threads-max (wolframalpha.com/input/?i=0x3fffffff) – James McGuigan Feb 28 '18 at 5:35
  • James' comment is the only thing that worked for me. – Jonathan Jun 23 '19 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.