Sort top 10 processes by the number of threads each created in Linux

Question

I have a Linux/Debian server running: postgresql, tomcat. Sometimes, I can't access server by ssh.

When I open ssh connection with my server, I can type in my log-in id(root) and password, but when I type in my password and press enter, there's no any response.

By 'sysinfo' function, I found that when server faces on this situation, there are about 1600 processes(this value is from sysinfo.procs), and load average is almost 300(in usual case, load average is between 0 and 1)

What I'm trying to is, listing TOP 5 process by number of thread it creates. However I cannot find easy method. There are some way I found.

1. Log NLWP value of ps -efL, so I can know which process creates too many threads abnormally.
2. When load average gets bigger and bigger, copy all of /procs/*/status file to log directory, and I can know which process have biggest value of 'Threads' later.

If there's any other suggestion, please let me know. It drives me crazy because this Linux server shouldn't fail service

Answer 1

Try this:

$ ps axo nlwp,pid,cmd | sort -rn | head -10

ps ax  - see every process
    o  - format
         nlwp: number of lightweight processes (threads) in the process