When I try to log in to a remote machine via ssh, I immediately get disconnected with "Write failed: broken pipe". I've tried logging in via mosh instead and the mosh client freezes.

I've tried googling around and the only answers I can come up with are related to timeouts (for example, this Unix stack exchange question). I'm pretty certain it's not a timeout related issue since I made no personal configuration changes since last night (when it was last working) and other users are able to log in just fine.

Running ssh with -vvv (very very verbose logs) revealed nothing helpful.

I already solved my specific issue (see my answer below), but I was wondering what other problems might cause the issue (ssh disconnecting immediately stating only "Write failed: broken pipe")?

I further wonder if sshd/mosh/etc could report on being unable to spawn a session to help diagnose problems?


So, in my specific case, I had left a screen open running gdb attached to a process trying to reproduce an issue a customer was having related to our product spawning too many threads. Well apparently the issue had been reproduced overnight and my user had reached the system's user thread limit.

So sshd could not spawn a new process under my user due to the process and thread limit. Ultimately what we had to do was to log in with a different user and kill my running screen, gdb, and the process being debugged. This freed up process and thread resources allowing me to finally reconnect.

| improve this answer | |

Another solution is detailed here: https://access.redhat.com/solutions/30316

I started with the same problem and during the investigation I had another user try to su as my user. He got the error that lead me to the link above (cannot set user id: Resource temporarily unavailable). I was indeed above the allowed thread limit for my user and increasing that limit allowed me login again.

Basically, run this against your user (this is the current number of threads active for that user):

ps -u testuser -L | wc -l

If the result is bigger than whatever limit is set in /etc/security/limits.d/90-nproc.conf (default is 1024) for that user, than you need to increase it. You will need to do that with another user that has not yet exceeded his limit (or somehow kill some of the threads to be able to login).

To increase it, just add a line similar to the existing one, but specific for your user:

testuser   soft    nproc     2048

I know it's an old question, but this might help people that have this issue.

| improve this answer | |
  • 1
    Please avoid posting link-only answers. Links can become broken in time making this answer useless for other visitors. Please include the actual solution in the answer. – mtak Aug 2 '17 at 13:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.