This question has come up quite a lot (really a lot), but I'm finding the answers to be generally incomplete. The general question is "Why does/doesn't my job get killed when I exit/kill ssh?", and here's what I've found. The first question is: How general is the following information? The following seems to be true for modern Debian linux, but I am missing some bits; and what do others need to know?

  1. All child processes, backgrounded or not of a shell opened over an ssh connection are killed with SIGHUP when the ssh connection is closed only if the huponexit option is set: run shopt huponexit to see if this is true.

  2. If huponexit is true, then you can use nohup or disown to dissociate the process from the shell so it does not get killed when you exit. Or, run things with screen.

  3. If huponexit is false, which is the default on at least some linuxes these days, then backgrounded jobs will not be killed on normal logout.

  4. But even if huponexit is false, then if the ssh connection gets killed, or drops (different than normal logout), then backgrounded processes will still get killed. This can be avoided by disown or nohup as in (2).

  5. There is some distinction between (a) processes whose parent process is the terminal and (b) processes that have stdin, stdout, or stderr connected to the terminal. I don't know what happens to processes that are (a) and not (b), or vice versa.

Final question: How can I avoid behavior (3)? In other words, by default in Debian backgrounded processes run along merrily by themselves after logout but not after the ssh connection is killed. I'd like the same thing to happen to processes regardless of whether the connection was closed normally or killed. Or, is this a bad idea?

Edit: Another, important way to keep jobs of being killed, that works (?) in either case is to run them through screen. But, the question is more about understanding when things get killed and when they don't: sometimes people want the jobs to be killed on logout, for instance.

More threads: - Clarification on signals (sighup), jobs, and the controlling terminal - https://serverfault.com/questions/117152/do-background-processes-get-a-sighup-when-logging-off - Continue SSH background task/jobs when closing SSH - Will a job put in background continue running after an SSH session is closed? - Prevent an already running background process from being stopped after closing SSH client - How can I start a process over SSH such that it will continue to run after I disconnect? - Unable to keep remote job running on OS X - Close SSH connection


Points 1-4 are correct.I know nothing about point 5. As for your final point, a fine application, screen, will allow you to let all processes run to their natural end, regardless of how you terminate your connection. Screen is in the repos.

The man description of screen is not easy to read, but, among other things, it states:

When screen is called, it creates a single window with a shell in it (or the specified command) and then gets out of your way so that you can use the program as you normally would. Then, at any time, you can create new (full-screen) windows with other programs in them (including more shells), kill existing windows, view a list of windows, turn output logging on and off, copy-and-paste text between windows, view the scrollback history, switch between windows in whatever manner you wish, etc. All windows run their programs completely indepen‐ dent of each other. Programs continue to run when their window is currently not visible and even when the whole screen session is detached from the user's terminal. When a program terminates, screen (per default) kills the window that contained it. If this window was in the foreground, the display switches to the previ‐ ous window; if none are left, screen exits.

I have highlighted the most important part: you can detach the window with the command Ctrl+a+d, and then you may kill/logout your session, and the now-detached window will continue to live, with the programs inside still running. When you connect back, for instance by initiating a new ssh session, the command screen -r will resume the screen session which had been detached earlier, with all output to standard error/output clearly visible.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.