How to automatically restart a linux background process if it fails?

Question

I have a process which is executed by init.d script on the background. Eg:

case "$1" in 
    start)
       /bin/myprocess &
    stop)
       killall myprocess
    restart)
       killall myprocess
       /bin/myprocess &
esac

In certain conditions, myprocess can fail and return. Is there any (standard) way how to detect its fail and restart in automatically?

Answer 1

The easiest way would be to add it to /etc/inittab, which is designed to do this sort of thing:

respawn If the process does not exist, start the process. Do not wait for its termination (continue scanning the /etc/inittab file). Restart the process when it dies. If the process exists, do nothing and continue scanning the /etc/inittab file.

For example, you could do this:

# Run my stuff
myprocess:2345:respawn:/bin/myprocess

Answer 2

Buildroot has three possible init systems, so there are three ways to do this:

BusyBox init

With this, one adds an entry to /etc/inittab.

::respawn:/bin/myprocess

Note that BusyBox init has an idiosyncratic /etc/inittab format. The second field is meaningless, and the first field is not an ID but a device basename.

Linux "System V" init

Again, one adds an entry to /etc/inittab.

myprocess:2345:respawn:/bin/myprocess

systemd

One writes a unit file in, say, /etc/systemd/system/myprocess.service:

[Unit]
Description=My Process

[Service]
ExecStart=/bin/myprocess
Restart=always

[Install]
WantedBy=multi-user.target

Enable this to autostart at bootup with:

systemctl enable myprocess.service

Start it manually with:

systemctl start myprocess.service

Further reading

• "3.1.3 init system". The Buildroot user manual.

Answer 3

What about creating a subshell with a loop that calls constantly the same process?

If it ends, the next iteration of the loop goes on and starts it again.

(while true; do 
    /bin/myprocess
done) &

If the subshell dies, it's over though. The only possibility in that case would be to create another process (I'll call it necromancer) that checks whether yourprocess is alive, start it if it isn't and run this necromancer with cron, so that you can check that regularly.

Next step would be wondering what could happen if cron dies, but at some point you should feel safe and stop worrying.

Answer 4

You could make use of Monit . It's really easy to use and quite flexible. See for example this configuration for restarting the Tomcat process on failure.

check process tomcat with pidfile /var/run/tomcat.pid
   start program = "/etc/init.d/tomcat start"
   stop  program = "/etc/init.d/tomcat stop"
   if failed port 8080 type tcp then restart

It also has a lot of configuration examples for many use cases.

Answer 5

You may use restarter

start)
   restarter -c /bin/myprocess &
stop)
   pkill -f myprocess

On newer systems systemd which solves all those trivial issues