A C# service that I have authored will occasionally not automatically start on VMs that are booting up -- we see the "A timeout was reached while waiting for the service to connect" entry in the Event Log.
The underlying issue is the same as what is described in this similar SO question -- but the difference here is that I have Recovery options set on the service to Restart the service on all failures. This recovery process is not being triggered, however -- when this happens, the service must still be started manually after bootup is complete.
Does the Recovery option only get triggered on a hard service crash, and not on a startup timeout?
I use log4net and see zero log entries when this happens, so I'm not confident that any of my authored code is being executed before the timeout is hit (therefore solutions to use ServiceBase.RequestAdditionalTime or WINAPI's SetServiceStatus would have no effect).
Assuming this is some underlying issue with the .NET framework or similar being very slow to start (i.e. physical system resource or I/O contention as many VMs start up simultaneously), will I need to author a lightweight C++ watchdog-type service to detect this issue and try to restart the service after the fact?
More details: This service was on a VM that rebooted twice to install Windows updates. The service started up fine after the first reboot -- based on log statements, there was ~1 sec between the service constructor running and the beginning of the OnStart handler, and the service finished starting up within 2 more seconds. On the second reboot, there were no log entries at all, just the Event Log "A timeout was reached" error, and upon inspection the service was not running. A separate but similar .NET service that started up successfully after both reboots did take much longer the second time, though -- again based on log statements, ~0.7 sec after the first reboot, and 6 seconds after the second.
(asked originally on StackOverflow, was recommended I ask here)