On 13th May 2020, our Windows server downloaded and began to install two updates: KB4551853 and KB4556441. It restarted to finish the installation at 1:05am on Thursday morning. However, when people arrived to work on Thursday morning (whether physically or virtually) the server was not working; it was unresponsive.
First of all I forced it to reboot by removing the power to it and then starting it again. Once it started up again, however, it exhibited symptoms of having a memory leak. It ran well for an hour or so, then became increasingly unresponsive, until it had to be restarted again.
I tried uninstalling the updates – which was possible for KB4551853. However, this had no effect on the problem. I looked in to going back to a restore point, but the “System Protection” tab is missing. Perhaps we did not install the relevant “Rôle”.
Having noticed an error message about svchost.exe having a buffer overflow, I ran Windows’ memory test, but this saw no issues. Neither did the BIOS RAM test. Exact message:
svchost.exe – System Error.
The system detected an overrun of a stack-based buffer in this application. This overrun could potentially allow a malicious user to gain control of the application.
The Task Manager program indicates that the RAM becomes full (slowly), but that no process is taking up much RAM at all. It has 16Gb of RAM, but the memory reports 99% full while the process taking up the most RAM is only using 17.1Mb! Windows Start Menu and Settings seem the first things affected by the memory leak, the first programs that I would want to use that fail to work.
Not the RAM
I did take the server apart and clean it and reseat the power connectors and RAM. It made no difference. Having researched saving the Active Directory settings, I converted the OS filesystem from FAT32 to NTFS using the Windows Server install disk, and then tried to do a “System state” backup. However, the system became unresponsive before it completed.
I tried to reinstall Windows Server 2019 over itself, while still keeping its settings. However, the reinstall failed with error 0x800706BA – 0x20003, in the SAFE_OS phase with an error during INSTALL_UPDATES operation. Suggested fixes included making sure the Windows update system is functional, so I ran the Windows Update Troubleshooter, and that found no errors. They also include uninstalling non-Microsoft antivirus software, so I uninstalled our copy of ESET from the server.
Other Attempted Fixes
I have also run
DISM /Online /Cleanup-image /Restorehealth followed by
sfc /scannow, and
chkdsk c:, but they have all found no errors and made no difference.
Possible actions going forward
- Try disabling all the Startup programs to see if we can isolate what is causing the problem.
- I think Dell Support Assist recently updated; perhaps try uninstalling this software.
- Try to reinstall Windows Server again, having uninstalled ESET it might work.