One of my customers has a Windows Server 2016 and uses wbadmin
to do some bare metal recovery backup to some Synology NAS using SMB. This worked in the past, where "past" is some unspecified weeks/months back in time for various reasons, and doesn't work anymore. It's pretty likely "something" has changed because of NAS updates, additionally installed packages and whatever, but of course no one knows anymore and it wasn't me. ;-)
After some testing and logging with different SMB versions, Wireshark, Process Monitor etc. the problem seems to be that for some reason wbadmin
is able to create the necessary raw VHDX files for the volumes to backup on the NAS share, but seems to be unable to initialize those disks and publish the needed file system into those. The backup process always aborts pretty early after the creation of one VHDX file with a fixed size, either of e.g. 4 MB for Esp.vhdx
or 6 MB for the VHDX containing C:
. It simply depends on the backup settings like -allCritical
vs. only -include:C:
.
Whenever this problem occurs, there's the following error message in the event viewer:
Log Name: System
Source: Virtual Disk Service
Date:[...]
Event ID: 9
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer:[...]
Description:
Unexpected provider failure. Restarting the service may fix the problem. Error code: 80070037@02000014
I don't see any other relevant error messages in Wireshark within the SMB communication, the Samba logs on the NAS or Process Monitor, but especially for the latter I'm not that sure, because Windows does a lot of things automatically in the background that are perfectly fine to fail. The NAS provides some Samba protocols which are a bit weird as well, because it logs lots of SMB communication for a user which is not configured to be used by wbadmin
. But as all tested users have the same/all permissions on that backup target, I don't think that is part of the problem and I tested with different users as well.
The interesting thing is that I'm able to attach the automatically created VHDX files by wbadmin
manually successfully, can initialize them, add some file system and afterwards e.g. the backup of only -include:C:
succeeds without any problems. -allCritical
keeps failing because Esp.vhdx
seems to be deleted and recreated each time, running into the same error again like before.
Another interesting thing is that using some iSCSI device hosted by the same NAS as backup target even -allCritical
succeeds each and every time tested. All VHDX files are created, initialized and formatted automatically without manual intervention. So it doesn't seem to be Windows/wbadmin or related to the network itself.
Therefore I'm assuming there is some problem regarding Samba, but I've already tried with the lowest SMB1 version the NAS is capable of, tried things like strict allocate = yes
people are suggesting, even though I made clear that the NAS isn't creating sparse files anyway and I'm able to attach, initialize and publish the VHDX files created by wbadmin
manually. Which shouldn't be possible in case of sparse files from what I've read.
The only thing left for now is the error code from Virtual Disk Service, which I don't find much about. The part 0x37
might be ERROR_DEV_NOT_EXIST
and that might make sense, but what's the @
-part? Where do I find what the whole syntax of that error means? Is the @
-part some line of code or some additional detail or something completely different? I'm hoping this gives me additional clues about the actual underlying problem, because I don't understand why things fail for wbadmin
itself which I'm able to do manually.
Thanks!