Wow, after a ton of research and failed attempts, I actually solved this!
Well, mostly solved it - it is functional, but not without drawbacks.
Thanks to @JourneymanGeek's answer, and in particular the post he linked to, I was able to follow the lead, and eventually discovered that amongst the many interfaces implemented by the Remote Desktop Client ActiveX, one of them supports a RelativeMouseMode property! That sounds like exactly what I need, it will force the RDP to support relative mouse movements!
However, I did note on this MSDN Forums post that "RelativeMouseMode is not supported in RDP RDSH/RDVH scenarios and should not be used", but I figured to heck with it, it's not a real production environment, and I was fine using a feature that is not supported. It was also poorly documented, but it seemed that I had what I needed - this question on StackOverflow also gave me hope that it was doable.
So, I set to work implementing a simple WinForms app to host the ActiveX control, with the "...Unsafe" interface properties set.
Except that as it turns out, by "unsupported", this time Microsoft meant "it does not work".
Well, if it did, this would be an SO question, sorry to lead you on (but I think its good to have it documented somewhere...)
However, not all was for naught, during all that digging into the RDP protocol, I looked at the RemoteFX USB Redirection - and that looked like a dead-end, since basic input devices (such as mouse, keyboard, printer) are explicitly blocked from the USB Redirection mechanism:
By default, devices in the aforementioned categories are accessible in the remote session by using high-level device redirection methods. These methods of redirection enable optimal performance and backward compatibility of the device in the majority of user scenarios. Consequently, these devices are not offered via RemoteFX USB redirection.
Well, the final answer, as it turns out, is that there is an override mechanism to this block.
Step 1: Enable RemoteFX USB Redirection
There is a nice elaboration here: https://www.itprotoday.com/server-virtualization/q-how-do-i-enable-remotefx-usb-redirection .
Basically, on the client machine, you use either Local Policy or Group Policy to set
RemoteFX USB Device Redirection to
Enable, and allow users (or just admins) the rights. Then
gpupdate /force and reboot.
Step 2: Enable the redirection override for the mouse.
As this MS KB article explains, you can set a registry key to enable a specific device (or class of device) for USB Redirection.
Again on the client, under
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services\Client\UsbSelectDeviceByInterfaces
you can add a string value with the device identifier.
For example, under this key I added the following string value:
Depending on your mouse, you might need a different identifier, so YMMV. But start with that.
Step 3: Set the RDP client to redirect the mouse.
As the article from step 1 stated:
Once it's enabled, access the Local Resources tab, click More under Local devices and resources, and you'll see a new Other supports RemoteFX USB devices setting. You can use this setting to map through many types of USB hardware that you couldn't with vanilla RDP USB redirection, as shown below.
Under that you should see an entry for your mouse; it might be a bit ambiguous such as:
Select your mouse (or other appropriate entry).
That's it! You now have a hardware mouse attached "directly" to your VM, which allows the 3D games to pull out the relative mouse movements directly from the mouse driver (-ish..)
However, as stated, there are some drawbacks.
The mouse is redirected to the RemoteFX'd VM - in other words, the client machine no longer has a mouse, at least while the Remote Desktop session is open.
A possible solution would be to attach a 2nd mouse, and redirect only one of them. (I have yet to try this, but it should work just fine).
For some reason, when redirected the mouse does not directly show a cursor in the remote session. While it still works, it does make it a bit confusing to aim what your mouse is pointing at.
A simple solution is to turn on Mouse Trails, with the length turned down all the way.
This will have the mouse location show up, but it is a bit sluggish.
Overall, while this solution does work well, the mouse is still a bit slow and sluggish.
You can turn up the mouse speed, play with pointer precision (sometimes turning it OFF will actually help with this), and even twiddle with the mouse acceleration registry settings under
HKEY_CURRENT_USER\Control Panel\Mouse (
This will help mitigate the problem, but not really make it go away.
So overall, this can make even 3D games very playable - for casual gaming, unfortunately not great for twitch gaming. (Though if you need that you can just boot metal straight into that VHD, and have both options....)