I have 4k laptop, so DPI scaling is what Windows uses for apps that don't automatically scale.
Rightclicking VirtualBox > Properties > Compatibility > Disable DPI Scaling should do the trick. However, it remains upsampled for guests.
The actual virtualbox management screen looks fine however. (Windows 8 host)