I am doing windows driver debugging and am trying to connect to Windows 10 on a virtual box guest. I have set up a serial port on virtual box on COM1 as a Host Pipe with a name \\.\pipe\windebugpipe and have left Connect to existing pip/socket unchecked.
On the guest, I have edited the dbgsettings w/ bcdedit to use these with a baud rate of 115200 (and have also tried it without having these settings). I can see in the logs that the pipe is actually created, and using Putty I can actually connect and send data.
I have tried on both the GUI and from powershell, both with administrator privileges, to connect with windbg. The command I use is
windbg -b -k com:pipe,port=\\.\pipe\windebugpipe,resets=0,reconnect. Both show no output beyond
Waiting to reconnect and saying that the Debugee is not connected.
Whats interesting is that using Windbg through visual studio DOES seem to work, although I can't seem to get it working through COM ports there either. Just a net connection. Using the net connection does not work with windbg outside either. I've found some articles online saying you can attach yourself to that process, but it seems like that is no longer possible in Windows 10?
I don't know, I'm a bit exasperated. I've exhausted my own ideas.