If a USB device is connected and is mounted in Windows it is recorded in the Windows registry.
You can use USBDeview to see any USB device ever connected to any PC you run it on. It pulls the information from the Windows Registry.
USBDeview is a small utility that lists all USB devices that currently connected to your computer, as well as all USB devices that you previously used.For each USB device, extended information is displayed: Device name/description, device type, serial number (for mass storage devices), the date/time that device was added, VendorID, ProductID, and more.
USBDeview also allows you to uninstall USB devices that you previously used, disconnect USB devices that are currently connected to your computer, as well as to disable and enable USB devices.
You can also use USBDeview on a remote computer, as long as you login to that computer with admin user.
Only way around this is to manually remove all entries from the registry that refer to that specific device, along with other Windows locations mentioned by Journeyman Geek. USBDeview uninstall feature may not remove all traces of the device in the registry.