I'll begin by stating that nik's accepted answer is wrong.
Anecdote: I blew up a monitor last year by unplugging it's signal cable while it was running, so I'd say it was still true.
Also: My USB-connected printer (with it's OWN power supply) reboots my PC if plugged in while everything is running. whoops!
Spoiler alert: actual facts/science coming up....
A Ground potential difference between the two devices may occur due to the vagaries of power supply design and where the devices are plugged in. Different outlets are not necessarily on the same ground potential.
So you plug them togheter, current flows places it shouldn't and this may fry electronics at both ends. If it doesn't happen to you you're lucky.
Some devices "float" their grounds, some tie their signal grounds to the chassis ( PC's are very bad at doing this, and it causes exciting issues in electrically noisy environments.)
When you turn everything on and its already plugged together, the grounds are tied together and it all runs smoothly.
(For short distances, and only most of the time, it breaks in electrically noisy environments)
Manufacturers who tie their grounds to mains ground are assuming this will work ( it it works great most of the time....)
Our American friends miss out on a ground connection in their standard power socket, some of the time.
Equipment designed to tolerate hot-plugging while running from separate power supplies must include serious ground surge supression on connection, or isolate the devices. Most hot swappable gear allows things like disk drives to plug into a chassis. Thus there is no issue with separate power supplies, or ground equalization surges. The US military's MIL-HBK (handbook) for electrical equipment design suggest that applying 110VAC to any pin of any connector may happen, so to design for it. This is how mil-spec gear ends up really heavy, but if done right, un-killable.
Some standards are really good in the real world. (USB, parallel printer, most RS232, not included)
Ethernet uses small (tiny, really) transformers to isolate the data. Security cameras installations regularly use video baluns ( transformers again) to do the same thing. Professional analog audio gear (balanced) uses... transformers. For everything else there is fibre-optic connections like TOS-link.
I hope you enjoyed this trip into the fascinating (and expensive) world of blowing up equipment by plugging it together while powered on.