"Plug and Play" is a Windows-specific term that originally meant the method later ISA and then PCI devices allowed the operating system to choose the resources they consumed. The BIOS could do this as well for DOS, or step aside and let Windows completely do it.
Resources are A) memory addresses, B) I/O addresses, C) IRQs, and/or D) DMA channels. Each, several, or all of these can be used by a device to communicate with the CPU.
So originally a "Plug and Play" driver was a driver that could understand how to initialize the device, i.e. configuring it to take resources that no other device was using.
One other important part of Plug and Play is a standard method for a device to identify itself, PCI (and possibly ISAPNP) makes provisions for each device to have a unique vendor and device ID. This carried over to USB. So the host OS can find out "what" the device is and load the proper driver.
Now, when we get into devices that are designed to be "hot-pluggable", that is slightly different, but evidently the term "Plug and Play" is still used to describe it. Most hot-pluggable devices are "hosted" by a non-hotpluggable device. USB devices are "hosted" by a USB controller, which is itself a PCI device consuming one or more resources usually assigned via Plug and Play. Talking to devices over the USB bus involves going "through" the USB controller. A similar situation exists for SCSI, IDE, and SATA devices.
So for hot-pluggability to work, the driver for the "host" controller needs a mechanism to tell drivers representing devices "above" it what is going on, i.e. a USB controller driver needs a way to tell a USB mass storage driver that the device has been disconnected. The "upper" driver needs to have code that can terminate cleanly when it is asked to be removed, instead of expecting to stay resident until rebooted.
Most bus types such as USB, SCSI, SATA are governed by standards which the drivers must recognize and implement.
On the hardware level, most hot-pluggable devices are designed where part of the connector discharges static electricity, and that the pins that carry power connect a fraction of a second before pins that carry data. Look closely at USB and SATA connector pins; this is why some are longer than the other. IDE connectors do NOT do this and are NOT safe for hotplugging.