There are two ways to interpret 'part of the kernel'.
The kernel starts off (on boot) as a file on disk. If you're asking about on disk, they are not part of the kernel, but separate files. Almost all systems now have them as separate files. In really old UNIX days, adding a driver meant adding .o files to the kernel .o files and relinking a kernel.
If you meant 'part of the kernel running image' that depends. Windows can do both. In fact the display driver was moved from being not part of the kernel to being part of the kernel running image some time ago (from nt 3.51 to NT4.0) for performance reasons.
There are two general schools of OS design:
One is called a macrokernel. All OS stuff is handled in a single kernel running image. Linux and most unixes run this way. The advantage is it's fast, all parts of the kernel can read other parts of the kernel and communicate using memory reads and writes. The downside is that this gets messy after a while, and now you need to coordinate usage. If you can do something on purpose, sometimes you can do it on accident. There are no protections, and you may get a kernel panic.
The other is called a microkernel (which Windows NT started out to be, and Windows still sort of is). The theory is that the kernel doesn't do work, but farms out work to special code, but code not running in the kernel memory space. This other code cant touch kernel memory, or anyone else's. The upside, isolation from faults - bad non kernel code can't ruin the kernel. The downside - going back and forth to kernel mode slows things down. This is why the display driver was moved into the kernel in NT 4.0, the slowness.
These are generalizations of course, I haven't followed Windows microkernel design in a while, though I can be relatively sure of Linux design.
MacOSX is actually interesting technically, using a hybrid microkernel/macro-UNIX-kernel design. It used to support old OS9 binaries too - farming out to non-kernel space things that were kernel calls in MacOS9.
DragonFly BSD is an interesting offshoot of FreeBSD that's still a macrokernel, but uses message passing as a kind of poor mans isolation, and makes kernel work easier as a result.