I’ve been through a few basic operating systems classes, video series, etc., and have this question regarding the OS drivers vs BIOS drivers. How does a basic OS know the memory address of each piece of hardware? How does an OS work out-of-the box on any computer. Is there a standard device memory layout that everyone knows about? Or would the OS re-direct its IVT to point to BIOS device drivers? Please point me towards some reading material for this topic.
How does a basic OS know the memory address of each piece of hardware? How does an OS work out-of-the box on any computer.
CPU knows only first address to execute. BIOS/Firmware should "answer" to this address for CPU to bootstrap.
And for memory and I/O (I hope you know that there are 2 spaces in x86: Memory and I/O, right?) there were a lot of standards.
First PCs era
There was semi-official area in memory called BIOS DATA AREA. It had some information about number of COM and LPT ports, floppy drive controllers, their base I/O addresses and so on. OS also used BIOS calls (documented interrups) to access hardware. Motherboard vendor hardcoded that addresses to BIOS. Some devices had officially documented I/O addresses.
Extension cards like ISA did not have any tools to be detected except extension card BIOSes: system bios scanned memory for special patterns to find extension-card BIOS and run it. Card BIOS may install itself as interrupt verctor to "shadow" BIOS functions to enable network boot etc.
It many cases it was user responsibility to configure card with jumpers and provide information to software (like
BLASTER env variable for sound blaster params etc).
PnP is a pack of specifications about hardware detection. There were ISA PNP, COM PORT PNP, LPT PNP etc. Specs describe what software should do to check which device is connected and how device should act. Some buses like PCI have PNP ability built-in. Software may enumerate all cards on PCI and ask them about resources they need, their vendor and class (to find driver) etc, and then cofigure them. Enumeration is also possible for USB.
"Software" in this case may be PNPBIOS or PNP enabled OS. So, BIOS used that specifications to find boot device and PCI Host bridge ("root" of PCI bus) and stored this info in special tables in memory (according to PNPBIOS spec) and OS read them and used PCI and USB capabilites to enumerate and enable devices (PCI and USB protocols are documented).
ACPI tables are filled by motherboard vendor and stored in firmware (modern firmware is not" BIOS, but it can emulate it).
Special table called DSDT describes any system device including special laptop buttons, laptop battery, fans, PCI-Express root complex and so on.
Devices are described in language called AML and OS should have its interpreter. DSDT may describe memory, IO and "methods" (also written on AML) to do some stuff like "change brightness".
ACPI is huge spec. OS uses DSDT to fill its internal "device tree" and, once it obtains access to "PCI-Express root complex" or root USB, it uses PCI-Express and USB protocols to do further enumeration and configuration: card may report it supports memory range A and B, and OS configures it to use "B".
Most operating systems are composed of various components such as memory manager, input output, file system manager, etc. Years ago with monolithic operating systems everything was all built together. Most modern operating systems have some way of dynamically changing the various components in and out. Some operating systems, micro kernels, have just a basic management functionality for the operating system with most of the additional components such as file system being add on parts.
Modern operating systems are also built in a series of layers somewhat like the OSI model for network stacks (see https://en.wikipedia.org/wiki/OSI_model) in order to provide as much independence from the actual hardware on which they are running as possible.
So the operating system is basically an abstract machine into which is plugged concrete, hardware specific modules. There are interface specifications, that tend to be somewhat of an abstract machine description, that describe the control and the data and the services the hardware specific modules are to provide to the operating system.
So an installer for a particular device driver stack knows about its specific, concrete hardware needs and interfaces to a device but the operating system does not know these specific concrete hardware needs. The operating system delegates all the gnarly device specific stuff to the device driver stack.
I use the word "stack" because the functionality that performs the actual device communication is normally made up of a series of layers. You can see this with Windows Device Manager with most devices which show a set of driver files that are being used for a particular device.
So there are various software components that may actually come from different vendors.
So if the operating system wants to know details about the hardware, it asks the device driver for those details.
An OS works out of the box on a variety of hardware platforms because the vendors of the specific hardware that makes up the platform provide the driver stack the operating system needs to work on the concrete and specific hardware.
Different operating systems will have different interface specifications for the device drivers the operating system is designed to use. So for Linux, you must have Linux operating system compatible drivers that provide the Linux driver interface. For Windows you have to have Windows interface drivers.
Since there is a driver stack, it may be that some of the software components in the stack are the same between different operating systems so long as the same hardware is being used. The parts of the layer that directly talk to the hardware device may be the same but the upper levels of the stack, the ones that must interface into the actual operating system will be different since different operating systems have different device driver interfaces.
There are tons of books available about operating systems and there are tons of open source projects for various operating systems.
Take a look at this discussion https://www.quora.com/What-is-the-best-book-on-operating-systems
There are a number of free online books such as http://pages.cs.wisc.edu/~remzi/OSTEP/
And take a look at this github page about open source operating systems https://github.com/showcases/open-source-operating-systems
On a modern PC, each piece of hardware reports some basic information about itself when asked. For example, many peripherals are connected to the PCI bus. When the OS wants to know what peripherals are connected to the PCI bus, it sends an instruction on the bus asking every peripheral to send a description. This is called enumerating (the devices on) the bus. This description uses a numerical code to identify each device model called a PCI ID. When the OS sees that a device with a certain PCI ID is present, it tries to load a driver for that PCI ID; if the OS doesn't have a driver it can try to download one, or report to the user that an unknown device is present.
For example, on Linux, you can run the command
lspci to list PCI devices. It has various options to display different kinds of information, play around a bit if you're curious. I describe the mechanism that Linux uses to load a driver for a PCI device in this answer. That's just one example; other PC operating systems have a mechanism to do the same thing but that mechanism may be completely different.
How does the OS know that there's a PCI bus at all, and how to access the bus? I think that's hard-coded if you build it for a PC architecture. Note that “PC architecture” is more specific than “x86 CPU” — you could put a different set of buses around an x86 CPU, but then it wouldn't be called a (modern) PC.
Different machine architectures may or may not have ways to enumerate drivers that are present. For example, PC up to the mid-1990s didn't have such methods. Before the PCI bus, there was the ISA bus. On an ISA bus, if you want to know whether a device is present at a certain address, you send a command to it. If you get an answer that you understand, good. If the command locks up the computer, or causes an unintended effect because there's a completely different peripheral, tough. On a 1908s PC, the OS did ask the BIOS about the devices that were configured (but the BIOS didn't list all devices, only device types that the BIOS knew about), and the user who installed an extension card needed to provide its configuration details either in the BIOS or in the OS configuration. The BIOS included some basic drivers for the keyboard and screen, and operating systems such as DOS called those drivers; other OSes (or applications) might access the hardware directly (especially for display, as the BIOS drivers were slow and limited).
Even today, most embedded systems don't have a way to enumerate built-in peripherals. Many devices (especially ARM-based, but not only those) describe their peripherals in a data structure called a device tree. This data structure is stored in ROM or flash memory and the bootloader (the equivalent of the BIOS) communicates the data to the operating system. This way the OS doesn't have to be built for a specific set of devices, it reads this information when it starts.