First, lets introduce the memory types. Once we understand them, we can understand booting process more easily.
DRAM (Dynamic Random Access Memory)
DRAM is a volatile (means the contents are gone when the power is off) memory type which stores data for CPU usage. Every data to be used, program to be ran should fetched from it's source to RAM first. In case you are interested, this link contains a huge image with numerous connectors, sockets, DRAM types of computer hardware.
EPROM (Erasable Programmable Read-Only Memory)
A ROM has it's circuitry configured during the manufacturing. There is no way to change the contents of the ROM. EPROMs, on the other hand, can have their internals re-configured. But to do that, the EPROM should be detached from the board. The next step from EPROM is Electrically Erasable Programmable ROM (EEPROM) where you do not have to detach the memory from the system. You can simply reconfigure it electrically. All types of ROMs are non-volatile.
CMOS (Complementary Metal-Oxide Semiconductor)
As seen from the title, it's hard to understand what type of memory is this since it says semiconductor, not memory. Well, there is a history behind the name. In early days of modern PCs the BIOS was stored in a volatile, low power consumption CMOS SRAM (Static RAM). If it's volatile, then when the power is off, the contents of BIOS should be lost. To overcome this issue, manufacturers added a battery for CMOS to the motherboard, as seen below (the huge metallic circle to the right and center).
Modern motherboards have the BIOS installed in non-volatile EEPROMs (Electrically EPROM) and still keep the battery. If the memory is non-volatile, why keep the battery? The answer is to keep track of peripherals and date-time, to power RTC (Real-Time Clock).
BIOS is stored in an EEPROM. When you turn on the computer, after the CPU gets ready for operation, it first goes to the EEPROM's address to find the BIOS program to execute. At the BIOS, the CPU checks for potential hardware malfunction, then starts the GPU. After that, peripheral devices are checked (USB, hard disk, keyboard, etc.). Finally, BIOS visualizes the startup screen. Below is an example of this screen.
Then, the BIOS reads system date and time and the connected hard drives from the CMOS to start the bootup sequence from the correct disk that stores the operating system files. Once the disk is determined, BIOS detaches itself. From that point on, operating system data are fetched from hard disk into DRAM. You know the rest :)
It's a misconception that BIOS resides in CMOS. This is WRONG . BIOS is stored in an EEPROM or a flash memory which are non-volatile. CMOS serves as a memory to the BIOS code in which the BIOS settings are stored.