The BIOS in a PC was meant to fufill a similar function as the BIOS in an 8-bit CP/M system, popular before the PC took over in the mid-80's. The BIOS was intended to contain a minimal bootloader and hardware-dependent low-level routines to do input and output to a few devices (screen, disk, tape, COM port). Knowledge to do this was built into the ROM - no driver needed, and of course, no additional hardware supported by this ROM. (Things like power management and ACPI came much later, in the 90's, after the PC had established itself as an ubiquitous platform.)
(The CP/M "filesystem" was in a component loaded off disk called the BDOS - likewise, knowledge of the FAT filesystem and it's interfaces is in (at least one of) two hidden files MSDOS.SYS or IO.SYS - not part of the BIOS ROM.)
However ... The PC BIOS, unlike CP/M, did support the notion of "Option ROMS" which could be included on an expansion card. So there was at least a minimal mechanism to extend the BIOS. Video cards starting with CGA (MDA, CGA's predecessor, may have done it too) would have an option ROM that extended or added I/O functions to the BIOS interface. (This is why you see an NVidia message before your BIOS boots.) So did hard controllers and SCSI cards. All of these still do. Many older network cards have a socket for a boot ROM.
Keep in mind also that PC clone manufacturers that arose in the 80's very quickly decided not to provide only a compatible BIOS interface, but ended up having to copy the PC platform as a whole, including all of the low level hardware such as the timer chip, interrupt controller, etc. (This was relatively easy since little of it was IBM proprietary.) This was because the BIOS was slow to do things and programmers accessed the hardware directly, particularly for games.
Thus, between option ROMs and this consensus of standard hardware that forms the PC platform, as well as the fact it's been kept backwards compatible throughout the evolution of the PC, something wishing to use the display without a driver can:
- use standard BIOS interfaces, which may be "hooked" by an option ROM in the video hardware
- or make assumptions about what hardware is in the system and access basic hardware directly
All PC display hardware still works in a "VGA compatible" mode upon boot. The original IBM VGA adapter had modes compatible with earlier EGA, CGA, and MDA cards. All this means is that something running from the BIOS or outside of an OS can assume it can still read and write the same memory connected to the display now as it could in 1985, through convention.