4

I have 2 questions related to PCI bus:

How device drivers access devices on PCI bus? Is it enough to write to memory region allocated for device (mov memory_space_allocated_for_dev, something, if device is memory mapped, or out io_space_allocated for_dev, something if it is not) (will hardware translate this attempt to access memory location to series of PCI commands on PCI bus etc. ?).

Is it correct that after setting up all devices on all PCI buses (ie. allocating memory space, enumerating buses etc.) device drivers don't have to know whether PCI bus exists or memory decoding is done with simple decoder (like in old computers) (ie. they simply write and read from some memory locations to access device)?

2 Answers 2

8

Memory-mapped PCI(e) devices will have BARs (base address registers) that let the host know how much memory should be allocated for the device. The BIOS (and the OS later) will allocate the requested memory space to the target device -- not that it is a memory address, physical bits are not being allocated. Additionally, this will generally be a physical memory address, not a virtual memory address. The Linux kernel will arbitrate access to these devices with functions such as mmap() that allow the mapping of physical memory to virtual memory addresses.

For example, say you have a PCIe device that controls 8 LEDs. As the device creator, I can request a very tiny BAR of 1K of address space. The BIOS may give me physical address on a 32-bit system of 0xE000_0000 to 0xE000_0400 (side note: you should see here now why 32-bit systems and GPUs with large VRAM did not work well together).

Now, if I write say 0xF0 to memory location 0xE000_0000 with a one-byte write, that will turn on LEDs 7 through 4, and leave 3 through 0 off. That's it -- it is simple memory-mapped I/O. Adding on the layers of abstraction, on Linux, you would utilize its excellent PCI subsystem and write a driver that gets loaded for a given Vendor ID + Device ID string. You could then write a userspace app that calls into that kernel driver, where a mmap() call can map the memory into userspace, allowing your userspace application to do byte-reads/writes on some virtual memory address that will get translated and end up in physical memory space where it belongs.

x86 of course has I/O space, and the behavior is pretty similar, except low-level in the kernel, the outb/outw/outl (and their input cousins) instructions will be used to write/read from I/O space, vs. memory read/write instructions. Again, a userspace app should be communicating through ioctls() and a mapped memory segment as the kernel is responsible for security / access to memory like that.

For your second question, kind of blended into the above, but a modern PCIe driver for Linux will lean on the PCI subsystem for a lot of the low-level house keeping stuff. You mostly define what Vendor/Device IDs you are responsible for, and then write a .probe() function that snags your IRQs and does device setup -- your memory is handed to you.

1

It's been a while, but if I recall correctly, your kernel driver will set up a device in /dev which maps to the memory addresses allocated to the PCI devices either by the BIOS or kernel. User space programs with the appropriate permissions will then open, read/write/seek, and close that /dev/ file to interact with the PCI device. See chapter 12 of Linux Device Drivers: https://lwn.net/Kernel/LDD3/

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .