14

Say I am using a computer, any typical modern computer more or less. A device driver is responsible for everything I see on my screen, including right as I type this the screen is updated(this is, of course, within the scope of an OS, like Windows).

Let us take this down to the level where the hardware does not care if there's a kernel or not, and just takes instructions.

Basically, how do GPUs "know" exactly how to do everything? Does a driver basically control every minuscule, intrinsic, or hardware circuit-level function possible or needed to compute binary data for display using special instructions that a driver provides?

If so, does the GPU use a "special" language or "assembly" to understand instructions sent to it like a CPU does?

I have too many gaps in my knowledge, and there's this unpleasant "mystery" behind GPUs and accessing them directly via hardware.

For example, a GPU and CPU have to be able to connect somehow, so it is possible to access a GPU via CPU assembly and implementing the correct data/address buses. There's no way a device vendor can make it impossible to access the GPU outside of a driver, because a driver is also compiled code, and the GPU, as a programmable circuit, must work with binary instructions.

I have asked these types of questions and they get deleted violently quick with no answer why on many sites, so what's the big mystery/secret here? It's a piece of hardware on a motherboard that is accessed like any other.

So really ... assuming this is an "ok" question here concerning software and hardware, anyone can directly access a GPU without vendor specific files, because the files are not magical.

So how are GPUs "programmed" from the bare-metal, direct hardware perspective on the hardware-specific level?

6
  • An "OK" question it seems in my opinion (albeit convoluted and maybe requiring a book to answer - which is not a good format, so first see the help center and tour pages to become familiarized with what kind of questions you can ask here). Oct 31, 2013 at 19:25
  • I'm not reading the help center.
    – user268540
    Oct 31, 2013 at 19:33
  • Mind if I ask why not? Oct 31, 2013 at 19:36
  • Because better time can be spent pondering, trying to solve and answer my "OK" question as it stands.
    – user268540
    Oct 31, 2013 at 19:36
  • And as I said, to properly answer this question you would probably need a book... long answers aren't usually good answers. Oct 31, 2013 at 19:39

3 Answers 3

9

Basically, how do GPUs "know" exactly how to do everything? Does a driver basically control every minuscule, intrinsic, or hardware circuit-level function possible or needed to compute binary data for display using special instructions that a driver provides?

No, the device has its own ICs which control all of the hardware-level aspects of the device’s functioning, and they interface with a central processor which coordinates the on-device aspects as well as provide a higher-level programming interface which can be used to access, read from, write to, and control the device.

If so, does the GPU use a "special" language or "assembly" to understand instructions sent to it like a CPU does?

Yes and no. Like most devices, video-cards use standards to simplifying matters. They use standards like VESA to access the regular functions like boot-time video, or CUDA to access GPU-programming functions. In addition, programs can access the non-standard, or device-dependent functions directly using machine code (often C compiled to assembler) in conjunction with the device’s programming documentations (which are available from the adapter’s web site).

I have too many gaps in my knowledge, and there's this unpleasant "mystery" behind GPUs and accessing them directly via hardware.

Drivers access the device directly, which is why they have special access (regular user-level programs are not allowed to access hardware directly in Vista+ for security and stability reasons). The access the device using ports, DMA (Direct Memory Access), and memory-mapping.

For example, a GPU and CPU have to be able to connect somehow, so it is possible to access a GPU via CPU assembly and implementing the correct data/address buses. There's no way a device vendor can make it impossible to access the GPU outside of a driver, because a driver is also compiled code, and the GPU, as a programmable circuit, must work with binary instructions.

A driver can read and write the device with regular CPU instructions by accessing memory-mapped ports and such. As an easy-to-understand example, you can put a character on screen in text mode by writing to “memory” at address B8000. This address was not RAM, but rather, it was mapped to correspond to the video-card’s memory, so writing to it would write to the screen. Likewise, you could write tothe screen by putting the character you want to write in the AL registry, 0x09 in the AH register, and then calling the BIOS interrupt 0x10. More advanced functions are not different; you can read and write the adapter’s memory, call the functions on its chip, and so on using whatever inetrface(s) that the device exposes and documents.

So really ... assuming this is an "ok" question here concerning software and hardware, anyone can directly access a GPU without vendor specific files, because the files are not magical.

Only for features that comply to standards, but everything else (i.e., the more advanced stuff), you’ll need to consult the manufacturer’s programming documents.

So how are GPUs "programmed" from the bare-metal, direct hardware perspective on the hardware-specific level?

Through multiple layers:

  1. Solid-state components like transistors and such
  2. Low-level ASICs
  3. On-board processor(s) and chipsets
  4. Low-level programming interfaces (assembly)
  5. Higher-level programming interfaces (DirectX, OpenGL)
  6. High-level programming language (C++, C#, Python, etc.)
2
  • 1
    note on 'bare metal'; in the case you hear most game dev's talking about, they are complaining that they have to use high level abstractions like DirectX/OpenGL rather than writing code that compiles to the cards own machine langague. see here for info on the assembler langague used by Nvidia's FERMI GPUs code.google.com/p/asfermi note that even CUDA sits atop a high level langague (PTX) that gets compiled into machine code, so the opcoodes you program in CUDA may not be the same as the compiled output. Oct 31, 2013 at 19:59
  • Hey, might you have a good book suggestion fit for a college student that covers the things you spoke of in this post? I'm not interested in ever working in GPU programming, but the gaps in my understanding of how it all works is frustrating.
    – lanza
    Sep 24, 2017 at 8:53
3

I'm not exactly a genius in computer architecture, but I'll try to treat your question by points, at the best of my knowledge.


A device driver is responsible for everything I see on my screen, including right as I type this the screen is updated(this is, of course, within the scope of an OS, like Windows).

It is true, mainly because direct interaction with the GPU is not easy to implement. Hence the development of graphic frameworks like DirectX and OpenGL.

This handy definition from Wikipedia explains this in more detail:

A device driver simplifies programming by acting as translator between a hardware device and the applications or operating systems that use it. Programmers can write the higher-level application code independently of whatever specific hardware the end-user is using.


Basically, how do GPUs "know" exactly how to do everything? Does a driver basically control every minuscule, intrinsic, or hardware circuit-level function possible or needed to compute binary data for display using special instructions that a driver provides?

GPU's are microprocessors. As such, they process. They process data that is fed from the CPU (i.e. position of elements) to a proper display. They "know" how to do everything because they are manufactured that way and a large group of people agreed on the proper ways to send data to and from the GPU.

Think of the display driver as a buffer, or a proxy. Imagine you're a program window and the user moves you up. You ask the GPU to refresh the display. But as the GPU speaks Klingon and you speak Vulcan, the driver receives your command, translates it to the GPU that in turn does what you commanded.

If so, does the GPU use a "special" language or "assembly" to understand instructions sent to it like a CPU does?


From Wikipedia:

ARB assembly language is a low-level shading language, which can be characterized as an assembly language. It was created by the OpenGL Architecture Review Board (ARB) to standardize GPU instructions controlling the hardware graphics pipeline.

There's this example, and remember that individual manufacturers (NVIDIA, ATI) have their own instruction sets.

Also, there exists OpenCL and many other ways to, in a way, directly program instructions to the GPU.

All this should answer your question, to some degree.

0

To augment the other answers, you should understand there are fundamental differences between programming a CPU and a GPU, which concerns openness and stability.

A CPU is controlled by machine code that you directly feed to it. When you compile a C program, you get a sequence of bytes which is machine code. The machine code is governed by an instruction set architecture (ISA), which is almost always publicly documented and long-lived (decades). This is why, for example, you can take a Windows program compiled in the 1990s and run it on modern x86-64 CPUs.

A GPU is controlled by reads and writes to its RAM or I/O address space. But there is no standard set of commands for large classes of GPUs. Manufacturers often keep the exact set of commands for their GPUs a trade secret, forcing you to painstakingly guess and reverse-engineer how to talk to a GPU to make it do useful things. Manufacturers provide a driver for each (GPU, OS, API) pair - for example, a Windows DirectX driver for the Radeon 1234 GPU, a Linux OpenGL driver for the Nvidia 9786 GPU. The driver translates from some stable, open, high-level API (OpenGL, etc.) down to the proprietary (usually secret) low-level commands to be executed by the GPU. If the manufacturer refuses to release documentation and doesn't provide a driver for your desired environment, then you're out of luck.

Note that GPU commands can do all sorts of things, like load code, load data, start/stop execution, define pipelines, change the video output mode, and lots more. The set of commands change from card to card; for example raytracing was not a feature in the past; programmable shaders didn't exist at some point.

I suppose an advantage of GPUs' approach compared to CPUs is that with fewer constraints, the internal architecture across GPUs could be radically altered while relying on the driver to translate appropriately. For example, a GPU with programmable compute units could emulate an older generation card that only had fixed-function units (transform, fill, lighting, etc.). Meanwhile, CPUs have a limit to how much the internal microarchitecture can vary because ultimately they have to consume a stream of instructions and behave (more or less) the same (i.e. same numbers computed, same memory locations read/written).

At the risk of sounding silly, why can't CPUs have drivers? In some sense, they do; the BIOS is responsible for loading microcode into CPUs at boot time to do some last-minute patching. But this is very minor compared to what device drivers normally do. The real reason is that you have to start somewhere. If a CPU requires a non-trivial driver to be useful, what will execute the driver? It can't be the CPU, because it hasn't been initialized yet and you don't know how to talk to it. Then it would be... a second CPU, which would make no sense. Hence, the CPU executes code in a publicly known and stable format, and this allows it to execute more code to control devices that don't have a public or stable set of commands.

Note that this doesn't preclude the CPU from having a hard-wired internal translator. Indeed, modern x86 CPUs translate machine code into private, internal, RISC-like formats that vary from generation to generation; they also have internal pipelining, reordering, branch prediction, speculation, and other features. Some of the internal code morphing can be even more radical, like Via changing x86 code to a VLIW, and Apple chips changing x86 code to ARM.

You must log in to answer this question.