What is "Above 4G decoding"?

Question

I've been dealing with a problem where if I don't enable "Above 4G decoding" in the BIOS settings, my server system won't boot up with a certain number of storage devices.

Google results give this: "The definition of “Above 4G decoding” is to allow the user to enable or disable memory mapped I/O for a 64-bit PCIe device to 4GB or greater address space, because the primary VGA card should always be mapped below 4GB address." (https://www.asus.com/support/FAQ/1004170/)

But what does this mean, for someone who doesn't know much about computers?

Answer 1

Since there was never an official answer and this is one of the first Google results:

This allows 64-bit PCIe devices to use addresses in the 64-bit address space. Since 32-bit operating systems cannot access the 64-bit address space*, this option is for compatibility reasons. In most cases, it does not need to be enabled. In my experience, most motherboards default to having this setting off. There's generally no harm in enabling it if you have a 64-bit OS.

Generally, people enable it when they have many PCIe cards installed, such as with triple and quad SLI and using GPGPU compute modules such as NVIDIA's Tesla. It's more commonly used in workstations and servers.

Since neither setting should cause issues on a 64-bit operating system, you can enable or disable it without any ill effect most of the time.
______________
* Note that a 32-bit processor, or a processor in 32-bit mode, can access 2³² bytes, which is 2²×2³⁰ = 4 gibibytes (4 GiB).

Answer 2

On x86 PCs, the memory space is shared in the system. System memory maps from the bottom up (0 -> xGB) & PCI memory space is in the upper part of system memory space (xxx --> FFFFFFFF 4GB-1). Note that "memory space" is not "memory". It's just address ranges for HW to decode & claim transactions.

This is usually ok because PCI/PCIe devices generally request a reasonable amount of memory for their PCI BAR (Base Address Range), such as 64K or less. BARs are where things like device-specific registers reside and what a device driver would access to control the device. Very large BARs don't apply to most devices because they use DMA engines to transfer data instead of the host CPU performing PIO accesses (very poor performance), so they don't need to map their internal storage to BARs.

Once you start using GPU type devices, for example, the higher end ones request very large PCI BAR spaces, such as 32GB or more. If your PCI memory space is limited to 32-bit, then there's no available address range to fit 32GB in. So, BIOS PCI enumeration started to support 64-bit PCI BAR assignment during enumeration. This generally started about 12 years ago and some of the first algorithms to support it still had bugs or limitations. Some systems would hang if a BAR was larger than 8GB. PCI enumeration algorithms have improved since then.

So, the "Above 4GB decoding" means that the BIOS PCI enumeration is "allowed" to assign PCI BARs memory ranges above 4GB (32-bit max). It may even do that for small PCI BARs, as long as they report themselves as 64-bit.

Note that in PCI/PCIe devices, PCI BARs are 32-bit. If a PCI BAR wants to support 64-bit, it "combines" 2 32-bit BARs to store the 64-bit address. A device with "64-bit PCI BAR 2" means that its BAR 2 is low 32b of the assigned address & BAR 3 then becomes the upper 32b.

Also note that 32-bit x86 can still access PCI BARs above 4GB due to PAE.

Answer 3

If you're running a 32 bit operating system, you will almost certainly need to turn this OFF, otherwise some of your hardware may not be seen at all by the OS. Since this will map your video memory into the ~3.5Gb available, you should use whatever tools came with it, and make sure it's not using a lot more memory than you need, since that memory won't be available to your operating system and apps. Most can be restricted to 128 or 256 Kb. If you're using a 64 bit OS, with 8Gb or more, you can probably leave it off, unless you're gaming with an expensive video card.

Answer 4

This option allows Tesla cards to map their video memory directly into the memory range of the machine where it's plugged into.

Since a lot of memory on a card won't fit below the 4GB range ( eg: 12/24GB vRAM on Teslas or newer Quadro/GF models ) the BIOS must allow the range declaration of the card above the 32Bit line ( being 4GB )

This is mandatory for Tesla and MultiGPU systems that map more and/or above 4GB.

Answer 5

Apparently Above 4G decoding is necessary to unlock "Smart Access Memory" with AMD's latest NAVI GPU... which allows for upto 16% increase in performance in games.