How are the CPU-chipset lanes shared on X570 motherboards?

Question

The basic question is: If there are four PCIe lanes between the CPU and chipset on an X570 motherboard, then how can everything connected to the many chipset lanes be on equal footing with the direct CPU connections?

The context: I have a standard, non-Threadripper 3000 Ryzen on an Asus X570-P motherboard, currently with a single PCIe Gen 3.0 x4 NVMe SSD installed, and obviously a graphics card which needs as many lanes as possible. The existing SSD is sitting in the CPU-connected Gen 4.0 x4 M.2 slot. There is a second 4.0 x4 M.2 slot that is connected to the chipset instead of the CPU (according to both the manual and the diagram below).

Now I would like to install a second SSD, either a Gen 3.0 x4 or Gen 4.0 x4. But I am fascinated and profoundly confused by the lane diagrams of the X570 setup, an unofficial analysis of which can be found here: https://hexus.net/tech/features/mainboard/131789-amd-ryzen-3000-supporting-x570-chipset-examined/

My understanding, and my questions: There is evidently only a 4.0x4 connection between the chipset and the CPU, and then a plethora of lanes from the chipset to everything else. So, if I install a second SSD in the chipset-connected slot:

1. It seems obvious the GPU will not be impacted (?)
2. The first SSD should also not be impacted (?)
3. ... but the connection to the new SSD must somehow be slower than the first one, since it is sharing its four lanes with all the other USB connections and whatnot? However, neither the X570-P manual/webpage nor the above webpage mention anything about such a bottleneck. Is the chipset doing some sort of black magic here? Also, conversely, would this second SSD be influenced by peripherals... meaning that e.g. a Gen 4.0 SSD might be rather pointless here?

Answer 1

Yes, you are right, but remember: PCIe gen4 x4 is 8GB/s. It is very, very difficult, to saturate this, even with 2 SSDs. Most SSDs are much slower. Like reading with 3.5GB/s is fast, but that would be 7 of 8GB/s. Still 1GB/s left for USB3 - wich usually is much more than enought. This would saturate at least 5 modern HDDs at the same time. Or almost 2 SATA 3 SSDs.

Don't be affraid of that. PCIe also is much less CPU intensive than SATA - this all contributes to much better real life performance. You will not notice a bottleneck. You could in theory reach it, but it is very, very unrealistic in reality. :D

Have fun with your nice Setup.

Answer 2

The basic question is: If there are four PCIe lanes between the CPU and chipset on an X570 motherboard, then how can everything connected to the many chipset lanes be on equal footing with the direct CPU connections?

The direct answer to that question is : depending on your usage scenario and the overall system build, the devices connected via the PCH (chipset) could be on an "equal footing", or not, to those connected directly to the CPU.

For example, if you are connecting a PCIe x4 NVME drive via the PCH, it will share the x4 connection between the PCH and CPU with all the SATA, USB, ethernet, wan, audio and other devices which are also connected to the PCH.

1. whether or not the NVME drive's performance would be impacted depends on the actual performance of the drive itself, the bandwidth requested by the SATA and USB (as well as other PCH-connected devices) and the overhead incurred inside the X570 PCH itself, i.e. how efficient the chipset is at dealing with, and prioritizing, all those concurrent connections;
2. as of Q1 2021, top consumer-level x4 PCIe 4.0 NVME drives have been shown to hit upwards of 7+ GB/s. For example, the Samsung 980 Pro hits 7.1 GB/s (peak sequential read, 128KB blocks) which represents 90% of the theoretical bandwidth of the x4 PCIe 4.0 connection between the PCH and a PCIe 4.0 CPU;
3. considering point #2 above, that x4 PCIe 4.0 NVME drive might be bottlenecked if, in parallel, there are some Sata 3 SSDs and/or USB 3.2 Gen 2 10 Gbps (which is the max USB port speed on X570) devices transferring data; that, however, depends on the overall build and its different usage scenarios, as well as the particular type of data transfer which is being processed (read vs. write, sequential or not, small vs. big files, etc.);
4. again considering point #2, if only the PCH-side NVME connection is being used for intense data transfer at some given time, then the bottleneck risk would be lower; but then again that depends on the amount of overhead data that is being processed on the potentially idling SATA, USB and other devices (1 Gb/s ethernet, WAN, audio, etc.) at that particular time;
5. numbers will be different for a CPU that is PCIe 3.0 and not PCIe 4.0 to begin with, but the same logic will apply.

As for your other sub-questions:

1. a GPU card will not be impacted, provided it runs on the GPU-dedicated x16 logical lanes via 1, 2 or more physical slot(s) connected straight to the CPU on the motherboard. However, if the card is running on a "secondary" x1, x4, x8 or x16 physical slot connected via the PCH, it would then be capped to x1, x4 or x8 logically at the slot level depending on motherboard implementation, and then overall still logically capped to x4 under the best circumstances via the shared x4 logical connection between the PCH and CPU;
2. you did not define what "first SSD" means; if it means NVME #1 (connected to CPU) then it will not be impacted as long as nothing else is connected to that x4 off the CPU. Because in fact the Ryzen architecture allows that x4 from the CPU to be shared with more devices, either two(2) x2 PCIe M.2, or one(1) x2 PCIe M.2 plus two(2) SATA (either M.2's or traditional ports). That however, depends entirely on the implementation on the particular motherboard model.

For reference, here are the bandwidth calculations for x1, x4, x8, x16 lanes :

PCIe 3 :

x1 = 8 GT/s bandwidth (nominal)

Due to 128b/130b encoding : 8 GT/s * (128b/130b) = 7.8769 Gbps

and 1 GB/s = 8 Gbps, so...

• x1 = 0.985 GB/s = 984.615 MB/s
• x4 = 3.938 GB/s = 3,938.462 MB/s
• x8 = 7.877 GB/s = 7,876.923 MB/s
• x16 = 15.754 GB/s = 15,753.846 MB/s

PCIe 4 :

x1 = 16 GT/s bandwidth (nominal)

Due to 128b/130b encoding : 16GT/s * (128b/130b) = 15.7538 Gbps

and 1 GB/s = 8 Gbps, so...

• x1 = 1.969 GB/s = 3,938.462 MB/s
• x4 = 7.877 GB/s = 7,876.923 MB/s
• x8 = 15.754 GB/s = 15,753.846 MB/s
• x16 = 31.508 GB/s = 31,507.692 MB/s