37

I am reading an article bemoaning i7-5820K Will Only Have 28 PCI-Express Lanes compared to its sibling processors having 40 lanes.

Isn't 28 lanes already too many? How many lanes would a normal home PC actually need and for what purposes?

I don't know how the following would be connected to PCIe, but do they even number 28?

2 HDs, 1 SSD, 1 CD-DVD-BR, card reader, printer, wifi or lan but seldom both, joystick, keyboard, mouse, graphics.

What else possibilities would need direct access to PCIe for a home/office PC? Or even a server.

61

Many devices use more than 1 lane.

For example - gaming graphics cards use 16 lanes. Some powerful gaming computers have two graphics cards - that's 32 PCIe lanes (two x16 ports).

Intel i7-5820K can't handle two x16 graphics cards. For some gaming enthusiasts or some engineers, that may be a serious problem. They may have to choose different CPU (maybe some Xeon) if they need more than 4 cores and two x16 graphics cards.

PCIe SSD drives use multiple PCIe lanes too (x4 or x8).

Many gigabit network adapters use PCIe x4, there are also 10-gigabit server adapters and they use PCIe x8.

28 lanes is not that much. If a motherboard manufacturer puts one x16 slot, one x8 slot and one x4 slot (x28 total) - you can use only 3 devices there and... that's all.

Here is an image from Wikipedia PCIe article. I added information about lanes on these PCIe slots.

enter image description here

You can read more in another answer written by reirab.

  • 1
    Two graphics cards probably could help in fast rendering a lot, but I don't know much about rendering animated movies. For Java/Eclipse - it will be more than enough. Eclipse is very slow IDE (I think NetBeans is faster and better for Java), but I'm using Eclipse with Java/Android SDK and on my old laptop (Core 2 Duo T9300, SSD) it works not that bad. – Kamil Nov 21 '14 at 20:42
  • 2
    Storyboard animation and anime is not intensive in rendering. It is intensive in calculating/extrapolating the joints and movement of a character, when the character is made to move. Netbeans is based on Swing/AWT. Eclipse is based on SWT. IBM invented SWT as a way to somewhat allow access to native graphics i/o, which they claim is faster than AWT. – Blessed Geek Nov 21 '14 at 20:57
  • 3
    Single port gigabit networking cards don't need multiple lanes. Even a PCIe 1.0 lane offers 250MBps = 2000Gbps of bandwidth; which is enough to allow for 50% overhead losses while still being able to keep a gigabit port saturated. Multiport gigabit cards can need more than one lane; but if that's what you were referring to you should be more specific since they're generally not something seen outside of a server room. – Dan is Fiddling by Firelight Nov 21 '14 at 21:38
  • 6
    Slight issue with the lane concept: PCI-e lanes are point-to-point. 28 of those lanes can end at the Core i7. However, some expensive motherboards have a PCI-e switch that allows 2 graphics cards to communicate directly bypassing the CPU. This means you don't have 2x16 lanes ending at the CPU. – MSalters Nov 22 '14 at 1:09
  • 2
    Great answer, but I just want to point out that LinusTechTips compared doing PCIe 3.0 SLI with a 28-lane CPU vs. a 40-lane CPU, and the 16x/8x vs. 16x/16x didn't seem to make much of a difference: youtube.com/watch?v=rctaLgK5stA So it can really depend on the hardware / use case whether or not everything getting full lane bandwidth matters or not. – Abe Voelker Aug 9 '15 at 19:34
26

A PCIe 'lane' consists of 2 differential pairs of signals. One differential pair is used for sending and the other is used for receiving, which allows simultaneous bi-directional communication. Each lane is point-to-point. That is, each lane directly attaches a single host to a single device. PCIe switches can, however, be used when a host lane needs to be shared between multiple devices. Per Wikipedia, the bandwidth of a single PCIe lane (in each direction) is as follows:

  • PCIe 1.x: 250 MB/s
  • PCIe 2.x: 500 MB/s
  • PCIe 3.0: 985 MB/s
  • PCIe 4.0: 1969 MB/s
  • PCIe 5.0: 3.9 GB/s

As Kamil said, most PCIe devices use multiple lanes. Some devices, such as NICs, sound cards, and other relatively low-bandwidth devices just use 1 lane. SSDs, RAID controllers, and other medium-bandwidth devices typically use 4 or 8 lanes. Graphics cards and other high-bandwidth devices (FPGAs, for instance) typically use 16 lanes. At system boot, the host and device will negotiate the number of lanes that will be used for a particular connection. Typically, the smaller of either the number of lanes that the card is wired for and the number of lanes that the slot it's installed in is wired for (i.e. the maximum physically possible) will be negotiated, though the number may be less in cases where so many PCIe devices are installed that the host does not have enough lanes to give each of them its maximum. The physical slots are designed such that devices with connectors for a smaller number of physical lanes will fit in and function properly in larger slots (e.g. a PCIe x4 card will fit in a PCIe x16 slot and will negotiate to run with 4 lanes.)

Also, some chipsets use some of the PCIe lanes to attach the Southbridge. This was how the Intel x58 chipset worked (the chipset for the Bloomfield chips, high-end of the first-generation Core i7 processors.) It used 4 lanes to attach the Southbridge, leaving 36 lanes for everything else. This typically was divided up as 2 16-lane links for graphics cards and 4 lanes for any other devices. Boards that supported 3 or 4 graphics cards would have to reduce some or all of the graphics cards to 8 lanes when 3 or 4 graphics boards were installed.

Having 2 graphics cards is very common in gaming systems and many gaming systems actually have 3 or 4 graphics cards. Even in a 2-card setup, at least one card will have to fall back to x8 mode in a system with only 28 lanes available. Additionally, systems that use graphics cards as computational accelerators often have 2-4 graphics cards installed. For these situations, only having 28 lanes is a problem, as that greatly limits the amount of host-to-device (and device-to-host) bandwidth available to each card. CUDA in particular has been gaining widespread popularity over the last several years, especially in the high-performance computing community. The PCIe bus can very easily become the bottleneck in GPGPU (General-Purpose computing on Graphics Processing Units) applications, so having as many lanes per card as possible is highly desirable in GPGPU systems.

  • 1
    Thats nice supplement for my answer. – Kamil Nov 23 '14 at 7:24
0

A PCIe lane is a pair of high speed differential serial connections, one in each location. A link between devices can be and often is made up of multiple lanes for higher data rates. The data rates of individual lanes also varies by generation, roughly speaking one lane of Gen x provides about the same data rate as two lanes of Gen x-1.

On modern Intel systems some PCIe lanes are provided by the CPU directly, while others are provided by the PCH in the chipset. The link from CPU to chipset is similar to PCIe but there are differences in the details.

Motherboard vendors have to decide how to allocate the lanes provided by the CPU and PCH to the onboard devices and slots. They can and often do include signal switches to give the user some options but there is a limit to how much signal switching can be affordably implemented.

Intels "mainstream desktop" platforms currently have 16 lanes from the CPU plus up to 24 (depending on which chipset is selected) from the chipset. However the lanes from the chipset are limited by the total bandwidth available from the CPU to the chipset (roughly equivalent to PCIe 3.0 x4 IIRC).

16 lanes from the CPU and 24 from the chipset more than enough for a normal desktop or small server, you can put your graphics card on the 16 lanes from the CPU and then the lanes from the chipset plus the integrated controllers in the chipset are generally enough for storage, networking etc. Even with two GPUs 8 lanes per GPU is enough most of the time.

However when building a high end system with 3+GPUs (or possiblly two top of the lines GPUs), lots of fast storage and/or very fast network interfaces more lanes are desirable. If you want to give each device it's maximum possible capacity you are looking at 16 lanes per GPU,

So for those with higher end needs Intel has a high end desktop socket, currently LGA2066. This socket also covers single socket workstation/server systems, though it seems officially at least you can't use the workstation/server processors in most desktop boards.

Unfortunately while with previous generations of high end desktop the number of PCIe lanes and ram channels was fixed, with LGA2066 the number varies by the processor you select. A desktop LGA2066 CPU can have 16, 28 or 44 PCIe lanes.

This puts motherboard vendors in a tricky position, they have to decide how they will handle giving the true high end customers the full functionality of their CPU while deciding what to disable or throttle for those with lower-end CPUs. System builders in turn must carefully read the manuals for the motherboards to find out what the limitations are before buying.

Grabbing the manual for one of the cheaper X299 boards https://dlcdnets.asus.com/pub/ASUS/mb/LGA2066/TUF_X299_MARK2/E12906_TUF_X299_MARK2_UM_WEB.pdf shows that the main limitation is the x16 slots, On a 44 lane CPU all three slots are usable with two running in x16 mode and one running in x8 mode. On the other hand on a 28 lane CPU you get one x16 one x8 and one unusable and on a 16 lane CPU you only get one x16 or two x8.

Grabbing the manual for a high end X299 board https://dlcdnets.asus.com/pub/ASUS/mb/LGA2066/ROG_RAMPAGE_VI_EXTREME_OMEGA/E15119_ROG_RAMPAGE_VI_EXTREME_OMEGA_UM_V2_WEB.pdf it seems they have decided not to support the 16 lane parts at all. This board does let you use three GPUs on a 28 lane CPU, but the second m.2 slot and the u.2 connector are only available with 44 lane CPUs

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.