114

In 2014, I hear a lot of programming languages touted for their concurrency features. Concurrency is said to be crucial for performance gains.

In making this statement, many people point back to a 2005 article called The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software. The basic argument is that it's getting harder to increase the clock speed of processors, but we can still put more cores on a chip, and that to get performance gains, software will need to be written to take advantage of multiple cores.

Some key quotes:

We're used to seeing 500MHz CPUs give way to 1GHz CPUs give way to 2GHz CPUs, and so on. Today we're in the 3GHz range on mainstream computers.

The key question is: When will it end? After all, Moore's Law predicts exponential growth, and clearly exponential growth can't continue forever before we reach hard physical limits; light isn't getting any faster. The growth must eventually slow down and even end.

... It has become harder and harder to exploit higher clock speeds due to not just one but several physical issues, notably heat (too much of it and too hard to dissipate), power consumption (too high), and current leakage problems.

... Intel's and most processor vendors' future lies elsewhere as chip companies aggressively pursue the same new multicore directions.

...Multicore is about running two or more actual CPUs on one chip.

This article's predictions seem to have held up, but I don't understand why. I have only very vague ideas about how hardware works.

My oversimplified view is "it's getting harder to pack more processing power into the same space" (because of issues with heat, power consumption, etc). I would expect the conclusion to be "therefore, we'll have to have bigger computers or run our programs on multiple computers." (And indeed, distributed cloud computing is a thing we're hearing more about.)

But part of the solution seems to be multi-core architectures. Unless computers grow in size (which they haven't), this just seems to be another way of saying "pack more pocessing power into the same space".

Why doesn't "add more cores" face the same physical limitations as "make the CPU faster"?

Please explain in the simplest terms you can. :)

18

13 Answers 13

145

Summary

  • Economics. It's cheaper and easier to design a CPU that has more cores than a higher clock speed, because:

  • Significant increase in power usage. CPU power consumption increases rapidly as you increase the clock speed - you can double the number of cores operating at a lower speed in the thermal space it takes to increase the clock speed by 25%. Quadruple for 50%.

  • There's other ways to increase sequential processing speed, and CPU manufacturers make good use of those.


I'm going to be drawing heavily on the excellent answers at this question on one of our sister SE sites. So go upvote them!


Clock speed limitations

There are a few known physical limitations to clock speed:

  • Transmission time

    The time it takes for an electrical signal to traverse a circuit is limited by the speed of light. This is a hard limit, and there is no known way around it1. At gigahertz-clocks, we are approaching this limit.

    However, we are not there yet. 1 GHz means one nanosecond per clock tick. In that time, light can travel 30cm. At 10 GHz, light can travel 3cm. A single CPU core is about 5mm wide, so we will run into these issues somewhere past 10 GHz.2

  • Switching delay

    It's not enough to merely consider the time it takes for a signal to travel from one end to another. We also need to consider the time it takes for a logic gate within the CPU to switch from one state to another! As we increase clock speed, this can become an issue.

    Unfortunately, I'm not sure about the specifics, and cannot provide any numbers.

    Apparently, pumping more power into it can speed up switching, but this leads to both power consumption and heat dissipation issues. Also, more power means you need bulkier conduits capable of handling it without damage.

  • Heat dissipation/power consumption

    This is the big one. Quoting from fuzzyhair2's answer:

    Recent processors are manufactured using CMOS technology. Every time there is a clock cycle, power is dissipated. Therefore, higher processor speeds means more heat dissipation.

    There's some lovely measurements at this AnandTech forum thread, and they even derived a formula for the power consumption (which goes hand in hand with heat generated):

    Formula
    Credit to Idontcare

    We can visualise this in the following graph:

    Graph
    Credit to Idontcare

    As you can see, power consumption (and heat generated) rises extremely rapidly as the clock speed is increased past a certain point. This makes it impractical to boundlessly increase clock speed.

    The reason for the rapid increase in power usage is probably related to the switching delay - it's not enough to simply increase power proportional to the clock rate; the voltage must also be increased to maintain stability at higher clocks. This may not be completely correct; feel free to point out corrections in a comment, or make an edit to this answer.


More cores?

So why more cores? Well, I can't answer that definitively. You'd have to ask the folks at Intel and AMD. But you can see above that, with modern CPUs, at some point it becomes impractical to increase clock speed.

Yes, multicore also increases power required, and heat dissipation. But it neatly avoids the transmission time and switching delay issues. And, as you can see from the graph, you can easily double the number of cores in a modern CPU with the same thermal overhead as a 25% increase in clock speed.

Some people have done it - the current overclocking world record is just shy of 9 GHz. But it is a significant engineering challenge to do so while keeping power consumption within acceptable bounds. The designers at some point decided that adding more cores to perform more work in parallel would provide a more effective boost to performance in most cases.

That's where the economics come in - it was likely cheaper (less design time, less complicated to manufacture) to go the multicore route. And it's easy to market - who doesn't love the brand new octa-core chip? (Of course, we know that multicore is pretty useless when the software doesn't make use of it...)

There is a downside to multicore: you need more physical space to put the extra core. However, CPU process sizes constantly shrink a lot, so there's plenty of space to put two copies of a previous design - the real tradeoff is not being able to create larger, more-complex, single cores. Then again, increasing core complexity is a bad thing from a design standpoint - more complexity = more mistakes/bugs and manufacturing errors. We seem to have found a happy medium with efficient cores that are simple enough to not take too much space.

We've already hit a limit with the number of cores we can fit on a single die at current process sizes. We might hit a limit of how far we can shrink things soon. So, what's next? Do we need more? That's difficult to answer, unfortunately. Anyone here a clairvoyant?


Other ways to improve performance

So, we can't increase the clock speed. And more cores have an additional disadvantage - namely, they only help when the software running on them can make use of them.

So, what else can we do? How are modern CPUs so much faster than older ones at the same clock speed?

Clock speed is really only a very rough approximation of the internal workings of a CPU. Not all components of a CPU work at that speed - some might operate once every two ticks, etc..

What's more significant is the number of instructions you can execute per unit of time. This is a far better measure of just how much a single CPU core can accomplish. Some instructions; some will take one clock cycle, some will take three. Division, for example, is considerably slower than addition.

So, we could make a CPU perform better by increasing the number of instructions it can execute per second. How? Well, you could make an instruction more efficient - maybe division now takes only two cycles. Then there's instruction pipelining. By breaking each instruction into multiple stages, it's possible to execute instructions "in parallel" - but each instruction still has a well-defined, sequential, order respective to the instructions before and after it, so it doesn't require software support like multicore does.

There is another way: more specialised instructions. We've seen things like SSE, which provide instructions to process large amounts of data at one time. There are new instruction sets constantly being introduced with similar goals. These, again, require software support and increase complexity of the hardware, but they provide a nice performance boost. Recently, there was AES-NI, which provides hardware-accelerated AES encryption and decryption, far faster than a bunch of arithmetic implemented in software.


1 Not without getting quite deep into theoretical quantum physics, anyway.

2 It might actually be lower, since electrical field propagation isn't quite as fast as the speed of light in a vacuum. Also, that's just for straight-line distance - it's likely that there's at least one path that's considerably longer than a straight line.

14
  • 23
    Also, in many applications the bottleneck is not the computation time, but the stall time to fetch data from the RAM (or, god forbid it, from disk); so, another major speedup comes from bigger, faster processor caches. Aug 15, 2014 at 16:59
  • 2
    @MatteoItalia Yup. And there's also branch prediction improvements, and probably far more I don't know about. Outside the processor, we also have faster buses, faster memory, faster disks and associated protocols, etc..
    – Bob
    Aug 15, 2014 at 17:01
  • 2
    You mentioned that issues related to the "hard limit" of the speed of light will occur at "somewhere past 20 GHz". Your calculations aren't correct; electrical signals travel at speeds lower than the speed of light, depending on the geometry of the wire. Aug 15, 2014 at 17:45
  • 5
    Please don't use "exponential" when there's entirely more correct words for this relationship (quadratic, cubic, etc.)... Aug 15, 2014 at 21:54
  • 7
    @OliCharlesworth Please read the footnote. This is precisely why the footnote is there, and why I have references to it everywhere exponential is used. This is a perfectly valid use of the word, and it would be tangential to the point of this answer to get bogged down in mathematical details. If you really want to try to "correct" it, feel free to suggest an edit. It won't be up to me if it gets accepted or not, as long as you don't significantly change the meaning.
    – Bob
    Aug 16, 2014 at 2:03
15

Physics is physics. We can't keep packing more transistors into ever smaller spaces forever. At some point it gets so small that you deal with weird quantum crap. At some point we can't pack twice as many transistors in a year as we used to (which is what moore's law is about).

Raw clockspeeds mean nothing. My old Pentium M was about half the clock speed of a contemporary desktop CPU (and yet in many respects faster) – and modern systems are barely approaching the speeds of systems 10 years ago (and are clearly faster). Basically 'just' bumping up the clockspeed does not give real performance gains in many cases. It may help in some singlethreaded operations, but you're better off spending the design budget on better efficiency in terms of everything else.

Multiple cores let you do two or more things at once, so you don't need to wait for one thing to finish for the next one. On the shorter term, you can simply pop two existing cores into the same package(for example with the Pentium Ds, and their MCM, which was a transitional design) and you have a system that's twice as fast. Most modern implementations do share things like a memory controller of course.

You can also build smarter in different ways. ARM does Big-Little – having 4 'weak' low power cores working alongside 4 more powerful cores so you have the best of both worlds. Intel lets you down throttle (for better power efficency) or overclock specific cores (for better single thread performance). I remember AMD does something with modules.

You can also move things like memory controllers (so you have lower latency) and IO related functions (the modern CPU has no north bridge) as well as video (which is more important with laptops and AIW design). It makes more sense to do these things than 'just' keep ramping up the clockspeed.

At some point 'more' cores may not work – though GPUs have hundreds of cores.

Multicores as such lets computers work smarter in all these ways.

7
  • 1
    Should be noted, that GPU cores are designed for a very specific purpose, as opposed to CPU cores which are more of a general purpose thing. Also should be noted, that video card boards are significantly larger than CPU's are (meaning even if the cores aren't using all of the room on the board, they still have MUCH more room to dissipate the heat). Aug 15, 2014 at 14:41
  • 3
    true, but that's an example where you do scale up cores by an extreme amount. I'll likely revisit this answer in the morning
    – Journeyman Geek
    Aug 15, 2014 at 14:44
  • "you can simply pop two existing cores into the same package". But how they achieve it without using more space for the extra cores How?! Magic? Unicorns? Puppies? (By your avatar, I suspect the latter) Aug 15, 2014 at 15:31
  • Pentium Ds were that en.wikipedia.org/wiki/Pentium_D basically
    – Journeyman Geek
    Aug 15, 2014 at 15:33
  • 6
    wierd quantum crap + 1 for that alone!
    – Dave
    Aug 16, 2014 at 7:07
9

Simple answer

The simplest answer to the question

Why doesn't "add more cores" face the same physical limitations as "make the CPU faster"?

is actually found within another part of your question:

I would expect the conclusion to be "therefore, we'll have to have bigger computers or run our programs on multiple computers."

In essence, multiple cores is like having multiple "computers" on the same device.

Complex answer

A "core" is the part of the computer that actually processes instructions (adding, multiplying, "and"ing, etc). A core can only execute a single instruction at one time. If you want your computer to be "more powerful" there are two basic things you can do:

  1. Increase throughput (increase clock rate, decrease physical size, etc)
  2. Use more cores in the same computer

The physical limitations to #1 are primarily the need to dump heat caused by the processing and the speed of an electron in the circuit. Once you split off some of those transistors to a separate core, you alleviate the heat issue to a large degree.

There's an important limitation to #2: you have to be able to split your problem up into multiple independent problems, and then combine the answer. On a modern personal computer, this isn't really a problem, as there are loads of independent problems all vying for computational time with the core(s) anyway. But when doing intensive computational problems, multiple cores only really help if the problem is amenable to concurrency.

3
  • 'multiple cores is like having multiple "computers" on the same device.' Right, but my confusion was, how do you fit them all in there? I thought "we can't go faster" was a symptom of "we can't shrink things much more." Aug 15, 2014 at 18:27
  • Multiple cores DO take up more space, chips are getting bigger. The burden has been shifted from the CPU to the software engineer... to actually run faster on one of these giant multi-core cpus the software has to be written such that you can cut its work in half and do both halves independently.
    – James
    Aug 16, 2014 at 4:57
  • 1
    A short answer is that power consumption is proportional to the cube of clock speed. With signals travelling longer distances, the single thread illusion gets harder to maintain. The detailed answer: amazon.com/…
    – Rob
    Aug 18, 2014 at 14:49
6

Why doesn't "add more cores" face the same physical limitations as "make the CPU faster"?

They do face the same physical limitations, but switching to multicore design gives us some breathing space before we hit some of them. At the same time other problems caused by those limitations arise, but they are easier to overcome.

Fact 1: Power consumption and emitted heat grow faster than computational power. Pushing a CPU from 1 GHz to 2 GHZ will push power consumption from 20 W to 80 W, same with dissipated heat. (I just made up these numbers, but it's quite how it works)

Fact 2: Buying second CPU and running both at 1 GHz would double your computational power. Two CPUs running at 1 GHz could process same amount of data as one 2 GHz CPU, but each one would consume only 20 W of energy, that's 40 W in total.

Profit: Doubling CPU number instead of clock frequency saves us some energy and we're not as close to the "frequency barrier" as before.

Problem: You have to split the work between two CPUs and combine results later.

If you can solve this problem in acceptable time and using less energy than you have just saved, then you've just profited thanks to using multiple CPUs.

Now you just have to merge two CPUs into one dual-core CPU and you're home. This is beneficial because cores can share some parts of CPU, for example cache (related answer).

2
  • We keep hitting physical limitations in 1000 different ways, size of the die (for multi-cores which leads to smaller manufacturing process), size of the mobo (for multi cpu), amps drawn thru the traces (both). its not like you could put 2 cpus on an m-atx or 4 and memory on an atx board and those designs take DECADES to change. I agree with the rest.
    – Rostol
    Aug 16, 2014 at 3:04
  • @Rostol That's a good point, I have edited the answer.
    – gronostaj
    Aug 16, 2014 at 10:46
5

Long story short: Speeding up single cores has reached its limits, so we keep shrinking them and adding more of them, until this reaches its limits or we can change to better materials (or achieve a fundamental breakthrough that overthrows the established tech, something like home-sized, actually working, quantum computing).

I think this problem is multi-dimensional and it will take some writing to paint the more complete picture:

  1. Physical limitations (imposed by actual physics): Like speed of light, quantum mechanics, all that.
  2. Manufacturing problems: How do we manufacture ever smaller structures with the needed precision? Raw material related problems, materials used to build circuits usw., durability.
  3. Architectural problems: Heat, inference, power consumption etc.
  4. Economical problems: What's the cheapest way to get more performance to the user?
  5. Usecases and user perception of performance.

There may be many more. A multipurpose CPU is trying to find a solution to scramble all these factors (and more) into one, mass-producible chip that fits 93% of the subjects on the market. As you see, the last point is the most crucial one, customer perception, which is directly derived from the way the customer uses the CPU.

Ask yourself what is your usual application? Maybe: 25 Firefox tabs, each playing some ads in the background, while you are listening to music, all while waiting for your build job you started some 2 hours ago to finish. That is a lot of work to be done, and still you want a smooth experience. But your CPU can handle ONE task at the time! On single thing. So what you do is, you split things up and make a looong queue and every one gets his own share and all are happy. Except for you because all the things become laggy and not smooth at all.

So you speed your CPU up, in order to do more operations in the same amount of time. But as you said: heat and power consumption. And that's where we come to the raw material part. Silicon becomes more conductive as it gets hotter, meaning more current flows through the material as you heat it up. Transistors have a higher power consumption as you switch them faster. Also high frequencies make crosstalk between short wires worse. So you see, the speed things up approach will lead to a "meltdown". As long as we do not have better raw materials than silicon or much better transistors, we are stuck where we are with single core speed.

This gets us back to where we started. Getting stuff done, in parallel. Let's add another core. Now we can actually do two things at one time. So let's cool things down a bit and just write software that can split its work over two, less powerful but more functional cores. This approach has two main problems (besides that it needs time for the software world to adapt to it): 1. Make the chip larger, or make individual core smaller. 2. Some tasks simply cannot be split into two parts that run simultaneously. Keep adding cores as long as you can shrink them, or make the chip larger and keep the heat problem at bay. Oh and let's not forget the customer. If we change our usecases, the industries have to adapt. See all the shiny "new" things the mobile sector has come up with. That is why the mobile sector is considered so crucial and everyone wants to get their hands on it.

Yes, this strategy WILL reach its limitations! And Intel knows this, that's why they say the future lies somewhere else. But they will keep doing it as long as it is cheap and effective and doable.

Last but not least: physics. Quantum mechanics will limit chip shrinking. Speed of light is not a limit yet, since electrons cannot travel at the speed of light in silicon, actually it's much slower than that. Also, it is the impulse speed that puts the hard cap on the speed offered by a material. Just as sound travels faster in water than in air, electric impulses travel faster in, for example, graphene than in silicon. This leads back to raw materials. Graphene is great as far as its electrical properties go. It would make a much better material to build CPUs of, unfortunately it is very hard to produce in large quantity.

2

Say (as an unrealistic example, but should still get the point across) you have a CPU that's running at 100F. How multicore usually works is by taking the clock frequency that the CPU that's running at 100F, and lowering it, thus lowering the speed some. Because it's no longer running as hot, they can plop a second, 3rd, or even 4th one right next to it without significantly affecting the overall temperature of the CPU, and gaining the benefits from multicore. This obviously comes with some overhead, as the cores still have to be controlled from one place, by one core. The more cores you add, the more overhead there is. As for single cores, the more you crank up the speed on them, the more heat they generate. This obviously has physical limitations (i.e. after a certain point, it starts becoming detrimental to performance, and even dangerous if it runs too hot)

As time goes by, they have found ways to decrease the physical size of the CPU's, which is why we haven't really ran into the requirement of needing more space yet, however for example, 16 core CPU's don't really exist (at the time of writing) outside of server grade equipment because of the physical size limitations, as server grade CPU's are significantly larger physically than standard consumer grade.

2

CPU = Car engine: It's easier to make a more powerful car with 16 valves, i.e a lamborghini, than a high rpm car that would have one giant valve / cylinder at 100 000 rpm.

The reasons are physical and chemical, silicon needs to be replaced with a computational rocket fuel to change the balance between number of cores and core speed.

2

I would say the primary restriction on computational power limits are primarily related to the limit of how fast we can move the electron through a circuit (speed of light electron drift). There are many more factors like you mentioned.

Adding additional cores would not make the processor faster, although it would allow it to process more in the same amount of time.

Moore's Law is a very interesting and informative read. This quote in particular is relevant here:

One could also limit the theoretical performance of a rather practical "ultimate laptop" with a mass of one kilogram and a volume of one litre. This is done by considering the speed of light, the quantum scale, the gravitational constant and the Boltzmann constant, giving a performance of 5.4258 ⋅ 1050 logical operations per second on approximately 1031 bits.

3
  • 2
    Incidentally, electrons actually move very slowly ("drift velocity"), in the order of millimetres per second IIRC. You're more referring to the speed at which the electric field propagates.
    – Bob
    Aug 15, 2014 at 14:43
  • 2
    I knew I ought not dare act like I understand physics in the slightest regard. :) Thank you for correcting/informing me on this though.
    – jredd
    Aug 15, 2014 at 14:44
  • Also, the actual clock speeds where propagation speed will have a significant effect are more around 10+ GHz. There are other reasons modern processors prefer multiple cores over higher clocks.
    – Bob
    Aug 15, 2014 at 14:51
1

Long story even shorter:

We really don't need faster CPUs. Outside of a few highly specialized uses* the CPU hasn't been the bottleneck for years - all the peripheral bits like memory, storage and network usually make the CPU wait for millions of clock cycles during which it can do other things. A second core can do more "other things", thus producing a perception of higher performance to the user.

Many mobile devices, laptops etc. will underclock the CPU for better battery life and cooler temperatures. Not much incentive to develop a 3.5GHz ARM core if your main customer runs it at 1.3GHz.

  • those specialized uses don't buy enough to justify development of a 5GHz core. They also don't care about the heat or power - buy the fastest available, overclock it and bolt on a water-cooled heatsink the size of a toaster.
3
  • 1
    It's not "perception", but "better latency". I think this is one of the main problems we have; a strong single-threaded throughput at all costs mentality. Consumer operating systems are not yet real-time oriented, nor fundamentally oriented to concurrency or parallelism.
    – Rob
    Aug 18, 2014 at 15:26
  • @peter You make a very good point, and thanks for explaining that. It's something I need to remember as a programmer. :) It's still a bit of a side issue for this question's purposes, though. My question was about why we can't get faster clock speeds; your answer is about why we don't currently need to. Aug 19, 2014 at 13:40
  • 1
    @NathanLong "can't" and "don't need" are linked. You can't get faster processors because no one is willing to invest the billion or five needed to develop it (5GHz is probably pushing physical limits anyway). No one is willing to invest because there is no realistic market when overall processor speeds are trending down - some current manufacture desktops are in the 1.5GHz class (top of the line 10 years ago) because it's cheaper, cooler and fast enough for that market segment.
    – peter
    Aug 20, 2014 at 2:05
1

Short and simple answer:

Why doesn't going from 1 truck hauling goods to 100 trucks hauling 100 times as much, face the same physical limitations as going from 1 truck driving at 100mph to 1 truck driving at 10,000mph?

Answer that question, and your question will also be answered. The concept is roughly the same.

1

I think another factor is temperature. If you increase clock frequency, the core temperature goes up. If you add more cores, even though the power consumption goes up, this is distributed over the cores, so the temperature stays the same (like if you add two hot liquids, at the same temperature, to each other, the temperature stays the same).

Another reason is that increasing clock frequency tends to increase power consumption by a square of the factor you increase the frequency (depending on which other barriers you are hitting at a given point). Hence, increasing clock frequency increases power consumption by a square, whereas adding more cores only increases it linearly.

0

The answer to the why doesn't "add more cores" face the same physical limitations as "make the CPU faster" question is that a multicore system faces the exact same limitations as a single core CPU. We have reached a point where we don't really have an option to make a single core system go faster, so we made it so we can do more things at once. The challenges of size and coordination are currently easier to solve than going faster. The downside is if a task can't be broken down into pieces, we may not end up going much faster than a single core system.

0

You can't make a CPU faster by simply cramming in more gates. Ultimately, instructions must be executed, and each instruction requires several "switching" operations. There are fundamental physical limits (quantum mechanics -- the Uncertainty Principle) that make it very difficult to increase the "clock speed" of a CPU beyond the speed of current top-end systems.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .