This question already has an answer here:

This has been something that I can't seem to wrap my head around. Just about every modern processor is able to execute more instructions per second than its frequency.

I can understand why lower class processors can execute fewer IPS than its frequency. For instance, the ATmega328 executes about 16 MIPS at 16 MHZ (or at least that's what I've been told), and the Z80 executes 0.5 MIPS at 4 MHz. But then the Pentium 4 Extreme can execute more than 9 GIPS at only 3.2 GHz. That is about three instructions per clock cycle!

How is this done, and why isn't this implemented in smaller processors, such as AVR microcontrollers?

I found all of my information, except for the ATmega328, from here.

marked as duplicate by bwDraco, DavidPostill, Steven, nc4pk, BlueBerry - Vignesh4303 Jul 24 '15 at 15:14

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 3
    Have a look at Wikipedia for buzz words like pipelineing, super scalar architecture, out of order execution, simultanous multithreading, multi core, vector extension (SSE,...,AVX), caching and so on. All in all the the CPI value can be lower then 1.0. – Paebbels Jul 15 '15 at 6:26
  • 7
    You meant 9 GIPS for the Pentium 4, not MIPS, didn't you? – Calimo Jul 15 '15 at 9:32
  • 5
    Self-promotion: How can a CPU deliver more than one instruction per cycle? was asked on the Electrical Engineering Stack Exchange (my answer was accepted and perhaps too highly upvoted). – Paul A. Clayton Jul 15 '15 at 10:56
  • I've always wondered, should I not vote something up if it's already as high as I think it should be? Or always upvote if I think the effort and quality of a post deserves a reward? – Peter Cordes Jul 15 '15 at 13:00
  • @PeterCordes There are different voting philosophies. Some claim "useful" is the only criterion, others consider relative merit (for answers). I tend to consider absolute vote count ("nice", "good", "great" post badges imply such should be considered) as well as relative vote count (which helps answer ranking). Surprisingly Meta.SE does not seem to have much on this topic and "How should I vote?" doesn't even have an answer! – Paul A. Clayton Jul 15 '15 at 14:27

This is due to a combination of features of modern processors.

The first thing that contributes to a high IPS is the fact that modern processors have multiple execution units that can operate independently. In the below image (borrowed from Wikipedia: Intel Core Microarchitecture) you can see at the bottom that there are eight execution units (shown in yellow) that can all execute instructions concurrently. Not all of those units can secure the same types of instruction, but at least 5 of them can perform an ALU operation and there are three SSE capable units.

enter image description here

Combine that with a long instruction pipeline which can efficiently stack instructions ready for those units to execute instructions (out of order, if necessary) means that a modern processor can have a large number of instructions on the fly at any given time.

Each instruction might take a few clock cycles to execute, but if you can effectively parallelize their execution then you can give yourself a massive boost to IPS at the cost of processor complexity and thermal output.

Keeping these large pipelines full of instructions also needs a large cache that can be prefilled with instructions and data. This contributes to the size of the die and also the amount of heat the processor produces.

The reason this is not done on smaller processors is because it substantially increases the amount of control logic required around the processing cores, as well as the amount of space required and also heat generated. If you want a small, low power, highly responsive processor then you want a short pipeline without too much "extra" stuff surrounding the actual functional cores. So typically they minimise cache, restrict it to only one of each type of unit required to process instructions, and reduce the complexity of every part.

They could make a small processor as complex as as larger processor and achieve a similar performance, but then the power draw and cooling requirements would be exponentially increased.

  • I never knew that there were specialized execution units, and more than one of them. But still, it would be nice to have a superscalar Arduino. I wouldn't mind the power and cooling requirements if it meant more throughput. Thanks for the answer. – Jaca Jul 16 '15 at 2:53
  • 1
    If you want a superscalar processor, use one. But Arduinos occupy a niche for people who do mind the power and cooling requirements (and cost and complexity) of superscalar processors. – David Schwartz Jul 20 '15 at 21:49

It's not hard to imagine. One cycle is all it takes to switch many thousands of transistors. As long as instructions are lined up in parallel, one cycle can be enough to execute them all.

Better than trying to explain it myself, here's a good starting point.


To get a bit more fundamental than Mokubai's answer:

Superscalar CPUs analyse the instruction stream for data (and other) dependencies between instructions. Instructions that don't depend on each other can run in parallel.

Typical x86 desktop CPUs fetch 16 or 32B of instructions every clock cycle. Intel designs since Core2 can issue up to 4 instructions per cycle. (Or 5, if there's a compare-and-branch that can macro-fuse).

See Mobukai's nice answer for links and details on how CPUs in practice go about the task of extracting as much instruction-level parallelism as they do from the code they run.

Also see http://www.realworldtech.com/sandy-bridge/ and similar articles for other CPU architectures for an in-depth explanation of what's under the hood.


Previous answers show how one gets more instructions executed by the processor's definition of "instruction" and one imagines that is actually the questioner's intent.

But another source of it may be that each "instruction" is actually a certain amount of data treated as an instruction input by the processor. If his source's counting just counts what the processor regards as instructions, the following adds nothing. But if his source counts all of what a human would call an "instruction," then: Add in that not every instruction is as physically long as every other instruction (one might be 12 bytes, another might be 56 bytes, etc.). So if it loads 64 bytes of material each cycle as "an instruction" (or as many full instructions as it can before hitting 64 bytes)and one has six instructions in that 64 bytes, then six instructions (as you and I might regard them) will be finished in that cycle.

Since many very basic instructions (our "sensible" definition) are leftovers from early days with 8 byte instruction lengths, and very basic instructions are, by definition, perhaps used disproportionately, just this would go a long way to having more "instructions" performed than frequency would seem to allow.

  • That's a little bit close to how CPUs actually work, but fetching multiple machine insns per cycle is only part of how a superscalar CPU works. (and average insns lengths are more like 4 bytes, for x86.) Deep pipelines mean that instructions being fetched now might not finish executing for 15 cycles (or a lot longer, if held up by a cache miss, but non-dependent instructions could keep executing.) Your explanation doesn't describe a pipelined design at all. Also not clear on what you think the difference is between an 8bit 8086 add al, bl and 80386 add eax, ebx. – Peter Cordes Jul 15 '15 at 17:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.