3

Yesterday I was thinking, when present x86 CPU wants to access memory there is some speed gap between them. Usually RAM operates at 1333MHz in case of some new Intel CPUs. But CPU itself works at about 3.2 GHz.

So, when CPU want to access memory, it sets the address to address bus, data to data bus (lets say we want to write), and now, does the CPU have to wait the extra time for memory, or is there some as CPU fast controller that handles that waiting time and CPU can do some more operations?

3
  • Yep, the chain is only as strong as its weakest link. This is why registry (which is located basically right on the cpu) is so fast, while ram is slower and physical devices like harddrives are even slower. Jul 14, 2010 at 10:08
  • 1
    @Javed Ahamed: Small point: It's called "registers" (not registry). Otherwise, you're perfectly right.
    – sleske
    Jul 15, 2010 at 8:35
  • Doh >.< I knew that I don't know why I typed that. Jul 15, 2010 at 9:33

6 Answers 6

2

when CPU want to access memory, it sets the adress to adress bus, data to data bus (lets say we want to write), and now, does the CPU have to wait the extra time for memory

From Ulrich Drepper's infamous memory article...

For write operations the CPU does not necessarily have to wait until the value is safely stored in memory. As long as the execution of the following instructions appears to have the same effect as if the value were stored in memory there is nothing which prevents the CPU from taking shortcuts. It can start executing the next instruction early. With the help of shadow registers which can hold values no longer available in a regular register it is even possible to change the value which is to be stored in the incomplete write operation.

Edit: Obviously for reads, all bets are off - I'm fairly sure you need to wait for the data to be read before operating on it;) As others have said, these delays can be lower if coming from a cache, or higher if accessing main memory.

5

In most CPUs a small amount of fast cache memory sits between the CPU and main memory, holding the code and data the CPU is currently working on as well as attempting to proactively read from the much slower main memory (or from the next level of cache).

2

"... say we want to write ... does the CPU have to wait the extra time for memory, or is there some as CPU fast controller that handles that waiting time and CPU can do some more operations?"

Yes, high-performance CPUs have caches and a bus interface unit and a write buffer so that, after the CPU does a STORE, the rest of the CPU can -- in most cases -- immediately continue to process instructions while the cache and bus interface unit and write buffer handle sending that data out to main memory.

Some CPUs prioritize LOAD instructions, so when that STORE instruction is immediately followed by a LOAD instruction, the CPU loads the requested data from RAM before storing the data to RAM.

2

does the CPU have to wait the extra time for memory

Yes, it does. As a matter of fact, the problem you describe is one of the biggest challenges CPU and system designers face when designing current hardware: RAM access is typically at least an order of magnitude slower than operations inside the CPU, so RAM access needs to be minimized.

This is the reason all modern CPUs have CPU caches, but even they can only mitigate the problem.

or is there some as CPU fast controller that handles that waiting time and CPU can do some more oparations

In general this is not possible, as when the CPU is waiting for data from RAM, it usually cannot go on without the data. With techniques like hyperthreading the CPU can switch to executing a different software thread while it is waiting, but this requires support from the running software (it must use threads).

In some cases, the CPU can do something useful while waiting for memory, by using "tricks" like out-of-order execution and speculative execution, but this is not always possible. The answers from Andy and harrymc also explain this.

For an extensive discussion of these problems, there's an excellent paper:

What Every Programmer Should Know About Memory by Ulrich Drepper

5
  • That link brings me to a page with nothing but the words "people.redhat.com" and a link to redhat.com.
    – garyjohn
    Jul 14, 2010 at 17:47
  • Oops, the document has moved, thanks for checking. Even Google is not yet up to date :-(. Link fixed.
    – sleske
    Jul 15, 2010 at 8:31
  • 1
    "CPU can do some more oparations" this is basically what hyper-threading does: en.wikipedia.org/wiki/Hyper-threading
    – pjc50
    Mar 8, 2012 at 13:52
  • @pjc50: Yes, hyperthreading allows the CPU to do something while waiting for RAM acces. However, this only works if the software uses multiple threads, hence my remark about "implement threads inside your software". This is not something that the CPU can do transparently for the application.
    – sleske
    Mar 8, 2012 at 14:54
  • Actually, in some cases the CPU can continue to work while waiting for RAM access. I edited my answer.
    – sleske
    Mar 8, 2012 at 14:55
1

If the CPU has to wait for RAM (and at today's CPU speeds this includes L2/L3 cache on the CPU die) the CPU will have to wait. This does slow processing (a thread running at 100% CPU could be spending most of its cycles idle waiting on RAM).

This is why consideration of data layout in memory (when cache lines are crossed) can have a large effect on the performance of tight data processing code).

(Correction to question: latest DDR3 RAM goes significantly beyond 1333, 1600 is common and faster is available without considering overclocking.)

1

is there some as CPU fast controller that handles that waiting time and CPU can do some more operations

Yes, but you would need a multi-core computer to make it happen, and with more than one thread actively using CPU.

This is the reason why the operating-system scheduler tries to distribute threads as evenly as possible between cores, and why each core normally comes with its own cache memory, to avoid inter-core waits.

In fact, when buying a multi-core computer, one should pay attention to the size of the memory cache, and specifically whether the amount claimed for the computer model is the total cache of all cores, so that in reality each core only uses a fraction of the total.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .