From which memory do CPUs actually read data?

Question

In accordance with my own understanding, for data to be processed by CPU, it should be placed in RAM, in the meantime CPUs themselves have cache memory, that is also used to access data. It's a pretty well known fact that CPU cache is 'closer' to the CPU and faster than any other kind of memory. But it's somewhat unclear for me, whether it is mandatory for data to be in CPU cache to be processed? If so, how the cache can keep big amount of data? (the cache size tends to be tiny, compared to RAM, up to a few mb, more often less than 1) If not, how a CPU decide which part of data is to be kept in its cache?

I also a little bit confused by swap. May CPUs treat it part of RAM and address the data kept there directly from HDD?

Answer 1

The cache on the CPU doesn't store the data, it just speeds up access.

The most basic explanation of the cache is that when the CPU goes to read a particular byte from RAM, that data gets copied into the cache as well. The next time the CPU needs to read that byte, the memory controller sees it's in the cache, and instead of making the slow call out to RAM, it just hands the CPU the data from the cache.

There are two specific issues with this explanation though:

1. The cache is a finite size. Because of this, when data is added, you eventually need to get rid of (or 'drop') old data to make room for it. There are a bunch of ways to do this, and the exact methodology doesn't exactly matter to this discussion, but what does matter is that after a while, the data may not be in the cache anymore, and then you ahve to go out to main memory anyway.

2. When data is written, you also need to either update or remove the data from the cache that was previously stored at that address. There are three main ways to handle this:

   • Writeback: Using the writeback method, the data is first writte to the cache, and then eventually written out to main memory. This has a number of advantages for performance, but can cause issues in some circumstances.
   • Writethrough: Using the writethrough method, the data is simultaneously written to both the cache and main memory. This is lower performance than writeback, but is safer.
   • Write-around: Using the write-around method, data is written directly to main memory, and then the data that was at the same location is removed from the cache. This is rarely used except for very specific use cases.

So, to answer your question regarding RAM versus the CPU cache, technically both, but the data ultimately comes from RAM in almost all cases. If you're interested in learning more about this specifically, I would suggest reading the following two Wikipedia articles:

https://en.wikipedia.org/wiki/Cache_(computing)

https://en.wikipedia.org/wiki/CPU_cache

Now, as to the matter of swap space, that is a completely different concept that is tied to virtual memory and paging. Trying to explain that here would result in me writing most of a couple of Wkipedia articles, so I'll just pooint you directly at the articles in question instead:

https://en.wikipedia.org/wiki/Virtual_memory

https://en.wikipedia.org/wiki/Paging