Is storage hierarchy the path of data flow?

Question

1. I was wondering if CPUs read and write data (almost) always following the hierarchy of storage:

   CPU <- register <- CPU cache memory <- main memory <- secondary storage ?

2. If without special treatment, can data, address or instructions in secondary storage or main memory sometimes be read or written by processing unit, skipping main memory, CPU cache memory and/or registers?

My questions come from the arrows in a picture of wikipedia,

Answer 1

You ask which steps in the access hierarchy can be skipped.

• skip registers

CPUs usually do not use registers as transparent caches. On all architectures I know, registers are usually explicitly addressed (or not), so yes, the CPU can bypass registers and read from RAM directly. However, on some CPUs some operations can only be done on registers (not directly on RAM), so here the hierarchy is enforced.

• skip cache

Usually RAM access goes via cache automatically. There are ways to disable this, however. E.g. on ix86 you can use the control register CR0 to globally disable the cache. Some CPUs also have commands to read or write while bypassing the cache. So usually the hierarchy is enforced, but you can avoid it if you want to.

• skip main memory

The virtual memory subsystem is implemented by the OS, so it would depend on the OS. I don't know of any OS that allows bypassing RAM and going directly to the backing store on disk. I don't think it would make sense, as it's meant to be transparent.

So in short: You can skip registers (they're only used explicitly), you can sometimes skip cache, you usually cannot skip main memory (except by reading files directly).

Answer 2

I was wondering if CPUs read and write data (almost) always following the hierarchy of storage:

CPU <- register <- CPU cache memory <- main memory <- secondary storage ?

No, that hierarchy is not always followed. That diagram you linked to is actually illustrating it not being followed, as it shows that the CPU can branch out to secondary/off-line storage directly (as opposed to through the southbridge). This concept is called MMIO (memory-mapped I/O), and maps these devices onto the system memory map.

What the diagram from Wikipedia is missing is another useful way for storage devices to place data into RAM directly through the northbridge. This is called DMA (direct memory access), and can greatly help performance. This allows your hard drive to place a program into memory without using the CPU (otherwise, the CPU would have to read one word at a time, and then place it into memory).

If without special treatment, can data, address or instructions in secondary storage or main memory sometimes be read or written by processing unit, skipping main memory, CPU cache memory and/or registers?

Again, while a CPU can directly access data on a device, if it wants to perform any operation on this data, it must be moved into a register. You cannot manipulate data without moving it in/out of a CPU register. Doing this involves instructions, which are held in the CPU's cache (when possible).

Caching mechanisms can be influenced by the programmer, but since the CPU cache is much faster then even your RAM, it is unwise to "skip" using the CPU cache and hold everything in RAM or on disk, unless you have a specific reason to.

As an example, this is useful in some video encoding applications, where certain data is only needed for a single instruction. If the video data was constantly being placed into the CPU's cache, and was only needed for those few clock cycles, the likelihood of a cache miss would be greatly increased.

Answer 3

AFAIK the CPU can only bypass the cache.

Some instructions can do that; e.g. the _mm_stream_pd instruction bypasses the cache. It's rarely useful.

You can't by pass registers, because you can only perform data operations on registers; of course, if you don't need any processing, then why do you need the data in the first place? :-)

You can't bypass main memory, because, well, the hard disk data needs to be loaded somewhere first (a hard disk or an optical drive is not a "special" of device in the CPU's perspective, unlike RAM; a CPU can very well run without any secondary storage, but not without RAM), and a cache isn't directly accessible* and registers are ridiculously small.

* Note 1:

IIRC Intel's I/O acceleration technology can actually load packet data from the network directly into the cache ("Direct Cache Access"), provided that all of your components match. But this is more of an exception than the rule, and I don't know of other exceptions right now.

Note 2:

There exist hard disk instructions (SCSI and/or ATA, I don't remember which) which let you tell the hard disk to copy data from one location to another. But at the same time, you don't have access to the data, so it's not really "bypassing" anything.