Kernel and User mode distinguishing

Question

Who is responsible for determining that the process is in the Kernel mode or User mode? I know kernel knows which process is belong to what space, but how the CPU determines that? I mean should CPU be aware of the mode of the process or executing statements? If yes how? and if no then what happens when the user want to do something forbidden? When we say user application can only see a subset of the machines resources, I know it means applications cant do specific tasks for example in CPU, but who stops applications for doing such things and more importantly how?

Answer 1

Hm, fairly interesting question. I'll try to help...

Who is responsible for determining that the process is in the Kernel mode or User mode?

The OS designers decide that :-). All modern OS run all processes in User mode ("ring 3" in "Protected mode" in the x86 architecture), because this is necessary to use features like memory protection and virtual memory. Older and/or simpler OS may run all processes in Kernel mode (e.g. "Real mode" in x86); this depends on the OS design. MS-DOS for example worked like this.

Note that the actual names and types of processor modes differ between different architectures (x86, Sparc, PowerPC...); however all modern processors have similar "protected" modes, which one might call "user" modes.

I know kernel knows which process is belong to what space, but how the CPU determines that?

The CPU does not "know" it, because the CPU knows nothing of processes; these are abstractions provided by the OS. The CPU just executes code that is fed to it. The CPU has instructions to switch between different modes, and the OS uses these instructions to put the CPU into the right mode as required.

On x86 for example, the computer will start in "Real mode" (for compatibility reasons). When an OS such as Linux or Windows (NT and up) boots, one of the first things it does is to switch to "Protected mode". It then uses the "rings" feature to control every program's access to the hardware. The OS kernel runs in ring 0 (full privileges); user software runs in ring 3 (restricted). Whenever the OS passes control to user software (i.e. when it starts or resumes a user process), it will first switch to ring 3. Then control passes back to the kernel, the CPU switches back to ring 0.

How exactly the switching between modes/rings works depends on the CPU architecture. Most architectures provide special instructions or mechanisms for switching. Once the CPU has been switched to a certain mode/ring, it will keep track of that mode (and any associated restrictions) on its own.

For details on how this works on the x86 architecture, see this article: http://duartes.org/gustavo/blog/post/cpu-rings-privilege-and-protection

As an aside: The protections/restrictions in the restricted CPU modes are mostly implemented by the Memory management unit of the CPU. Older and/or simpler processors (such as the Motorola 68000 used by Amiga and Atari ST or the 6510 from the C64) do not have a MMU; therefore they cannot run an OS that distinguishes kernel and user mode. That is why for example the Linux m68k port requires at least a Motorola 68020 processor to run; the earlier 68000 and 68010 do not have a MMU.

I mean should CPU be aware of the mode of the process or executing statements? If yes how?

Yes, the CPU is aware of it in the sense that it knows which mode is is currently running in (though it does not know why).

and if no then what happens when the user want to do something forbidden? When we say user application can only see a subset of the machines resources, I know it means applications cant do specific tasks for example in CPU, but who stops applications for doing such things and more importantly how?

A very good question. The CPU itself stop the application.

If code (an application) is running in a mode with limited privileges (such as ring 3 in Protected mode on x86), there are certain things the code may not do (such as access memory outside the area it was assigned). The CPU knows this and checks every instruction for possible violations before executing it. If a violation is detected, the CPU stops executing the offending code (this is referred to as an "exception" or a hardware interrupt), and jumps to a special error handling code (that was set up by the OS in advance).

This effectively passes control back to the OS, which can then do as it sees fit: fetch memory from disk if the exception was due to accessing memory that had been swapped out to disk (this is how "paging" works), terminate the process it it accessed memory illegally (the dreaded "protection fault" or "segmentation fault"), etc.

Answer 2

Kernel and user mode are different security modes of the CPU. Often CPUs provide even more than these two modes that are implemented on the OS level.

For further details see

CPU Modes

and

Interoperation between CPU and OS levels of abstraction

Answer 3

The basic answer to your question is that: "The CPU switches to the kernel mode when an interrupt occurs.". As the OS itself is setting, in the boot process, the vectors of interrupts (the handlers), all these are pointing towards... the kernel. Every system call made by a process is nothing else than a (software) interrupt, that means calling the interrupt handler which is nothing else than the kernel itself (with parameters specified in the registers).

The kernel, before giving the control to a user process, sets the timer to generate an interrupt after some time, sets the processor into user mode and then gives control to the user process. After some time, the timer generates the interrupt and the control is back to the kernel.

Of course: playing with the timer, playing with the interrupt vectors (handlers) and playing with the cpu modes are all privileged instructions and are allowed to kernel-mode processes (that is the kernel itself).