CPU usage is computed by the operating system's process/task scheduler. Indeed, if a CPU usage is 10%, that indicates that the task is actively running for 10% of the task scheduler's unit periods; other programs may run in the remaining 90% CPU time, or the OS will simply idle. Likewise, if the total CPU usage for all programs is 10%, that indicates that no programs on the system are being executed 90% of the time.
Since no programs run on the "bare metal" in a multitasked operating system (like Windows or Linux), CPU usage is a measure of what percentage your CPU's cycles are dedicated to running that one particular program. This is why if you have an infinite loop in a program, even though no "work" is being done, the CPU usage still approaches 100% (as the program is attempting to use every scheduling period offered to it by the operating system to execute some code).
Although processes are always running, they don't use 100% of the CPU in most cases since a process can wait for a particular event/interrupt to occur, or have indicated to the operating system to suspend/sleep its' operation for a short amount of time.