Why is the CPU usage reported by top in Linux over 100%?

Question

I've seen the CPU usage go over 100%. How is this possible? Is it because I have four CPU cores, so 100% isn't really the max?

Also, when a program does this, what else can I do in top besides using the k command?

Answer 1

AFAIK top shows the CPU usage in relation to the power of 1 core, i.e if your task takes up 200% CPU then it's occupying two cores.

Answer 2

From the top man-page

   k: %CPU  --  CPU usage
  The task's share of the elapsed  CPU  time  since  the  last  screen
  update,  expressed as a percentage of total CPU time.  In a true SMP
  environment, if 'Irix mode' is Off, top will operate  in  number  of
  CPUs.  You toggle 'Irix/Solaris' modes with the 'I' interactive com-
  mand.

So, yes, top shows CPU percentage PER core, but you can toggle this behaviour be pressing 'I'.

Answer 3

Yes, if a process in top goes over 100% or top shows >1.0 in load, this means that more than one core is occupied, or the system is oversubscribed. Oversubscription means that there is more work to be put onto the core than is phsyically manageble.

In top you can also sort the view by different parameters (like used memory). If you want to look what you can sort by, refer to the help or man page for top.

You can also see all kind of information about the processes themselves, like Virtual or resident memory used by the process and if it's sleeping or currently running, as well as cpu time used.

The load average in the top, if needed, can also be viewed by the uptime command.

Answer 4

If hyper threading is enabled, the CPU usage shown in top is the usage per CPU, not core. For example, on a server with 24 cores and 2 threads per core (total of 48 CPUs), the usage could go up to 4800. Running lscpu -p provide a decent print of the cores and CPUs:

$ lscpu -p
# The following is the parsable format, which can be fed to other
# programs. Each different item in every column has an unique ID
# starting from zero.
# CPU,Core,Socket,Node,,L1d,L1i,L2,L3
0,0,0,0,,0,0,0,0
1,1,0,0,,1,1,1,0
...
...
...
46,22,1,1,,22,22,22,1
47,23,1,1,,23,23,23,1

I did the following steps to verify the above answer:

1. On the server, I started this loop:

$ while true; do x="reee";x="asdasd";z="asdasdasd;";  done

2. On a separate thread, I ran top and verified bash is using around 100% cpu:

   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 14791 hpaknia   20   0  129156   5188   1856 R  99.7  0.0  28:08.54 bash

3. Then in the same terminal of top, I pressed 1 to see the per CPU usage. Only CPU thread 3 out of 48 CPUs is using 100% CPU:

%Cpu3  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

In my case each core has 2 CPU threads. If top percentage showed the per core usage, the value must have been 50% but it's not. Because top shows per CPU thread usage.