In my opinion none of the above answers is satisfactory.
I think the article I am referring on the following link is well targeted to answer this question:
The idea behind HT is to allow a different application thread to run
when the currently running app stalls; due to branch misprediction,
bubbles in the pipeline, etc. To make that possible, there has to be
another port or AS register. That register becomes visible to the OS
when HT is enabled. However, the OS (and all the way up the food chain
to whatever perf tools you are using) now thinks twice the processor
capacity is available, i.e., 100% CPU at each AS port.
But under the hood, there is still only one execution unit: the
single, physical, core you started with before HT was enabled. The
difference is that it is being shared in some way between the 2 AS
ports. How the single core gets switched between the two ports is very
complicated but is most easily understood in terms of polled queues. I
go into that level of detail in my GCaP classes.
The best-case test measurements I have, indicate that each HT port
cannot become more than 75% busy, on average, or 150% of the total
expected 200% capacity according to the OS. The "missing" 50%
capacity, that I referred to earlier, is an illusion. Intel has
claimed that something in the range of 120% to 130% can be expected
for general applications.
In fact, I am pretty sure the operating system can reach 100% on each virtual core, no doubt about that.
I have just done a:
mvn clean install -DskipTests -T 5
And I can assure you my 8 virtual cores, and 4 physical cores all went to 100% CPU utilization. And I definitely do not have 8 cores on my machine.
Long story short, you can assume the following if the total CPU load goes above 100% you are at most, and most likely quite accurately, using exactly 100% of physical core.
That means, if you have physical CORE 1 split into operating system CPU 1 and CPU 2.
And on CPU 1 you have a total usage of 50% and on CPU 2 you have total usage of 50%, most likely in real life you are putting a pressure of a total usage of 100% on that CPU. You have maxed it out.
But of course the operating system in its system monitoring tools has no idea that it is selling you an illusion.
From the perspective of the operating system and how it manages resources, it will just believe each of those two virtual cores is still 50 percent idle so if there are more tasks to be put to run it will try to distribute them uniformly over those two cores.
So when you go over 100% CPU utilization, during a period of CPU usage, there is always queued work to run in that period of time that never had a chance to get a time slice on the CPU. Eventually it will get it, but there are always some threads that are actually not even running even though they are scheduled to run.