10

How can I get the true usage of a multicore hyperthreading enabled cpu?

For example lets consider a 2 core CPU, expressing 4 virtual cores.

A single threaded workload would now show up as 100% in top, as one core of the virtual cores is completely used. The CPU and top work as expected, like there would be 4 real cores.

With two threads however, the things get arkward: If all works well, they are balanced to the two real cores, so we got 200% usage: Two times 100% and two idle virtual cores, and are using all of the available CPU power. Seems ok to me.

However, if the two threads would run on a single real core, they would show up as using two times 100%, that makes 200% virtual core usage. But on the real side, that would be one core sharing its power on the two threads, which are then using only one half of the total CPU power.

So the usage numbers shown by top can not be used to measure the total CPU workload.

I also wonder how hyperthreading balances two virtual on a real core. If two threads take a different amount of cycles, would the virtual cores 'adapt' so that both show a 100% load even if the real load differ?

  • 1
    You do understand the operator system is not aware of the difference between a hyperthreading virtual core and the physical core right? – Ramhound Jun 28 '13 at 10:15
  • It seems so, but it doesn't have to? The real vs. virtual core mapping is a simple one to two map. The problem is how to measure load on a virtual core that actually changes its available performance by getting scheduled with another one on the real core. But all data is avalable I think, the question is just where are the tools that get a proper result out of them? – dronus Jul 19 '13 at 0:14
  • I just like to have a load measure where 100% would mean that every cycle of every real core is used. – dronus Jul 19 '13 at 0:17
  • 1
    Simply spoken: How to tell at a given moment, if my CPU would be capable to do further work, without slowing down the currently ongoing work? – dronus Aug 24 '13 at 20:13
  • 1
    @Ramhound, so if I have a physical 4-core processor with 8 logical cores, and my load averages say 4.00, am I at 100% utilization or 50%? – Buttle Butkus Oct 8 '15 at 21:26
5

Martin Tegtmeier at Oracle has written an interesting blog-post about this last year: https://blogs.oracle.com/solaris/cpu-utilization-of-multi-threaded-architectures-explained-v2

The short answer; Hyperthreading really messes with top's ability to report overall cpu-utilisation / cpu-idle percentages.

In the worst case, a 2-core 4-virtual-core CPU running 2 threads at 100%-utilisation-per-core, could nearly saturate the cpu. (Depending on execution port usage; only threads that use entirely different computing resources on the cpu could still run without affecting the performance on the current thread.) However, top will still report 50% idle in this case.

3

Core utilization is very different than the load on the system. Core utilization is only showing how much the core is calculating something or waiting for instructions. It can be 100% which correspons to any given time the cpu is calculationg someting.
But load is a different thing, load is a generally measured to determine if any process has to wait for any resource or not. If processes not waiting any resources you will see a very performant system. But sometimes you will see slow systems but low CPU utilization. Thay generally means some processes is waiting a resource and not releasing the cpu. At this kind of scenario you will not see high cpu utilization but the system may be well over its capacity. In a linux system Load avarage is a calculated value to measure the overall performance of a system. Value of the load avarage should be compared to the paralell computing resources, cores to be spesific. So if a system with 4 physical cores has a load avarage of 4 or more we can safely say that some processes will wait for a resource. It is not important if the cpu util is 100 or 10 percent. Load average can be as high as 200 or 300, in this cases system will be barely responsive. In a normal operating condition server load average should not exceed the number of cores for long duration. Short spikes are not important in my opinion. 3 numbers which you will see in a w output is load av. for 1/3/15 minutes.

  • The load averages reported by uptime or w are conventionally given for 1, 5 and 15 minutes. – Ján Lalinský Aug 15 '17 at 19:23
0

In my opinion none of the above answers is satisfactory.

I think the article I am referring on the following link is well targeted to answer this question: http://perfdynamics.blogspot.ch/2014/01/monitoring-cpu-utilization-under-hyper.html

QUOTE:

The idea behind HT is to allow a different application thread to run when the currently running app stalls; due to branch misprediction, bubbles in the pipeline, etc. To make that possible, there has to be another port or AS register. That register becomes visible to the OS when HT is enabled. However, the OS (and all the way up the food chain to whatever perf tools you are using) now thinks twice the processor capacity is available, i.e., 100% CPU at each AS port.

But under the hood, there is still only one execution unit: the single, physical, core you started with before HT was enabled. The difference is that it is being shared in some way between the 2 AS ports. How the single core gets switched between the two ports is very complicated but is most easily understood in terms of polled queues. I go into that level of detail in my GCaP classes.

The best-case test measurements I have, indicate that each HT port cannot become more than 75% busy, on average, or 150% of the total expected 200% capacity according to the OS. The "missing" 50% capacity, that I referred to earlier, is an illusion. Intel has claimed that something in the range of 120% to 130% can be expected for general applications.

In fact, I am pretty sure the operating system can reach 100% on each virtual core, no doubt about that. I have just done a:

mvn clean install -DskipTests -T 5

And i can assure you my 8 virtual cores, and 4 physical cores all went to 100% CPU utilizations. And i definitely do not have 8 cores on my machine.

Long story short, you can assume the following if the total CPU load goes above 100% you are at most, and most likely quite accuratenly, using exactly 100% of physical core. That menas, if you have physical CORE 1 split into operating system CPU 1 and CPU 2. And on CPU 1 you have a total usage of 50% and on CPU 2 you have total usage of 50%, most likely in real life you are putting a pressure of a total usage of 100% on that CPU. You have maxed it out.

But of course the operating system in its system monitoring tools has no idea that it is selling you an illusion. From the prespective of the operating system and how it manages resources, it will just believe eeach of those two virtual cores is still 50 percent idle so if there are more tasks to be put to run it will try to distribute them uniformly over those two cores. So when you go over 100% CPU utilization, during a period of CPU usage, there is always queued work to run in that period of time that never had a change to get a time sclice on the CPU. Eventually it will get it, but there are always some threads that are actually not even running even though they are scheduled to run.

Thanks

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.