33

I am just wondering how I can find out in bash how many CPU cores a user is now using on a Linux Server?

I am submitting a fair amount of background jobs to the server, so I'd like to write a bash script to check before I submit a job if the running jobs (processes) submitted by me are too many, because I have to let other users have enough cores to use for their jobs. I'd like to know what bash command can tell me how many cores my jobs are using now.

Thanks and regards!

0

6 Answers 6

41

To get the number of CPU cores per CPU:

grep "^core id" /proc/cpuinfo | sort -u | wc -l

Or to get the number of physical CPUs:

grep "^physical id" /proc/cpuinfo | sort -u | wc -l
1
  • how to check this for a given user?
    – seralouk
    Oct 1, 2019 at 15:08
26

I don't know if it helps, but you can use the mpstat utility to get a breakdown of CPU usage by individual processor (or core). For example:

$ mpstat -P ALL 1

12:49:59 PM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
12:50:00 PM  all    7.89    0.00    1.25    0.88    0.00    0.00    0.00   89.97   1359.00
12:50:00 PM    0   14.00    0.00    0.00    0.00    0.00    0.00    0.00   86.00   1043.00
12:50:00 PM    1   15.84    0.00    7.92    3.96    0.00    0.99    0.00   71.29    297.00
12:50:00 PM    2    3.96    0.00    0.00    1.98    0.00    0.99    0.00   93.07      0.00
12:50:00 PM    3    3.96    0.00    0.99    2.97    0.00    0.00    0.00   92.08      0.00
12:50:00 PM    4    4.00    0.00    0.00    0.00    0.00    0.00    0.00   96.00      0.00
12:50:00 PM    5    4.95    0.00    0.99    0.00    0.00    0.00    0.00   94.06     18.00
12:50:00 PM    6   10.89    0.00    0.99    0.00    0.00    0.00    0.00   88.12      0.00
12:50:00 PM    7    5.05    0.00    0.00    0.00    0.00    0.00    0.00   94.95      0.00

In this example, you can see that CPU's 0, 1, and 6 are doing more work than the rest of them. Sometimes you will see that a single CPU is near (or at) 100% while others are at zero. This can be an indicator of program (or portion of a program) that is single-threaded and only able to use a single CPU at a time.

To install mpstat on a Fedora, RHEL, or CentOS system, use yum install sysstat.

6
  • Thanks a million for providing this answer! My colleagues were using the "top" command and making all the wrong conclusions about processor utilization until I showed them the mpstat command.
    – Nav
    Mar 11, 2011 at 5:07
  • Not available on Ubuntu: # aptitude search mpstat #.
    – Alix Axel
    Dec 2, 2012 at 0:16
  • 2
    @AlixAxel try aptitude search systat instead. Dec 2, 2012 at 5:07
  • @MattSolnit: Still no luck.
    – Alix Axel
    Dec 2, 2012 at 10:08
  • mpstat is provided by the package sysstat on Debian and on Ubuntu. I would expect most Debian-derived distributions use the same package name. On Debian-derived distributions, start by apt-cache search --full mpstat.
    – user
    Jun 22, 2016 at 8:46
2

Unless explicitly configured not to (ie. pinning a process to a specific CPU), all cores can be assumed to be in use at all times. The scheduler will allocate processes the next available core. Case in point, "System Monitor" (part of GNOME) is showing my load almost the same across all 4 cores of my machine.

3
  • I am submitting a fair amount of background jobs to the server, so I like to write a bash script to check before I submit a job if the running jobs(processes) submit by me are too many, because I have to let other users have enough cores to use for their jobs. So I like to know what bash command can tell me how many cores my jobs are using now.
    – Tim
    Oct 1, 2009 at 18:09
  • 3
    I don't think you understood my answer. The number of cores in use is ALWAYS the total number of cores available. Use the overall system load as a factor to determine whether or not your job submission should be throttled.
    – EmmEff
    Oct 1, 2009 at 18:45
  • Thank you! I do check the overall system load. I have been told by one of my colleagues that I can not use up all the cores because who knows others will run their jobs soon and my jobs will last for quite a while. Do you also think I have to limit the cores I am using?
    – Tim
    Oct 1, 2009 at 19:00
2

So you'll see there are responses here that will tell you how your cores are being utilized.

HOWEVER - this isn't really doing you a service. You've made a basic assumption that just doesn't hold - that your jobs will tend to group themselves onto some subset of the cores.

Instead, your jobs will be spread out across all of the cores, unless you implement something that keeps them "penned in" somehow. (Note: I'm not recommending that; just saying, "unless")

Here's an alternative strategy: Identify for your particular system what the LOAD level is when you feel it's "acceptable" for other users to be adding more jobs. Then, build something that only submits a new background job when the load level is down below that limit.

That way, the solution will be core-count-independent, more portable, more flexible, and easier to "tweak" as well.

3
  • I just now noticed a comment you added to another poster's answer, indicating this is because of something a colleague told you. So, indeed, it appears that the misconception is his, not yours. Sorry about that!
    – pbr
    Oct 1, 2009 at 21:22
  • Thank you! As to "Identify for your particular system what the LOAD level is when you feel it's "acceptable" for other users to be adding more jobs", if the current load is not mainly caused by my running jobs then I would feel more free to use the rest CPUs, but if in the current load my running jobs take a fair amount then I probably have to give up using the rest CPUs. So do you still think I don't have to estimate the CPU usage of my total running jobs?
    – Tim
    Oct 1, 2009 at 22:54
  • No, I really don't think it makes sense to distinguish between how much of the current load is from your jobs vs. other jobs. Consider the simplicity: your job tries to be a good citizen, and only submits itself if the load is below a level which you consider acceptable. The idea is NEVER to utilize all the remaining capacity; that leaves other future users in a bind. Ideally, all of the users would be using the same logic for this.
    – pbr
    Oct 2, 2009 at 16:08
2

If you want to do this so it works on linux and OS X, you can do:

CORES=$(grep -c ^processor /proc/cpuinfo 2>/dev/null || sysctl -n hw.ncpu)
1

You can get a rough estimate by running top, hitting 'U' and specifying your username, and then adding up the CPU utilization of your top few processes.

9
  • I like to know a bash command that can tell me the number, since I want to use it in a bash script.
    – Tim
    Oct 1, 2009 at 18:10
  • @Tim top can run in batch mode with the -b switch. Combine it with the -n switch to get the number of iterations you need.
    – user4358
    Oct 1, 2009 at 18:15
  • 1
    can you give an example of code for obtaining the number of cores used by all the jobs (processes) submit by a user?
    – Tim
    Oct 1, 2009 at 18:20
  • @Tim Not off the bat. I'd either have to write such a script by parsing the top output or google to see if someone's already done it. However, I'm not convinced its worth the effort because I think I'll get different outputs for every run given the speed at which context switching occurs. Isn't the system load a good enough indicator? It effectively tells you how many virtual cores are required to handle the process queue (processes waiting on the CPU).
    – user4358
    Oct 1, 2009 at 18:41
  • @ nagul: Yes system load is good and I also check that too. However some people keep telling me that I can not use up all the cores because who knows others will run their jobs soon. So I guess I have to limit the cores I am using. By the way, have you seen somewhere online scripts that check the CPU and memory load and dynamically submit jobs based on the usage? I have been looking them for so long. Thanks!
    – Tim
    Oct 1, 2009 at 18:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .