How to calculate mean CPU and memory usage of a command line?

Question

The following command in UNIX:

time sh -c julia src/script_etudedtsi.jl 80 20 100 366 5 10 0 1 0 0 1 1

returned the elapsed time for my Julia script.

I tried:

pidstat -u -p $(pgrep -f julia src/script_etudedtsi.jl 80 20 100 366 5 10 0 1 0 0 1 1) 1
pgrep: only one pattern can be provided

How could I get the mean CPU and memory usage for the same command during the elapsed time?

Answer 1

The time command can give a way to generate basic stats.

time -v your_command

After running, time will show the "User" and "Sys" times, representing the CPU time used by the program. Do this a number of times. Calculate the "mean" CPU usage by averaging. Sum up your values, then divide by the number of CPU cores.

For memory usage, time outputs "maximum resident set size", which gives the "peak usage" in kilobytes. Average the values. If you want bytes, multiply the total by 1024.

You could use ps with bash scripts to do the polling:

Create run_command.sh with the following:

#!/bin/bash

"$@" &
pid=$!
echo $pid > /tmp/command.pid
wait $pid
rm /tmp/command.pid

Make executable: chmod +x run_command.sh

Create calc_mean_usage.sh with:

#!/bin/bash

show_help() {
    echo "Usage: $0 [OPTION]... COMMAND"
    echo "Calculate mean CPU and memory usage."
    echo "    --help                       Shows this help"
    echo "    --log=[file]                 Log polling intervals"
    echo "    --show                       Output to console"
    exit 1
}

log_file=""
show_output=false

# Parse options
while [[ $# -gt 0 ]]; do
    case "$1" in
        --help)
            show_help
            ;;
        --log=*)
            log_file="${1#*=}"
            shift
            ;;
        --show)
            show_output=true
            shift
            ;;
        --)
            shift
            break
            ;;
        *)
            break
            ;;
    esac
done

if [ $# -eq 0 ]; then
    show_help
fi

# Assign the command to an array
your_command=("$@")

# Run the wrapper and capture PID to a file
./run_command.sh "${your_command[@]}" &

# Wait for file creation and read the PID
while [ ! -f /tmp/command.pid ]; do
    sleep 0.1
done
pid=$(cat /tmp/command.pid)

samples=0
interval=1

# Initialize data arrays
cpu_usages=()
mem_usages=()

# Collect CPU and memory usage samples for duration
while kill -0 $pid 2> /dev/null; do
    cpu_usage=$(ps -o %cpu= -p $pid)
    mem_usage=$(ps -o %mem= -p $pid)

    cpu_usages+=($cpu_usage)
    mem_usages+=($mem_usage)

    if $show_output; then
        echo "CPU Usage: $cpu_usage%, Memory Usage: $mem_usage%"
    fi

    if [ -n "$log_file" ]; then
        echo "CPU Usage: $cpu_usage%, Memory Usage: $mem_usage%" >> "$log_file"
    fi

    sleep $interval
    samples=$((samples + 1))
done

# Calculate mean CPU and memory usage
total_cpu=0
total_mem=0

for cpu in "${cpu_usages[@]}"; do
    total_cpu=$(echo "$total_cpu + $cpu" | bc)
done

for mem in "${mem_usages[@]}"; do
    total_mem=$(echo "$total_mem + $mem" | bc)
done

mean_cpu=$(echo "scale=2; $total_cpu / $samples" | bc)
mean_mem=$(echo "scale=2; $total_mem / $samples" | bc)

echo "Mean CPU Usage: $mean_cpu%, Mean Memory Usage: $mean_mem%"

exit 0

Usage:
./calc_mean_usage.sh bash -c "julia src/script_etudedtsi.jl 80 20 100 366 5 10 0 1 0 0 1 1"

This runs your_command in the background and polls every second, for the entire runtime of your_command, and will output CPU and memory usage each second, then calculates the mean values.

This way you can set any polling interval and not be worried about it stopping.