1

I need to create a line chart to display the cpu usage of each process over time. How do I create the chart with time as the X axis, %CPU as the Y axis, then use the command name to indicate which line on the chart the data belongs to?

My data was created using the unix command:

pidstat -hdul 1 > file.txt

I then reformatted the data into csv using this command:

sed '1d;/^[#]/d;/^$/d;s/^[ ]*//;s/[ ]+/,/g' raw_data_file_input > nice_data_file.csv

My data is organized into the following columns:

Time, PID, %usr, %system, %guest, %CPU, CPU, KB_rd/s, KB_wr/s, KB_ccwr/s, Command

In other words, I want every command "kjournald" to be one line on the graph at various times, and "gnome-panel" to be another separate line.

Here is a sample of the data in csv format:

1320713878,680,0.00,0.00,0.00,0.00,0,0.00,35.64,0.00,kjournald
1320713878,2831,1.98,1.98,0.00,3.96,0,0.00,0.00,0.00,/usr/bin/X,:0,-br,-verbose,-auth,/var/run/gdm/auth-for-gdm-LiEP18/database,-nolisten,tcp,vt7,
1320713878,4360,0.00,1.98,0.00,1.98,0,0.00,0.00,0.00,gnome-terminal,
1320713878,7897,1.98,0.00,0.00,1.98,0,0.00,0.00,0.00,gnome-panel,
1320713878,24834,0.00,0.99,0.00,0.99,0,0.00,0.00,0.00,networking,networking,file:///usr/local/src/opensplice/install/HDE/x86.linux2.6/etc/config/ospl.xml,
1320713878,24986,0.00,1.98,0.00,1.98,1,0.00,0.00,0.00,pidstat,-hdul,1,
1320713879,2426,1.00,1.00,0.00,2.00,3,0.00,0.00,0.00,/usr/bin/prltoolsd,-p,/var/run/prltoolsd.pid,
1320713879,2831,2.00,1.00,0.00,3.00,2,0.00,4.00,0.00,/usr/bin/X,:0,-br,-verbose,-auth,/var/run/gdm/auth-for-gdm-LiEP18/database,-nolisten,tcp,vt7,
1320713879,7904,14.00,0.00,0.00,14.00,1,0.00,0.00,0.00,nautilus,--no-desktop,--browser,
1320713879,24834,0.00,1.00,0.00,1.00,0,0.00,0.00,0.00,networking,networking,file:///usr/local/src/opensplice/install/HDE/x86.linux2.6/etc/config/ospl.xml,
1320713879,24992,0.00,2.00,0.00,2.00,0,0.00,0.00,0.00,/bin/sh,./killAll.sh,
1320713880,2831,0.00,1.00,0.00,1.00,1,0.00,0.00,0.00,/usr/bin/X,:0,-br,-verbose,-auth,/var/run/gdm/auth-for-gdm-LiEP18/database,-nolisten,tcp,vt7,
1320713880,3466,0.00,1.00,0.00,1.00,2,0.00,0.00,0.00,/usr/sbin/nscd,
1320713880,4129,0.00,2.00,0.00,2.00,0,0.00,0.00,0.00,/usr/bin/prl_wmouse_d,-d,
1320713880,24986,0.00,2.00,0.00,2.00,2,0.00,0.00,0.00,pidstat,-hdul,1,
1320713880,24992,0.00,2.00,0.00,2.00,3,0.00,0.00,0.00,/bin/sh,./killAll.sh,
2

I don't know how familiar you are with the "Text-to-columns" tool on Excel's "Data" tab, but you can use that to quickly split apart your comma-delimited file.

I may be missing something, but, it appears to me that your sample data has only one instance of each "Command". I couldn't construct a time-dependent line chart with a single data point, so I made up some additional "dummy" data, with dummy values for each "time".

A pivot table will easily handle this. The pivot table will sort the data and you can filter it to only show certain categories ("Commands", in this case).

Once you've built your pivot table you can then click anywhere in the pivot table and "Insert" a chart. See below.

enter image description here

1
  • There are definitely multiple instances of command. Is it possible for you to post a link to that sample file you created so I can figure out how to correctly create the pivot tables? Oh I see, you simply filter each column based on each "Command" in the pivot table, then graph that. – Andrew Hundt Nov 9 '11 at 20:02
0

To analyse process resource consumption over time on a Linux system you can use Procpath (which I'm the author of). The built-in SVG visualisation cannot produce the chart you want (it shows PIDs instead of command names), but it's possible in the ad-hoc visualiser (i.e. Sqliteviz). You don't need extra software besides Python and a browser.

Record

I assume (looking at the pidstat arguments) that you want to record activity of all processes on the system and decide what to visualise at a later stage.

  1. Install Procpath by

    pip install --user Procpath

  2. Record all processes' procfs metrics with default 10-second interval

    procpath record -d all.sqlite

  3. Stop recording with Ctrl + C when you think you've collected enough data

Visualise

  1. Run Sqliteviz and drop the recording database, all.sqlite, there

    procpath explore

  2. You can use predefined CPU per PID visualisation (in My queries) as a basis. Add stat_comm to both SELECT statements. Add WHERE to inner SELECT according to the processes you want to visualise. If you're interested in kjournald and gnome-panel, then it can be WHERE stat_comm IN ('kjournald', 'gnome-panel') (and I'll use a couple of Cinnamon processes that I have running on my system). The modified query would look something like:

    WITH diff AS (
      SELECT
        ts,
        stat_pid,
        stat_comm,
        stat_utime + stat_stime - LAG(stat_utime + stat_stime) OVER (
          PARTITION BY stat_pid
          ORDER BY record_id
        ) tick_diff,
        ts - LAG(ts) OVER (
          PARTITION BY stat_pid
          ORDER BY record_id
        ) ts_diff
      FROM record
      WHERE stat_comm IN ('cinnamon', 'cinnamon-screen')
    )
    SELECT
      datetime(ts, 'unixepoch', 'localtime') ts,
      stat_pid,
      stat_comm,
      100.0 * tick_diff
        / (SELECT value FROM meta WHERE key = 'clock_ticks') / ts_diff cpu_load
    FROM diff
    
  3. Change the split field from stat_pid to stat_comm (in Transforms)

  4. Run the query and open Chart tab

    sqliteviz

  5. Optionally, you can tweak the chart in the UI (e.g. to set the title) or the SQL query, e.g. adding moving average over 10 records to smooth the lines by replacing cpu_load expression to the following window function expression

    AVG(
      100.0 * tick_diff
        / (SELECT value FROM meta WHERE key = 'clock_ticks') / ts_diff 
    ) OVER (
      PARTITION BY stat_pid
      ORDER BY ts
      ROWS BETWEEN 9 PRECEDING AND CURRENT ROW
    ) cpu_load
    

    with moving average

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.