Record a process's User/System/Real time in Linux/Bash with greater precision

Question

What I am looking for is something along the lines of /usr/bin/time, however, this usually does not have enough precision.

I am running a C program (that I can edit and recompile) and I need to monitor User/System/Real time, processor usage, peak memory, etc.

However, the normal runtime is beyond the precision of the time command. time gives you millisecond precision, however, nanosecond precision is preferred. Tenth of a milisecond may work, but I digress.

What I have now is a simple bash script that records the start time in nanoseconds, runs the program, and then records the end time in nanoseconds and then reports the difference. This is great for Real (wall clock) time, but not so good for execution time because this is a multiuser machine.

Notes: I am restricted to Bash as a shell; the machine is running CentOS 6.4; I am open to solutions in C that don't require the installation of any new software.

Answer 1

You can do that using systemtap. From one of the examples in their website:

general/stopwatches.stp - See the amount of wall clock time a process spends in various states keywords: TIME

The stopwatch.stp script illustrates how to use multiple stopwatches record how much wallclock time a process spends in kernel- and user-space. On exit the script prints out the time in seconds, milliseconds, microseconds, and nanoseconds. Note that this output of this script is not directly comparable to the time command because time records the time that the process is actually active in kernel- and user-space.

# stap stopwatches.stp -c "sleep 1"

Code from that example reproduced here for completeness:

#! /usr/bin/env stap
# Copyright (C) 2012 Red Hat, Inc.
# by William Cohen <[email protected]>
#
# exercise the stopwatch tapset

probe begin
{
  start_stopwatch("wall");
  /* The next two lines assumes that target is running and in user-space. */
  start_stopwatch("user");
  stop_stopwatch("system")
}

probe syscall.*
{
   if (pid() != target()) next
   stop_stopwatch("user")
   start_stopwatch("system")
}

probe syscall.*.return
{
   if (pid() != target()) next
   start_stopwatch("user")
   stop_stopwatch("system")
}

probe end
{
  stop_stopwatch("wall")
  stop_stopwatch("user")
  stop_stopwatch("system")

  w = read_stopwatch_us("wall")
  u = read_stopwatch_us("user")
  s = read_stopwatch_us("system")
  printf ("wall clock                 %12dus\n", w);
  printf ("wall clock in user-space   %12dus\n", u);
  printf ("wall clock in kernel-space %12dus\n", s);

   delete_stopwatch("wall")
   delete_stopwatch("user")
   delete_stopwatch("system")
 }