I'm hoping to find an existing Linux tool for measuring latency:

I'm trying to diagnose how much latency my OS and NIC card are adding to TCP latency in Linux. I looked at the following tools but all (at least at the surface level) measured bandwidth (Mbps/sec) rather than latency (nanoseconds for transmition of 1 packet of size x).

nttcp nuttcp netperf iperf

If I were to write a custom test, it would likely do the following:


  1. Create a fake message of the specified length with some padding for an incrementing identifier. Open a TCP connection (with the right
    parameters like TCP_NODELAY, etc)
  2. Loop and send messages containing an incrementing identifier. Store the current system time (in nanos) associated with the identifier.
  3. Listen for responses, record the current time and record the latency of that identifier asynchronously


  1. Listen for a connection
  2. Echo back any message received

Assuming both boxes had the same setup (configs, cards, OS, CPU, etc), are relatively close to eachother in the network, and one knows roughly the network's latency contribution, one could get a rough estimate of the OS + NIC contribution and begin testing various configurations.

One could also get a more accurate picture of the latency by using a network sniffer and snooping on the lines between the two hosts, calculating the latency between the two sides for an ID, and then subtracting that from the internally measured latency.

So, does anyone know a tool or even some boilerplate code for testing this sort of thing?

  • @obrok I wish it were so easy. Ping uses ICMP, rather than TCP, so I wouldn't be testing the effectiveness of the whole stack. Also, there would be significant latency on the server box handling each packet, as typically inetd is doing the listening and forking processes to handle the ICMP packets. In practical terms, I'm expecting to see latencies around 2-10 microseconds, but I can see ping taking over 25 microseconds. So to clarify, it needs to test TCP and be optimize to introduce no extra context switches/forks/etc besides those part of the standard TCP handoff. – AnArcticPuffin Sep 14 '11 at 21:50

There is a tool called qperf that can provide you such statistics. Though, it is bound to OFED/RDMA packages.

# qperf -v xx.xx.xx.xx tcp_lat udp_lat
    latency        =  29.2 us
    msg_rate       =  34.2 K/sec
    loc_cpus_used  =    41 % cpus
    rem_cpus_used  =  11.5 % cpus
    latency        =  24.8 us
    msg_rate       =  40.4 K/sec
    loc_cpus_used  =    34 % cpus
    rem_cpus_used  =     4 % cpus
| improve this answer | |

Were you searching for ping by any chance?

| improve this answer | |
  • 1
    and traceroute/tracert ? Good luck to all! – shellter Sep 14 '11 at 21:32

Take a look at the in-kernel performance tool (tools/perf). It can at least provide you with some data on the NIC part.

There is also the LTTng suite with some net and syscall tracing support. What you need on the software side is the time between entering sys_write() and the exit of the function at the end of the start_xmit() call chain, so you might have to insert your own trace points.

| improve this answer | |

Just found almost exactly what I'm looking for (minus the sequencing that I can cross reference with a sniffer, but that's OK): http://linux.die.net/man/1/netpipe

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.