22

I used IPTraf, Iftop, vnstat, bwm-ng, ifconfig -a. None of them is helping me to find real-time packets that are being sent/received from my application in KB or MB format. The reason is I am writing an application, where I need to be very sure my compression is correct, but I can't test to move forward.

What I can use to track very specific and accurate real-time network statistics?

enter image description here

26

Your application is probably sending packets to a specific UDP or TCP port number or to a specific IP-address.

You can therefore use something like TCPdump to capture that traffic.

TCPdump doesn't give you the real-time stats you desire but you can feed it's output to something that does (I'll try to update this answer with an answer later).


Update:

$ sudo tcpdump -i eth1 -l -e -n | ./netbps
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
11:36:53    2143.33 Bps
11:37:03    1995.99 Bps
11:37:13    2008.35 Bps
11:37:23    1999.97 Bps
11:37:33    2083.32 Bps
131 packets captured
131 packets received by filter
0 packets dropped by kernel

I interrupted that after a minute by pressing Ctrl+C.

You'd need to add a suitable filter expression at the end of the tcpdump command to only include the traffic generated by your app (e.g. port 123)

The program netbps is this:

#!/usr/bin/perl
use strict;
use warnings;
use Time::HiRes;

my $reporting_interval = 10.0; # seconds
my $bytes_this_interval = 0;
my $start_time = [Time::HiRes::gettimeofday()];

STDOUT->autoflush(1);

while (<>) {
  if (/ length (\d+):/) {
    $bytes_this_interval += $1;
    my $elapsed_seconds = Time::HiRes::tv_interval($start_time);
    if ($elapsed_seconds > $reporting_interval) {
       my $bps = $bytes_this_interval / $elapsed_seconds;
       printf "%02d:%02d:%02d %10.2f Bps\n", (localtime())[2,1,0],$bps;
       $start_time = [Time::HiRes::gettimeofday()];
       $bytes_this_interval = 0;
    }
  }
}

It's just an example, adjust to taste.

  • Many thanks, i will wait for your working example. My goal was like i did with $ bwm-ng -o plain -N -d which shows output as bit by interfaces but that failes if the interface is used except lo. In other hand the IPtraf shows excellent bytes realtime. But there is no tools which can tell me like bits and bytes realtime in RX/TX for specific interface or any interface as total etc. I am missing it :-( – YumYumYum Nov 13 '11 at 11:05
  • @YumYum: answer updated. – RedGrittyBrick Nov 13 '11 at 11:46
  • 1
    @RedGrittyBrick You absolute legend! Great script! Thanks so much for sharing. This answers my question at superuser.com/questions/395226/… – Eamorr Feb 29 '12 at 8:52
  • This one should work in busybox bash on embedded routers: gist.github.com/dagelf/8a842ed755354b31e79214042a4a4695 – Dagelf May 23 '18 at 16:35
  • This is also achieved with vnstat -tr – St0rM Sep 28 '18 at 13:54
14

Usage as below from same folder:

To check packer per interface: ./netpps.sh eth0

To check speed per interface: ./netspeed.sh eth0

Measure Packets per Second on an Interface netpps.sh as file name

#!/bin/bash

INTERVAL="1"  # update interval in seconds

if [ -z "$1" ]; then
        echo
        echo usage: $0 [network-interface]
        echo
        echo e.g. $0 eth0
        echo
        echo shows packets-per-second
        exit
fi

IF=$1

while true
do
        R1=`cat /sys/class/net/$1/statistics/rx_packets`
        T1=`cat /sys/class/net/$1/statistics/tx_packets`
        sleep $INTERVAL
        R2=`cat /sys/class/net/$1/statistics/rx_packets`
        T2=`cat /sys/class/net/$1/statistics/tx_packets`
        TXPPS=`expr $T2 - $T1`
        RXPPS=`expr $R2 - $R1`
        echo "TX $1: $TXPPS pkts/s RX $1: $RXPPS pkts/s"
done

Measure Network Bandwidth on an Interface netspeed.sh as file name

#!/bin/bash

INTERVAL="1"  # update interval in seconds

if [ -z "$1" ]; then
        echo
        echo usage: $0 [network-interface]
        echo
        echo e.g. $0 eth0
        echo
        exit
fi

IF=$1

while true
do
        R1=`cat /sys/class/net/$1/statistics/rx_bytes`
        T1=`cat /sys/class/net/$1/statistics/tx_bytes`
        sleep $INTERVAL
        R2=`cat /sys/class/net/$1/statistics/rx_bytes`
        T2=`cat /sys/class/net/$1/statistics/tx_bytes`
        TBPS=`expr $T2 - $T1`
        RBPS=`expr $R2 - $R1`
        TKBPS=`expr $TBPS / 1024`
        RKBPS=`expr $RBPS / 1024`
        echo "TX $1: $TKBPS kB/s RX $1: $RKBPS kB/s"
done

Please refer this site for more info http://xmodulo.com/measure-packets-per-second-throughput-high-speed-network-interface.html

  • This is not a feasible solution for "specific port or application processID" as it only gives you the rate per interface. – Mene Sep 5 '16 at 11:17
6

Easiest to use and easiest to control output and redirect to file for continuous logging:

ifstat

Probably comes with most linux distributions, and can be installed with brew on mac

  • thanks for that. how do the others fair with Mac? I'll try to install ifstat and see how it goes. Some more info would definitely have been useful. – nyxee Apr 14 '17 at 0:51
3

I think you can use the proc interface to get the information you need. I created this little shell script called rt_traf.sh:

#!/bin/bash

cat /proc/$1/net/netstat | grep 'IpExt: ' | tail -n 1 | awk '{ print $8 "\t" $9 }'

This will print the in and out octets separated by a tab. Octets multiplied by 8 will give you bits/second and then divided by 10^6 will give you megabits/second. Of course you can add this to the shell script to format the output how you want it. You can call this with the PID of your application like so ./rt_traf.sh <PID> which will give you an instantaneous reading of your application since startup. To watch real time stats per second you can wrap the shell script in the watch command:

watch -n 1 ./rt_traf.sh <PID>

The -n parameter can be adjusted all the way down to tenths of a second. To do a calculation over time I would do something like this:

PID=<PID>; START=`./rt_traf.sh $PID`;IN_START=`echo $START | awk '{ print $1 }'`; OUT_START=`echo $START | awk '{ print $2 }'`; sleep 10; END=`./rt_traf.sh $PID`; IN_END=`echo $END | awk '{ print $1 }'`; OUT_END=`echo $END | awk '{ print $2 }'`; IN_BPS=`echo "scale=2; (($IN_START-$IN_END)/10)/8" | bc`; OUT_BPS=`echo "scale=2; (($OUT_START-$OUT_END)/10)/8" | bc`; echo "In: " $IN_BPS "Bits/second"; echo "Out: " $OUT_BPS "Bits/second"

Again the math can be adjusted for the size/times you need. Not the most elegant or shrink wrapped solution but it should work in a pinch.

  • 3
    This is wrong. /proc/<PID>/net/netstat doesn't contain per process data. – nab Nov 4 '13 at 21:22
  • To expand on nab's comment: /proc/<pid>/net/netstat returns the same data as proc/net/netstat, ie. you get the same data for any/all processes. – EML Apr 25 '17 at 8:16
2

I just needed to measure how much mysql traffic is going in and out on an old system where perl didn't work properly, so I couldn't resist writing few lines of awk that serve the same goal of measuring total traffic seen by tcpdump:

# tcpdump -l -e -n port 3306 | \
  awk '{
  t=substr($1, 0, 8);
  n=substr($9, 0, length($9)-1);
  if(t != pt){
    print t, sum;
    sum = 0;
  } else {
    sum += n
  }
  pt=t;
}'

tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
14:41:54 
14:41:55 466905
14:41:56 765220
14:41:57 741511
14:41:58 688219
14:41:59 492322
14:42:00 800087
14:42:01 1248608
14:42:02 1276476
14:42:03 755586
14:42:04 1029453
14:42:05 818298
^C32515 packets captured
32633 packets received by filter
107 packets dropped by kernel

And if you like one-liners better, here's one for you:

tcpdump -l -e -n port 3306 | awk '{t=substr($1,0,8);n=substr($9,0,length($9)-1);if(t!=pt){print t,sum;sum=0;}else{sum+=n}pt=t;}'
  • The else statement should be removed otherwise lines will be skipped, and sum won't be accurate. tcpdump -l -e -n port 3306 | awk '{t=substr($1,0,8);n=substr($9,0,length($9)-1);if(t!=pt){print t,sum;sum=0;};sum+=n;pt=t;} – david May 28 '18 at 20:19
0

Per application can be done with a firewall rule using xtables and a modification of the below.

This doesn't answer the "per application" question but just the "per interface" one.

Below is a script that works on most embedded Linux routers such as Ubiquiti and OpenWRT compatible ones and gets its details from /proc/net/dev.

(And easy to change to packets etc.)

#!/bin/sh

SLP=1 # output / sleep interval
DEVICE=$1
IS_GOOD=0
for GOOD_DEVICE in `grep \: /proc/net/dev | awk -F: '{print $1}'`; do
    if [ "$DEVICE" = $GOOD_DEVICE ]; then
        IS_GOOD=1
        break
    fi
done

if [ $IS_GOOD -eq 0 ]; then
    echo "Device not found. Should be one of these:"
        grep ":" /proc/net/dev | awk -F: '{print $1}' | sed s@\ @@g 
    exit 1
fi

while true; do

LINE=`grep $1 /proc/net/dev | sed s/.*://`;
RECEIVED1=`echo $LINE | awk '{print $1}'`
TRANSMITTED1=`echo $LINE | awk '{print $9}'`
TOTAL=$(($RECEIVED1+$TRANSMITTED1))

sleep $SLP

LINE=`grep $1 /proc/net/dev | sed s/.*://`;
RECEIVED2=`echo $LINE | awk '{print $1}'`
TRANSMITTED2=`echo $LINE | awk '{print $9}'`
SPEED=$((($RECEIVED2+$TRANSMITTED2-$TOTAL)/$SLP))
INSPEED=$((($RECEIVED2-$RECEIVED1)/$SLP))
OUTSPEED=$((($TRANSMITTED2-$TRANSMITTED1)/$SLP))

printf "In: %12i KB/s | Out: %12i KB/s | Total: %12i KB/s\n" $(($INSPEED/1024)) $(($OUTSPEED/1024)) $((($INSPEED+$OUTSPEED)/1024)) ;

done;

Copy the above to your clipboard and then in a terminal session on your router:

$ cat > /tmp/n.sh

then: Ctrl+V (or right click / Paste)

then: Ctrl+D

$ chmod +x /tmp/n.sh

$ /tmp/n.sh eth0

You can also paste it in a notepad, then just repeat the above if you need to edit it - not all embedded routers have an editor! Make sure you copy everything from the # at the top to the done; at the bottom.

The netpps example above is great BTW - but not all devices have a mounted /sys filesystem. You might also have to change the /bin/bash to /bin/sh or vice versa.

Source: https://gist.github.com/dagelf/ab2bad26ce96fa8d79b0834cd8cab549

  • Please do not post the same answer to multiple questions. If the same information really answers both questions, then one question (usually the newer one) should be closed as a duplicate of the other. You can indicate this by voting to close it as a duplicate or, if you don't have enough reputation for that, raise a flag to indicate that it's a duplicate. Otherwise tailor your answer to this question and don't just paste the same answer in multiple places. – DavidPostill May 23 '18 at 17:15
  • You posted exactly the same comment on my other post... which was incidentally customized... This kind of response just makes me wonder why I took the time to contribute here. – Dagelf May 24 '18 at 19:52
  • If it was customised then that happened after the posts were autoflagged by the system. I don't have time to read every word to see if they are customized or not. – DavidPostill May 24 '18 at 20:24
  • 1
    The only customization you did was to add two sentences at the start of your anser. The remainder seems identical. And those two sentences were not in the original answer as posted - which is why it was automatically flagged as a duplicate answer. – DavidPostill May 24 '18 at 20:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.