30

I would like to keep track of the processes that initiate outbound connections on a Linux desktop. The best I can come up with is this:

iptables -A OUTPUT -m state --state NEW -j LOG --log-uid

This logs the uid/gid that initiates the connection, but not the process/command name or even the pid. If I could just get the pid, I could probably whip up a script that pulls the process name when the log is written, but it seems like that is not even possible.

Ideally I'd also like to log the processes that accept incoming connections too.

Any ideas how this might be possible with iptables [or anything else] on a Linux box?

2
  • I believe (not entirely sure) this question was answered on serverfault, look it up.
    – niXar
    Sep 18, 2009 at 7:23
  • Personally, I'd use sysdig for this job. Dec 13, 2016 at 23:42

6 Answers 6

7

You could write a program to monitor /proc/net/tcp, whose output looks like this:

obi-wan ~ # cat /proc/net/tcp
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
   0: 00000000:0050 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 4847458 1 e6060560 300 0 0 2 -1
   1: 00000000:04D2 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 4847477 1 f2e64da0 300 0 0 2 -1
   2: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 7109 1 f2e65ac0 300 0 0 2 -1
   3: 0100007F:177A 00000000:0000 0A 00000000:00000000 00:00000000 00000000  1000        0 4864457 1 d2726540 300 0 0 2 -1
   4: 00000000:01BB 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 4847462 1 e60609c0 300 0 0 2 -1
   5: 6B00A8C0:0016 30F4B5CA:C3AB 01 00000044:00000000 01:00000031 00000000     0        0 4982752 3 f2e64940 55 4 0 2 -1
   6: 0100007F:B143 0100007F:BC5E 01 00000000:00000000 00:00000000 00000000  1000        0 2130283 1 d59cce40 21 4 1 2 -1
   7: 0100007F:BC5E 0100007F:B143 01 00000000:00000000 00:00000000 00000000  1000        0 2130285 1 d59cd2a0 21 4 0 2 -1
   8: 6B00A8C0:0016 3276C35B:8E11 01 00000000:00000000 02:000ADAB1 00000000     0        0 4982629 2 d2727260 40 4 8 2 2
   9: 6B00A8C0:0016 6500A8C0:DD5D 01 00000538:00000000 01:00000029 00000000     0        0 4864416 5 e6061b40 42 12 27 3 -1

You can then relate opened ports to inodes, which can be related back to processes and file descriptors by doing readlink on the file descriptors listed for each process:

obi-wan ~ # readlink /proc/28850/fd/3
socket:[4847458]

See here that inode 4847458 corresponds to the first tcp socket in the list above. The output of netstat -tapn verifies this for me (and recall that 0x50 == 80):

obi-wan ~ # netstat -tapn
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN     28850/cherokee-work

When the monitor program notices a change in /proc/net/tcp, parse the data and determine if the change is a newly opened socket. Then you could just enumerate all the file descriptors for each process listed in /proc, doing readlink on each on to find the matching inode. Once you find that, you have the owning pid, from which you can get anything else you might want, particularly if you have process accounting on.

If you don't need your notification to be instantaneous, then your monitor program could use a slow poll (perhaps a period of 50ms or 100ms, or even 1000ms).

4
  • 2
    Thanks for providing an option! But requiring polling and querying every open file descriptor each time is not very robust and is very inefficient. I'm still hoping someone will find a better solution, or clarify why this isn't part of iptables any more, and why --cmd-owner is deemed unfixble.
    – nealmcb
    Jan 26, 2012 at 0:31
  • Unless the kernel changes its layout of /proc or unless netstat and readlink or ps change (unlikely), I'd say this is quite robust. What kind of efficiency issue(s) are you concerned with? If you want instantaneous processing, you'll have to write a kernel module for use with iptables. May 16, 2012 at 15:51
  • If I am logging rejected connections then the socket is destroyed immediately by kernel, so I have very little chance to see it in /proc. Maybe only change the REJECT to DROP to have the connection time out...
    – Marki555
    Nov 2, 2015 at 16:46
  • It doesn't help in this scenario beause of the super-small time window, but as far as the fragility of parsing /proc goes, one could also just use "lsof -F -i" and get a nicely abstracted dump of the network data. That can also be filtered (say, on just a specific port), and has already done all of the filename/pid/user mapping for you.
    – dannysauer
    Nov 12, 2015 at 22:16
10

You want the owner match module, which only works on the OUTPUT chain (and maybe PREROUTING...?). Read the docs, but it will work something like this:

iptables --append OUTPUT -m owner --cmd-owner "$app" \
--jump LOG --log-level DEBUG --log-prefix "OUTPUT $app packet died: "
3
  • 1
    Can an iptables log command set and interpolate a variable this way? It doesn't seem to work for me. Also, it looks like the --cmd-owner option was removed in kernel >= 2.6.15. This is unfortunate, because it seems like a useful option.
    – Nack
    Apr 8, 2009 at 17:00
  • 4
    Yes, --cmd-owner was removed: "unfixable broken"
    – guettli
    Aug 29, 2011 at 9:22
  • 2
    Thanks for the info, @guettli. There are more specifics at permalink.gmane.org/… which quotes more of the changelog: "[NETFILTER]: Remove tasklist_lock abuse in ipt{,6}owner ; Rip out cmd/sid/pid matching since its unfixable broken and stands in the way of locking changes to tasklist_lock." But I'd still like some more background, or better alternatives.
    – nealmcb
    Jan 26, 2012 at 0:44
5

Nothing to do with iptables or logging; but here is a "top" like interface that polls the /proc/ directory and displays bandwidth per program/pid:

http://sourceforge.net/projects/nethogs

"NetHogs is a small 'net top' tool. Instead of breaking the traffic down per protocol or per subnet, like most tools do, it groups bandwidth by process. NetHogs does not rely on a special kernel module to be loaded. "

1
  • I've found nethogs gives unreliable stats, but atop 2 (+ netatop) does this well.
    – Tobu
    Nov 2, 2012 at 13:53
1

As I'm looking into a similar question, trying to speed-limit skype, I've found

$ netstat -p | grep <mycmdname>

is a good way of linking port number to pid/cmd, now that pid-owner/cmd-owner are no longer directly supported in iptables; you'll then need to parse the result, then add the iptables rule according to the port; naturally you'll need some cleanup code afterwards / at system shutdown/restart, etc; save the port number/s to a file for reference at cleanup time

in fact a good answer to the port numbers question is

$ sudo netstat -p | grep "tcp.*<mycmdname>" | sed -r "s/.*<MYCOMPUTER>:([0-9]+).*/\1/"`

you may need to adjust the grep tcp element according to your needs

then for my purposes, it was simplest to add tc u32 filters according to port numbers, iptables entries according to port numbers similar

0

You can do that with ss command on Linux. I used it on Debian 10 and Ubuntu 20.04.

Here is the command:

ss --extended --processes --ipv4 -o state all '( dport = :ssh )'

This command list processes initiating connection to port 22 from the machine.

That is it. Read ss man for detail.

0

All the answers here (except for the "--cmd-owner" suggestion, which won't work with modern Linux) are based on polling. This does not work if it's just a single small packet that you are looking for.

Answers on https://serverfault.com/questions/192893/how-i-can-identify-which-process-is-making-udp-traffic-on-linux suggest using either 'auditd' or 'systemtap' and provide some usage examples.

1
  • 1
    Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center.
    – Community Bot
    Oct 11, 2021 at 13:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .