2

For example, I see that the incoming packet in tcpdump, but not in strace for the program that is listening the appopriate socket.

How do I trace the "fate" of this incoming packet on Linux?

I expect to get a report like this:

  • ✓ Packet received by network interface;
  • ✓ Packet successfully decoded as IPv4 packet (checksum correct, etc);
  • ✓ Packet passed iptables (not DROP'ed);
  • ✘ Packet routed to local socket (No, it's not our IP address);
  •   Packet passed traffic control (not dropped because of some overload);
  •   Packet received by local socket

Where I can get summarized info without manually investigating each possible stopping point (maybe I don't even know all stopping points)?

-2

tcpdump simply make a copy a data passing through NIC with the help of raw socket, working in TCP/IP stack in OS. Being able to capture data by tcpdump means packets really arrive at NIC.

strace simply prints every system calls. Not being able to capture it in strace means read or write system calls are not made. e.g. a read syscall invokes NIC driver to move data from NIC to kernel memory associated with a socket file descriptor and then copy them to user space buffer. strace can inspect arguments in read call and then prints them.

In addition, if it is a stream socket, I think accept syscall is already made and a file descriptor is allocated, so that client can send packets over and tcpdump is able to capture them. I got a live example here.

15:21:25.285198 IP (tos 0x0, ttl 55, id 0, offset 0, flags [DF], proto TCP (6), length 597)
    10.93.235.127.51524 > 10.225.131.26.8040: Flags [P.], cksum 0xefc8 (correct), seq 1:546, ack 1, win 2052, options [nop,nop,TS val 633828071 ecr 2357545597], length 545
POST /api/query HTTP/1.1
Content-Type: application/json
User-Agent: PostmanRuntime/7.15.2
Accept: */*
Cache-Control: no-cache
Postman-Token: e5584554-5747-4f2c-84c4-8e621b3bfeb2
Host: 10.225.131.26:8040
Accept-Encoding: gzip, deflate
Content-Length: 256
Connection: keep-alive

{"start":1567397847315,"end":1567398147315,"queries":[{"aggregator":"avg","metric":"endpoint.status.ok","rate":true,"rateOptions":{"counter":false,"diff":false},"tags":{"dc":"*","host":"n225"},"topK":""}],"allowCoprocessor":false}
15:21:25.285210 IP (tos 0x0, ttl 64, id 17260, offset 0, flags [DF], proto TCP (6), length 52)
    10.225.131.26.8040 > 10.93.235.127.51524: Flags [.], cksum 0x83fe (incorrect -> 0xb98e), ack 546, win 7513, options [nop,nop,TS val 2357545605 ecr 633828071], length 0
5 packets captured

but strace show it's only accepted but not read, so no packet is seen.

2524390 accept4(9, {sa_family=AF_INET6, sin6_port=htons(51524), inet_pton(AF_INET6, "::ffff:10.93.235.127", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [128->28], SOCK_CLOEXEC|SOCK_NONBLOCK) = 178687
2524390 getsockname(178687, {sa_family=AF_INET6, sin6_port=htons(8040), inet_pton(AF_INET6, "::ffff:10.225.131.26", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [128->28]) = 0
2524390 setsockopt(178687, SOL_TCP, TCP_NODELAY, [1], 4) = 0
2524390 getsockopt(178687, SOL_SOCKET, SO_SNDBUF, [65536], [4]) = 0
2524390 getsockopt(178687, SOL_SOCKET, SO_SNDBUF, [65536], [4]) = 0
2524390 setsockopt(178687, SOL_SOCKET, SO_SNDBUF, [131072], 4) = 0
2524390 getsockopt(178687, SOL_SOCKET, SO_SNDBUF, [262144], [4]) = 0
2524390 getsockopt(178687, SOL_SOCKET, SO_SNDBUF, [262144], [4]) = 0
2524390 setsockopt(178687, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
2524390 setsockopt(178687, SOL_TCP, TCP_NODELAY, [1], 4) = 0
2524390 setsockopt(178687, SOL_SOCKET, SO_RCVBUF, [131072], 4) = 0

it's similar for datagram socket.

| improve this answer | |
  • 1
    How does it answer the question? For example, how does it discriminate tcpdump-visible packet being dropped by iptables, by traffic control or by routing problem? – Vi. Sep 4 '19 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.