1

Problem

ping: service.sys-dev.company.com: Name or service not known

host, nslookup and dig works

# host service.sys-dev.company.com
traefik-proxy.ingresscontrollers.svc.cluster.local has address 10.0.60.226

Details

Coredns zone

sys-dev.company.com:53 {
errors
log
ready
health
rewrite name regex (.*)\.sys-dev\.company\.com traefik-proxy.ingresscontrollers.svc.cluster.local
kubernetes cluster.local in-addr.arpa ip6.arpa {
  pods insecure
  fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
# dig @10.0.0.10  service.sys-dev.company.com

; <<>> DiG 9.18.19-1~deb12u1-Debian <<>> @10.0.0.10 service.sys-dev.company.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27146
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 9ea30e2a15ac33b7 (echoed)
;; QUESTION SECTION:
;service.sys-dev.company.com. IN        A

;; ANSWER SECTION:
traefik-proxy.ingresscontrollers.svc.cluster.local. 5 IN A 10.0.60.226

;; Query time: 3 msec
;; SERVER: 10.0.0.10#53(10.0.0.10) (UDP)
;; WHEN: Mon Jan 08 14:15:48 UTC 2024
;; MSG SIZE  rcvd: 139

I found a difference in the strace that I would like someone to explain. I have two containers and this command behaves differently in them.

  • The settings of /etc/nsswitch.conf and resolv.conf are the same.

/etc/nsswitch.conf

passwd:         files
group:          files
shadow:         files
gshadow:        files

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

/etc/resolv.conf

search xxx.svc.cluster.local svc.cluster.local cluster.local m3ked1pts4futor0jcy3sg2y4d.ax.internal.cloudapp.net
nameserver 10.0.0.10
options ndots:5

OK

LSB: Ubuntu 22.04.3 LTS

/etc/hosts

127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
10.142.82.26    dotnet8

Full strace

Here you can see that the last connect call already contains the IP

3095  connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.10")}, 16) = 0
3095  poll([{fd=5, events=POLLOUT}], 1, 0) = 1 ([{fd=5, revents=POLLOUT}])
3095  poll([{fd=5, events=POLLIN}], 1, 5000) = 1 ([{fd=5, revents=POLLIN}])
3095  recvfrom(5, "\273y\205\0\0\1\0\0\0\1\0\0\vservice\tsys-dev\6company\3com\0\0\34\0\1\7cluster\5local\0\0\6\0\1\0\0\0\5\0D\2ns\3dns\7cluster\5local\0\nhostmaster\7cluster\5local\0e\234\0\30\0\0\34 \0\0\7\10\0\1Q\200\0\0\0\5", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.10")}, [28 => 16]) = 143
3095  poll([{fd=5, events=POLLIN}], 1, 4997) = 1 ([{fd=5, revents=POLLIN}])
3095  recvfrom(5, "o\7\205\0\0\1\0\1\0\0\0\0\vservice\tsys-dev\6company\3com\0\0\1\0\1\rtraefik-proxy\22ingresscontrollers\3svc\7cluster\5local\0\0\1\0\1\0\0\0\5\0\4\n\0<\342", 65536, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.10")}, [28 => 16]) = 116
3095  connect(5, {sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr("10.0.60.226")}, 16) = 0
PING traefik-proxy.ingresscontrollers.svc.cluster.local (10.0.60.226) 56(84) bytes of data.

NOK

LSB: Debian GNU/Linux 12

/etc/hosts

127.0.0.1   localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
fe00::0 ip6-mcastprefix
fe00::1 ip6-allnodes
fe00::2 ip6-allrouters
10.142.83.141   dotnet8-test

Full strace

1153  connect(5, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.10")}, 16) = 0
1153  poll([{fd=5, events=POLLOUT}], 1, 0) = 1 ([{fd=5, revents=POLLOUT}])
1153  poll([{fd=5, events=POLLIN}], 1, 5000) = 1 ([{fd=5, revents=POLLIN}])
1153  recvfrom(5, "\211\226\205\0\0\1\0\0\0\1\0\0\vservice\tsys-dev\6company\3com\0\0\34\0\1\7cluster\5local\0\0\6\0\1\0\0\0\5\0D\2ns\3dns\7cluster\5local\0\nhostmaster\7cluster\5local\0e\233\377\316\0\0\34 \0\0\7\10\0\1Q\200\0\0\0\5", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.10")}, [28 => 16]) = 143
1153  poll([{fd=5, events=POLLIN}], 1, 4997) = 1 ([{fd=5, revents=POLLIN}])
1153  recvfrom(5, "|\227\205\0\0\1\0\1\0\0\0\0\vservice\tsys-dev\6company\3com\0\0\1\0\1\rtraefik-proxy\22ingresscontrollers\3svc\7cluster\5local\0\0\1\0\1\0\0\0\5\0\4\n\0<\342", 65536, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.0.0.10")}, [28 => 16]) = 116

After the last recvfrom call there is only

1186  close(5)                          = 0
1186  write(2, "ping: ", 6)             = 6
1186  write(2, "service.sys-dev.company.com: Name or service not known", 59) = 59
1186  write(2, "\n", 1)                 = 1
1186  close(1)                          = 0
1186  close(2)                          = 0
1186  exit_group(2)                     = ?
1186  +++ exited with 2 +++
ping: service.sys-dev.company.com: Name or service not known

I am very interested in why the behaviour is different and I will be glad if someone can help me to clarify or direct me. Thank you for your time.

16
  • I think your issue is related to this : superuser.com/a/1417265/1875547
    – hidigoudi
    Commented Jan 8 at 18:40
  • In Ubuntu 22.04.3, the /etc/nsswitch.conf file is like : hosts: files dns - In Debian 12, the /etc/nsswitch.conf is like : hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname- In Ubuntu, you reach the DNS server, in Debian, you don't. - So the ping does not work because it uses nsswitch file to resolve names, the host command uses directly the resolv.conf file.
    – hidigoudi
    Commented Jan 8 at 18:50
  • I also added nsswitch.conf. They are identical and contain hosts: dns files.
    – Neurobion
    Commented Jan 8 at 19:53
  • Do you have this behavior with all names ? Or only this one ? - With a special domain (.local) ?
    – hidigoudi
    Commented Jan 8 at 20:46
  • Furthermore, Ubuntu uses systemd-resolved service as default for DNS resolver, do you have something in the file : /etc/systemd/resolver.conf ? - What do you have inside both /etc/resolver.conf ? Thanks.
    – hidigoudi
    Commented Jan 8 at 21:12

1 Answer 1

1

The problem is caused by the difference between the ANSWER and QUESTION sections. The following modification is required to be used with CoreDNS:

https://github.com/coredns/coredns/discussions/6460

rewrite stop {
        name regex service.sys-dev.company.com traefik-proxy.ingresscontrollers.svc.cluster.local
        answer auto
    }

Explanation: https://github.com/coredns/coredns/tree/master/plugin/rewrite#response-rewrites

When rewriting incoming DNS requests' names (field name), CoreDNS re-writes the QUESTION SECTION section of the requests. It may be necessary to rewrite the ANSWER SECTION of the requests, because some DNS resolvers treat mismatches between the QUESTION SECTION and ANSWER SECTION as a man-in-the-middle attack (MITM).

The tests showed that for curl/dotnet in combination with Debian/Alpine it requires this match to work properly. Difference between the ANSWER and the QUESTION, it evaluates it as a negative and non-existent record.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .