Dns resolving very slow

I have a Lenovo X1 Carbon Yoga (4th Gen) with NixOS installed. The only problem so far is, that dns resolving take about 5 seconds.

A ping to 8.8.8.8 or 1.1.1.1 takes around 6ms. If I ping www.google.com, it takes 5 seconds to resolve the name and then the ping also takes around 6ms.

I don’t see this problem when booting from an Ubuntu live image.

I tried a minimal NixOS installation without gui and one using gnome3. 19.04 channel and unstable channel.

Are there any kernel modules which I should load. I don’t know howto debug this issue.
Can anybody help?

How are you configuring your DNS resolution?

  • Are you running a lock stub resolver?
  • Did you hard-code some resolver?
  • What resolver is your computer learning from e.g. RA/DHCP?

I configure the network via Gnome network manager.

I tried both, dhcp with the dns provided by the dhcp-server (192.168.1.1) and hard coded my IP + the following dhcp-servers: 1.1.1.1 8.8.8.8 192.168.1.1

I got an Ethernet extension link this with my notebook which I tried first. I also checked wireless, but the same problems are there.

I don’t know about lock stub resolver and never configured one.

cat /etc/resolv.conf
# Generated by resolvconf
domain lan
nameserver 192.168.1.1
options edns0

Host name resolution is controlled by the Name Service Switch (NSS) facility. Check the hosts value in /etc/nsswitch.conf for your configuration.

$ grep ^hosts /etc/nsswitch.conf
hosts:     files mymachines dns myhostname

The space-delimited list of services determines the order that services are queried.

  • files queries /etc/hosts
  • mymachines provides hostname resolution for the names of local systemd-machined containers
  • dns queries DNS according to the configuration in /etc/resolv.conf
  • myhostname provides local hostname resolution
  • resolve queries systemd-resolved, a caching DNS stub resolver

Note that there are other services, many of which are used in networks that include non-Linux systems (Unix, Windows, Mac, etc.). The services can be configured using system.nssHosts, but there are various other options that insert services as well.

When using a command like ping, a host is resolved by querying the NSS services in order until one is successful. Name resolution fails if all of the queries fail.

From what you wrote, the delay that you are experiencing does not sound like a DNS issue. This is easy to confirm using the dig utility, available via nixos.dnsutils. This utility queries DNS servers without going through NSS.

For example, here is a query using your configured nameserver:

$ dig +qr @192.168.1.1 A oeis.org

; <<>> DiG 9.14.9 <<>> +qr @192.168.1.1 A oeis.org
; (1 server found)
;; global options: +cmd
;; Sending:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26299
;; flags: rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 781a7ab0f2237616
;; QUESTION SECTION:
;oeis.org.			IN	A

;; QUERY SIZE: 49

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26299
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;oeis.org.			IN	A

;; ANSWER SECTION:
oeis.org.		7200	IN	A	104.239.138.29

;; Query time: 379 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: 水  3月 04 07:55:31 JST 2020
;; MSG SIZE  rcvd: 53

Here is a query using Google’s public nameserver:

$ dig +qr @8.8.8.8 A oeis.org

; <<>> DiG 9.14.9 <<>> +qr @8.8.8.8 A oeis.org
; (1 server found)
;; global options: +cmd
;; Sending:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63253
;; flags: rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 9753e8ab44749369
;; QUESTION SECTION:
;oeis.org.			IN	A

;; QUERY SIZE: 49

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63253
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;oeis.org.			IN	A

;; ANSWER SECTION:
oeis.org.		7199	IN	A	104.239.138.29

;; Query time: 492 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: 水  3月 04 07:59:03 JST 2020
;; MSG SIZE  rcvd: 53

DNS query times are highly dependent on caching. The query may be significantly faster on subsequent queries if the result is cached by the first query.

The getent utility performs name resolution using NSS. Unfortunately, it does not include features useful for debugging. (I wish that it had a --verbose mode that shows each step, with timing information.)

$ time getent hosts oeis.org
104.239.138.29  oeis.org

real	0m0.527s
user	0m0.001s
sys	0m0.003s

If your dig query time is short while your getent query time is long, then it indicates that the delay in name resolution is happening in an NSS service listed before the dns entry.

1 Like

Thanks a lot for the detailed explanation. My /etc/nsswitch.conf contains

hosts: files mymachines mdns_minimal [NOTFOUND=return] dns mdns myhostname

The dig and getent commands return immediately.

I configured system.nssHosts = ["files" "mymachines" "dns" "myhostname"]; and restarted. Now /etc/nsswitch.conf contains

hosts: files mymachines dns myhostname files mymachines mdns_minimal [NOTFOUND=return] dns mdns myhostname

> time getent hosts www.google.com
2a00:1450:4001:81a::2004 www.google.com

Executed in    5.07 secs   fish           external
   usr time    2.02 millis    0.00 micros    2.02 millis
   sys time    3.03 millis  1009.00 micros    2.02  millis
dig +qr @1.1.1.1 A www.google.com

; <<>> DiG 9.14.10 <<>> +qr @1.1.1.1 A www.google.com
; (1 server found)
;; global options: +cmd
;; Sending:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64755
;; flags: rd ad; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 6852f01b02f1c86f
;; QUESTION SECTION:
;www.google.com.			IN	A

;; QUERY SIZE: 55

;; connection timed out; no servers could be reached

You are welcome!

mdns and mdns_minimal are multicast DNS services. Multicast DNS is used for zero-configuration name resolution on local networks. Used with Avahi, it allows one to discover other hosts on a network (such as printers) without having to configure a DNS server.

With your original NSS hosts settings, before modifying your configuration, both dig and getent commands return quickly? After modifying your configuration, the results are the same? (Your getent query indicates at least a 5 second delay…) You still experience a long delay when using programs such as ping? You can time ping with a command such as the following:

$ time ping -c 1 www.google.com

The /etc/nsswitch.conf file is generated by nixos/modules/config/nsswitch.nix. As you can see in your tests, the values set in system.nssHosts are not the only ones configured. An array of fixed and optional (depending on settings) values are appended to the system.nssHosts values. There is no way to change these values (including order) without modifying nsswitch.nix.

Note that the [NOTFOUND=return] syntax is used to specify options for the preceding service. See man 5 nsswitch.conf for details.

If you are not using mdns, you can disable it. To do so, remove your system.nssHosts configuration and set services.avahi.nssmdns to false instead.

You can’t reach 1.1.1.1, apparently. That might be the source of the delays you see. I see your resolv.conf points to 192.168.1.1; I suspect from the description that the address (router?) forwards to 1.1.1.1 and 8.8.8.8 (but it’s not clear to me).

Note that dig command you used just asks 1.1.1.1 directly, ignoring any nsswitch/mdns/…

Thank you all for the help. I just reinstalled NixOS and now it works. I suspect that the gnome network manager mixed something up.

Reinstalling NixOS should never be necessary unless you actually destroyed the filesystem :rofl:
I noticed you mentioned 19.04, don’t you mean 19.09?

1 Like

yes, 19.09. It was not NixOS fault, but my usage of gnomes network manager. I know I can configure the network via configuration.nix, but this is a laptop and requires different network profiles per location which I can easily switch with a click or two.