How to add network tests?

Help
1

I’d like to add a test for the crowdsec module.
For that, the setup script needs to pull some data from the crowdsec hub.
The test looks like this at the moment:

{ lib, ... }:
{
  name = "crowdsec";
  meta.maintainers = with lib.maintainers; [ tornax ];

  nodes = {
    simple = { ... }: {
      services = {
        # for `cscli hub update`
        resolved.enable = true;

        crowdsec.enable = true;
      };
    };
  };

  testScript = ''
    machine.start()
    machine.wait_for_unit("multi-user.target")

    # crowdsec should simply work...
    machine.succeed("pgrep --exact crowdsec")
  '';

  interactive.sshBackdoor.enable = true;
}

However, I’m still stuck at resolving the domain, if I execute the test with nix-build -A nixosTests.crowdsec:

simple # [   10.342303] crowdsec-setup[766]: Updating hub...
simple # [   10.377359] systemd-resolved[394]: Switching to fallback DNS server 1.1.1.1#cloudflare-dns.com.
simple # [   35.407857] crowdsec-setup[781]: level=warning msg="Failed to check last modified: failed to make HEAD request for https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json: Head \"https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json\": dial tcp: lookup cdn-hub.crowdsec.net: Temporary failure in name resolution" url="https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json"
simple # [   35.413364] crowdsec-setup[781]: Downloading /var/lib/crowdsec/hub/.index.json
simple # [   40.431115] systemd-resolved[394]: Using degraded feature set UDP instead of UDP+EDNS0 for DNS server 2606:4700:4700::1111#cloudflare-dns.com.
simple # [   45.603737] systemd-resolved[394]: Using degraded feature set UDP instead of UDP+EDNS0 for DNS server 2001:4860:4860::8888#dns.google.
simple # [   50.852856] systemd-resolved[394]: Using degraded feature set UDP instead of UDP+EDNS0 for DNS server 2620:fe::fe#dns.quad9.net.
simple # [   55.433918] crowdsec-setup[781]: Error: cscli hub update: failed to update hub: failed http request for https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json: Get "https://cdn-hub.crowdsec.net/crowdsecurity/master/.index.json": dial tcp: lookup cdn-hub.crowdsec.net: Temporary failure in name resolution

I assume that’s by design because each test should run without any external “dependency”?
Any ideas how I can fix this issue? (if I can create such a test at all)

Tbh. I’m a bit unsure about this one, because wouldn’t that mean that every PR of nixpkgs would create a GET request of the crowdsec server? Isn’t that going to create a lot of traffic for them? Doesn’t that mean that I should not test it (at least like that)?

Can you recommend any other modules with a similar problem and how they fixed it so that I can look at their approach?

2

Yes, the test clearly wouldn’t be reproducible if it were to depend on some internet server doing its thing.

Any ideas how I can fix this issue? (if I can create such a test at all)

I can think of these:

  1. make it so that the test runs without that file, maybe it’s not actually needed

  2. fetch the file with a fixed-output derivation like pkgs.fetchurl (you need to somehow pin the file to a specific version and hash it, but if the file returned by the URL changes unpredictably this is impossible)

  3. download the file manually, copy it to the Nix store (you just need to make a reference to it like{./thing.json} in the test script), set up a local webserver that serves it and redirect the domain to localhost. (assuming you can redistribute this file without violating some license or agreement)

3

Have a look at nixos/tests/redlib.nix, which tests a tool that wants to contact reddit dot com. The solution is indeed to stand up a local web server that pretends to be the web server the SUT wants to contact.