NixOS tests with internet access

#1

Hi, I am currently playing with some NixOS tests for a personal project. At this point I am not even sure if they are the correct thing to use or what alternatives would exist.

My current problem is, that I want to create 2 nodes, one running a k3s in server mode and another one that wants to apply some kubernetes resource. This test is meant as a basic evaluation of the test infrastructure that is provided by the testing framework.

My current problem is, that k3s can’t properly deploy itself as it requires an internet connection to download some container images from the internet.

Also final tests will require downloading images from a remote registry anyway, therefore I need internet access for the server at least.

Is there a way to enable internet acces for a node in the tests?

#2

Relying on internet is definitely not a good idea in your context. I would be surprised if k3s doesn’t provide a way to specify/override all images it requires. If you find a way to specify all of these images, you could then use dockerTools.pullImage to pull these image properly and then run docker load -i ${SUCH-IMAGES} in your NixOS vm to make these images already available when you start k3s.

Note you could get internet access by disabling the nix sandbox.

#3

The way to override those images basically means giving up all of the advantages k3s gives and having to self-provision the cluster from scratch…

Thats at least how I understand it.

Also enabling/disabling the sandbox doesn’t seem to make any difference. Is the QEMU VM even started “within” the sandbox?

#4

Are you using the k3s nixos module?
Also, did you look at the NixOS test of this module?

Yes.

#5

The test for the k3s module strips everything from k3s, injects a single NOOP container and then checks if a non-priviliged user has indeed no access to the cluster info, while a “priviliged” one has.

Nothing tests if indeed the k3s would be able to self-provision and then eventually the services will be available from another computer.