Recommendations for introducing a shared nix store or cache for CI/CD and development

I’ll chime in on my experience integrating nix to do the CI for a small, private Rust project self-hosted GitLab.

I did this a year ago so details are fuzzy.

I wanted to experiment with a shell runner that invoked a single user nix installation. This acted as an alternative to the daemonized world of Docker.
The performance seemed better. Our project pulls in clang and other system dependencies so nix’s store wonderfully avoiding duplicating those downloads.

Is having this read/writeable recommended? How would people manage garbage collection of packages over time?

This is where I struggled. We self-host the GitLab runner on an EC2 instance and we kept filling the disk space from the size of /nix/store. I would manually run nix-collect-garbage -d to prune packages. Looking back, I probably should’ve created a scheduled pipeline to do that!

In the end I moved away from the Nix solution because I am the only one on my team familiar with Nix and I ran out of energy. I moved to a GitLab Docker runner and various Docker images for our jobs for ease of maintenance.

I’m wondering about my decisions as I investigate optimizing our Rust project’s CI pipeline.

makes looks promising! I also may try my shell runner with nix directly again. This time I’ll implement garbage collection or see if I can use S3 fs or some LInux trickery to mount larger disks for nix to use.

1 Like