How to test all agda packages?

@alexarice, me and other contributors are in the process of improving the Agda language support. Agda is a great programming language, but it has little tooling. For example Agda libraries typically don’t fix version numbers of their dependencies. This leaves us with manually curating a consistent package set. This is feasible because the Agda ecosystem is small enough. One can build all Agda packages on nixpkgs in an hour or two. But there is no tooling yet that makes this easy, or even runs this automatically on a PR.

What I’d like is a simple test that one can run, and that is ideally run on CI. I started a naive attempt here: https://github.com/NixOS/nixpkgs/pull/98214 but it’s unclear whether this is the right direction. What’s unclear to me is:

  1. Whether putting such a test in nixos/tests is the right approach at all
  2. How the test would be picked up by CI
  3. Whether it’s possible to run a test that builds all agda packages only when any dependency of the derivations in question changes
1 Like