tl;dr - how do you handle a migration to Nix for a project that already has an extensive list of tools and libraries pinned to specific versions?
I work as a devops engineer, and in my day-to-day, I use packages in three different contexts:
-
where I don’t care about the version of packages, e.g. editors, consoles. Outside of Nix, I install these either with my distribution’s package manager, possibly using a third-party repo, or as a pre-compiled binary.
-
where I care about the specific version of a tool, e.g. terraform, when I want to use the same version as in CI, the rest of my team etc. In these circumstances, I almost always get away with using the pre-compiled binary provided by most of the tools, or sometimes e.g. installing from pip or other language package manager.
-
where I’m packaging an application - e.g. a Python application for Docker - where I care about the Python library versions. These are defined in requirements.txt at fixed versions, with older / newer versions often breaking things. These get installed with the language package manager.
When it comes to adopting Nix for existing projects, it seems like adopting for 1) is straight forward: I can use Nix + packageOverrides or home-manager and get benefits of a repeatable workstation, a set of packages coming from a very up-to-date distribution etc.
For 2) and 3) - basically where the version of an application or library matters - it seems like it is harder to adopt Nix in existing projects.
If I was starting a new project, it would be OK. I’d pin nixpkgs and go with whatever version of the tools and libraries existed in nixpkgs at that time and evolve my code and dependencies as nixpkgs updates.
But in an existing project where teams have already got fixed versions of tools and libraries, the choices seem limited:
-
Make a massive effort to sync versions with what’s in Nix at the time of migration. Get colleagues and CI all using those versions, deal with any breakages by making code updates.
-
Use one of the search tools to discover a close enough version from an old revision of nixpkgs and pin to that for each of the tools. You may not get the exact version you were using before, but hopefully it is close enough to limit the amount of code changes required to work in your project.
-
Use language specific tooling like mach-nix or gomod2nix to work with language versions and fix to specific versions (only useful for 3). Varies from langauge to language - a lot of complexity in a polyglot environment.
-
Use overrides and try and figure out how to point at a different source / version for each application or library you care about and build it, figuring out the nuances of that application’s language packaging in nixpkgs, debugging any build failures etc.
I’m wondering if anyone else has experience with adopting Nix in existing projects? Does my breakdown of the different package contexts make sense? And does my breakdown of the different options for handling these in a migration to Nix accurate or am I missing options or approaches?
It’s taken me a while to understand this, I think because I’ve seen Nix billed as a replacement for pip, gems etc., but actually I think the way it solves the problem is quite different to these tools?