Nix: 'make, but awesome'?

Random shower thought: I’ve been happily using NixOS for about half a year now, and I’ve always found it somewhat hard to wrap my brain around calling nix a ‘package manager’. Today it hit me: would it be easier to think about nix as ‘make, but awesome’?

nix is like ‘make’, but awesome:

  • ‘targets’ are called ‘derivations’
  • like with ‘make’, they can contain arbitrary scripts and command invocations
  • unlike ‘make’, determining whether a derivation needs to be rebuilt is actually reliable (since nix keeps track of dependencies for you, and enforces them by building in a sandbox)
  • derivation outputs can be cached, and this cache can be shared across machines (or globally)
  • derivations can be configured, and different configurations of a derivation can exist side-by-side
  • the ‘standard library’ of nix derivations, ‘nixpkgs’, contains derivations adding up to a complete Linux distribution: NixOS
  • ‘nixpkgs’ is managed as a monorepo to keep everything up-to-date, but…
  • it’s perfectly fine to mix different branches (even your own) of ‘nixpkgs’ in the same build. This is possible because the nix dependency tracking and isolation works so well, and it will still get most things from the standard caches, only rebuilding what’s unique to your build.

I guess it’s like “top-down vs bottom-up”, and calling nix a ‘package manager’ or a ‘build tool’ is just a matter of perspective - but this seems to make it easier for me to think about and explain it :wink:

4 Likes

It’s both. Unlike make, it’s not very incremental. Changing a single C file requires rebuilding the entire derivation. In that sense, it’s a package manager.

1 Like

That’s true, they’re typically used at different granularity.

not sure the status of it, but it looks like eelco tried to do some work in a related fashion https://github.com/edolstra/nix-make. I think it was mentioned at one of the nixcons

Relevant paper: Mokhov, Andrey, Neil Mitchell, and Simon Peyton Jones. “Build systems à la carte.” Proceedings of the ACM on Programming Languages 2.ICFP (2018): 1-29. Build systems à la carte | Proceedings of the ACM on Programming Languages

There are some prototype of haskell incremental building with nix: GitHub - matthewbauer/ghc-nix: Exploring the options for incremental Haskell builds using Nix

1 Like

There is also GitHub - nmattia/snack: Nix-based incremental build tool for Haskell projects

1 Like

But this is not a shortcoming of Nix itself, right? Theoretically, you could have per-C file derivations that are realized into a store path which is just the object file. And then you’d only have to recompile C files for which the inputs have changed (e.g. some header).

5 Likes

IIRC that’s similar to the nix-make I linked above

4 Likes

Kind of, but that increases the number of derivations in a system by multiple orders of magnitude, which quickly becomes a massive performance burden. The Nix evaluator and your file system will both severely struggle.

1 Like

For some filesystems, there is not a strong correlation between opening/writing a file and the number of files in a directory, only enumerating the files in the directory:

http://genomewiki.ucsc.edu/index.php/File_system_performance

Since Nix will open the files based on the output hash, that shouldn’t be a performance problem, of course, you could hit other limits (max inodes?).

The Nix evaluator would probably struggle indeed.

But for local C/C++ development (as opposed to building the whole system) it would probably work.

1 Like