Wow. My answer turned into a huge wall of text. Guess I have a lot to say on this issue…
Well anyway, here it is:
-
Replace the nix language with something else, still lazy & purely functional, but with strong typing and clean abstractions. Some specific differences I’d like to see:
- Strong separation of the eval-time filesystem path space from the runtime filesystem path space. Ideally with a guarantee that eval-time paths cannot escape into build outputs and runtime paths cannot be accessed at eval time. Nix has some nice defaults to make this separation easier, most notably the way the
${foo}notation handles paths vs strings, but nothing in the language forces you to use them correctly. See the strange behavior of home-manager’shome.file.<name>.sourceoption. - A built-in override system that can be applied broadly with little effort. “Please make this thing overrideable” should be a trivial request to complete.
- Full evaluation caching for flakes would be nice. (Not just caching outputs, but intermediate shared values as well)
- Strong separation of the eval-time filesystem path space from the runtime filesystem path space. Ideally with a guarantee that eval-time paths cannot escape into build outputs and runtime paths cannot be accessed at eval time. Nix has some nice defaults to make this separation easier, most notably the way the
-
Devshells need a ground-up restructuring:
- Specify a datastructure describing a “shell environment” in a way that’s as independent of one’s choice of shell as possible, and where it can’t be independent, it can support multiple shells.
- Specify a representation of that datastructure in the nix store, which nix commands will understand.
- Create helper functions to create these environments easily.
- If possible, re-implement the build-alike environments, that is, the original conceived usage of
nix-shell, in terms of these new shell environment specifications.
-
Build the infrastructure for sharing build artifacts between devshell builds and nix builds. Recursive nix should make it possible to do this even with upstream build systems, if we put in the work to make wrappers for compilers, etc.
-
Think about flakes a bit differently, hopefully cleaning up the rough edges of the abstraction:
- Flakes should allow arguments, but require default values for them.
- When evaluating a flake, the copy in the nix store should be modified to reflect the specified arguments, input overrides, etc, so that if you keep a reference to
self, and evaluate it again, you get exactly the same outputs, guaranteed. See nixpkgs#6894 and nixpkgs#6895. - In short, a well-formed flake should always specify a fully reproducible set of outputs, but it should also be thought of as specifying automatic processes to modify itself in various intended ways, such as updating inputs, overriding inputs, and altering arguments.
- Allow computation in flake inputs, and inputs depending on other inputs. Mostly the nixpkgs lib in practice, I expect. Perhaps also cache some computable flake information, keyed to the hash of the whole flake not including the cache itself. This could allow an extra user check before running potentially long computations because the cache is invalid.
- Don’t auto-lock flakes. Instead, error out when attempting to evaluate a flake that’s not fully locked (this should be checked after any specified input/argument overrides are applied).
- Make special transformations of flakes, such as
git crypt unlocking them, part of the flake uri schema. As this produces a different flake, it should have a different uri.
-
Fixed-output derivations are too easily abused, and often lead newcomers into a trap of effective non-reproducibility through fragile hashes. Remove them in favor of a combination of:
- Builtin impure fetchers; “impure” meaning they do not specify a hash and are always rebuilt rather than using a cached version.
- Impure derivations which are still fully sandboxed, with no network connectivity, but can rely on impure inputs and are themselves “impure” in the same sense as above. Note that this is not quite the same thing as the “impure derivations” that have been implemented as an experimental feature in nix.
- A “purify” operation, which takes a list of impure derivations and a hash, and tries the derivations in order until one succeeds and gives the correct hash, renaming it to a content-based name. Nix might also be configured to attempt other methods of finding something with the specified hash beyond what’s specified in the nix code.
-
Explore well-abstracted methods of managing runtime state, and how they might integrate with nix.
- Nix profiles and, more generally, gcroots, should ideally fit into this state-management system cleanly, alongside general system/service/program state.
- Any given bit of state should have a stable filesystem path, regardless of actual storage location on the underlying system. This minimizes the interface that must be stable across distinct generations of the static config and distinct low-level hardware setups.
- Backups and restoration of state, or even synchronization of state, where possible, could be understood by this system. Restoring from backups could be as straightforward as just installing from the same flake again, and having the system automatically realize at runtime that it’s missing some needed state, and go fetch it from the backup.