Ideas to fix flakes other than lazy trees?

Development Nix
1

I am new to nix and in my research so far, I have seen various comments on flake design issues. Some of these could perhaps be addressed iteratively but it seems like the copying of local file trees to the store is the biggest blocker?

Regarding this, I have read that lazy trees offer only a partial solution, and in any case it looks like they won’t be merged soon.

So are there any serious alternatives that could be worked on?

Ideas come to mind easily: containers, bind mounts, overlay filesystems, symlinks, or just skipping it and accepting the impurity risk; and then, a configurable choice over such options. But I am sure these aren’t new ideas and there have probably been good reasons to rule them out. Hence my question, as I’d like to know if there is any feasible alternative, and anyone who wants to work on it.

Thanks!

3 Likes
2

There’s only one way: slow deprecation and externalization, with a long backward compatibility period.

16 Likes
3

One eval purity issue Flakes solve is the issue that embedding toString ./. into a derivation can yield different results on different machines. Now “/home/myuser/src/myproject” becomes part of my derivation input, and on another machine, the path might be different.

Flake’s conclusion is that if the whole repo is copied into the /nix/store, and that we use Git as the data source, this path will be the same on two different machines.

Another approach would be to disallow embedding non-store paths into derivations, since they are a source of impurity.

Implementing this is fairly easy. It requires adding non-store paths to the string context. And then rejecting them at the point where the derivation calculates the inputs.

In terms of impact, this would be a breaking change. In my experience, we don’t lose much from reducing the language, and examples I found in the wild were usually mistakes.

Change this, and remove the flakes eval cache, and I believe lazy trees are not needed anymore (to be verified, I haven’t looked at this for a while now).

19 Likes
4

One example of something that would break with losing ./. is the case of setting nixos-config in nix.path to /home/user/some/folder/that/might/occasionally/be/moved.

On the other hand, if that folder gets moved it’s not like the older generations get updated to point at the right place, so it’s more a convenience thing than a strictly correct thing.

5

Yes in that case your config is impure. Which is fine but might require an --impure flag, or to pass that specific value as an argument to the eval.

1 Like
6

Your framing of flakes solves a problem that shouldn’t exist in the first place. toString ./. is still a bad practice (with or without flakes) due to rebuilds and unnecessary copies. Hence Working with local files — nix.dev documentation.

IMO the way to fix flakes is to kill off flakes entirely - they don’t bring us anything new.

5 Likes
7

Sometimes we have the reverse problem. Take the modulesPath argument to NixOS modules:

{ modulesPath, ... }:
{
  imports = [ "${modulesPath}/virtualization/qemu-guest.nix" ];
}

Imagine modulesPath is of type “path”. Because of the automatic path to string interpolation, the interpolated string would look like /nix/store/<hash>-modules/virtualization/qemu-guest.nix. This is is extra copying into the store that’s not needed.

That’s why modulesPath is first converted to a string using toString.

Now the interpolated string is /home/myuser/path/to/nixpkgs/nixos/modules/virtualization/qemu-guest.nix. It’s fine because the string doesn’t end up in a derivation, and so doesn’t affect the inputs.

These are subtle issues that I have seen many users encounter or trigger, without even realizing it happening.

If you re-read my quick and dirty proposal, you’ll see that it’s a language change that both fixes flakes, and also improve the language itself. Regardless of flakes, I still want my evaluations to be pure and reproducible.

11 Likes
8

Okay. The only aspect of them I’ve seen so far that I’d miss is

nix profile install nixpkgs/nixos-unstable#whatever

Please let’s keep a syntax like that in the rewrite :slight_smile:

1 Like
9

We already do have that without flakes, though.

nix profile install -f '<nixpkgs>' hello

Making diamond-syntax reproducible is a different endeavor, but doable.

And we have the pure-eval option for that.

It’s really unfortunate that everything gets credited to flakes, because while some of these improvements may have been done in service of flakes, they can stand on their own.

4 Likes
10

Nice! Although I did like the other “installables” syntax.

Say I’m on 25.05, can I pluck a package from unstable this way?

11

All of the below are non-reproducible obviously, but I think nix profile is not exactly built around reproducibility.

You can use the channel tarball directly:

nix profile install -f 'https://channels.nixos.org/nixos-unstable/nixexprs.tar.xz' hello

Or using the channel: syntax:

nix profile install -f 'channel:nixos-unstable' hello

Or add a new channel for nixos-unstable and use

nix profile install -f '<nixos-unstable>' hello

Though nix profile upgrade hello will not work as it’s currently reliant on flakes :frowning_face: IMO that could be improved.

EDIT: added shorthand command

1 Like
12

if you use a different mechanism for dependency management and use flakes as a thin wrapper, then you get:

  • open-source (not dependent on determinate nix for lazy trees, as the issue that manages is one of flake inputs, which you would no longer be using)
  • control over dependencies, rather than having to keep maintaining exploding recursive dependencies
  • no copying when you don’t want it to (can run your code without going thru the flake)
  • flake-powered result caching when you do want it (e.g. for running tests in CI)
4 Likes
13

I was on this train for so long but gave up; it seems like the convergence to Flakes feels inevitable.

I like the idea though of just adding --impure and restricting the language a little more (i.e. what @zimbatm said about paths like ./. to get us 90% there). Something like niv then gets you the remaining 10%.

Having contributed to the codebase slightly; I’m always in favor of less entropy and removing what you can from the system to the smallest amount necessary to keep it great.

(probably offloading the sandbox code would be worthwhile too… but then we start to get spiritually similar to Snix)

2 Likes