Get-flake: builtins.getFlake without the restrictions

I recently realized that flakes in subdirectories do not have the ability to import the top level flake. This is a big problem for examples. If you put an example flake in a subdirectory, it is unable to directly access the project which it is an example of, within the flakes input system. So I created a simple tool to do just that:

It is just a slight modification of flake-compat which makes it more ergonomic for this particular use case. Here is an example of it being used to fix the hello world example in cargo2nix:

3 Likes

It has come to my attention that Nix 2.6 actually does allow paths of this kind. However, get-flake still has the benefit of not adding an entry to your flake.lock. If you add the parent flake to the inputs, every time it changes, you need to manually update all the lockfiles.

1 Like

Can you clarify what you mean or what workflow is currently awkward? It would be preferable to refine the fetchers/builtins if possible in Nix itself to avoid the need for a wrapper to re-parse and manipulate the flake.lock. I’m glad you made this as it helps explore the design and consequences, though ideally it wouldn’t have to exist.

1 Like

The work flow is perfectly exemplified by cargo2nix. It is a project for helping you build rust packages, and it has an examples directory with some examples to show you how to use it. Ideally, the examples in a project like this should be using the current code of the project. If you import that code with a flake, it will be added to the flake.lock, and will become invalid as soon as you make a code change. It will then have to be updated manually via nix flake lock --update-input ... for each example. I think the same can be said about giving tests their own flake, but I am not super familiar with testing in nix, so I’m not sure how big of an issue that is.

My confusion for it being an example is that something like:

let cargo2nix = get-flake ../..; in

will only work while the example is in the context of the parent flake. As soon as someone tries to copy it into a fresh project, bad things happen.

separate question

For this case, what about not committing the example’s flake.lock to the repo? And point it at the upstream location of the parent flake so it can be copy-pasta’d? (apologizes if this seems like avoiding/dodging the problem). Last time I played with lock-less flakes it had some oddities/rough edges if using them directly, but it’s a possibility.

That is true, it wouldn’t work if you just copy-pasted and didn’t understand what was going on.

I think this is a good idea in theory, it would keep the example true to exactly what the project would be like if you copied the flake code. However, I don’t want running the examples to make my working tree dirty (stuff showing up on git status). You may think there’s a simple solution to that - put flake.lock on a .gitingore in the example directory, but nix build will actually cause git to track the flake.lock despite it being ignored.

We’ve run into a similar issue where we have a flake at the top level of a monorepo setup where each subdirectory holds a separate program that can be built individually using the top-level flake. Using https://github.com/arrterian/nix-env-selector for vscode to hack on individual projects requires a shell.nix that has to reach back to the top level flake.

It will create a flake.lock. But git won’t track it automatically, or perhaps I’m not understanding? You can also --no-write-lock-file or --recreate-lock-file if you don’t want one created at all if you want to ensure it is never stale.

$ ls
.git flake.lock  flake.nix  .gitignore
$ cat .gitignore
flake.lock
$ git status
...
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	.gitignore
	flake.nix
...
$ git add .
$ nix build
$ git status
...
Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
	new file:   .gitignore
	new file:   flake.nix

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	new file:   flake.lock

I understand now, thanks. This is caused by:

When the flake is interpreted as a git repo, it will do this git -C $PWD add --force --intent-to-add -- flake.lock.

Options:

  1. remove that “–force”. would also need to add logic to handle this gracefully, and it may break other things
  2. force non-git semantics: nix build path:.
  3. do nothing

My vote would be 1. I don’t think we should be overriding what people put in their .gitignores.

Just had a follow-on discussion:

Summary is that the benefits of get-flake might be attained with only Nix by making Nix aware of “partial locks” or that segments of a subflake’s lock file come from somewhere else. This breaks an invariant that a flake.lock is self contained, but that might be acceptable if we retain the idea that the collection of flake.lock’s in a source tree only point to each other and are as a group self-contained.

Relevant PR: https://github.com/NixOS/nix/pull/5437

1 Like

Another use case that has popped up is fixed lockfile explosion when packages depend on each other. You can instead set flake = false; and use get-flake to import the flake.

Hosted by Flying Circus.