My painpoints with flakes

I wanted to play with flakes, and therefore kumped into them, roughly after summer holiday.

So I’m using flakes since ~3 months now.

I really like the evaluation cache, and its well defined interface about inputs and outputs. Though there are also some things, that are troubling me and I want to talk about them in a broader context than the chat, perhaps I’m still misunderstanding things about it, perhaps my view makes people think differently and causes a change? Or perhaps its just that someone finds my thoughts and considerations a nice read.

Despite my thoughts, I will continue to use flakes for my personal stuff, but in their current form probably not for work (see below)

The global flake registry

It is there, and I can not do anything against it.
flake:nixpkgs points at github:nixos/nixpkgs (master branch), while flake:nixpkgs/release-19.09 points to a last years fork of github:nixos/nixpkgs/release-19.09 by edolstra.

Also there are other entries in the global registry, that seem to be random personal projects of edolstra (which is fine, though why are those in, but not eg. github:numtide/flake-utils).

Also flake:nixpkgs pointing at master makes it barely usable and running the same nix shell nixpkgs#hello -c hello results in redownloading and re-evaluation of the attributes quite often, as the evaluation cache can’t be re-used with the new commit. I think individual branches wouldn’t advance that fast and therefore be able to re-use evaluation cache more often.

Overwriting flake:nixpkgs with a local registry entry (eg. github:nixos/nixpkgs/nixos-20.09) seems to cause much less downloads and re-evaluations. And after using nix flake pin there haven’t been any downloads anymore, though then the local registry entry points to github:nixos/nixpkgs/$sha1, the information to which branch it pointed is lost. To “advance” the registry entry I have to actually know where it pointed to before and overwrite, and then pin again.

Also once I have a local entry and create a flake it will create a lock file according to my local entry, and when contributors with a different local entry (or no local entry at all) would update or recreate the lockfile, all of a sudden the lockfile would point not only to a different commit, but a different branch then before!

Forced VCS

I love git, I like to use flakes, especially as they make pinning easy.

I’d like to use flakes for defining my dev-env at work, though I can not commit any nix file to the repo, I have to sideload them.

So I do not have any choice than to not use flakes, but instead manually fiddle around with rev-shas in builtins.fetchGit and builtins.fetchTarball as well as the slowness of IFDs (which luckily aren’t that bad when it is just a nixpkgs commit within a shell.nix).

I’m missing some “ad-hoc” mode, that enables me to use version controlled flakes at the root, of course it is impossible to use those as inputs in any other kind of flake (even other ad-hoc ones).

Manipulation of registry

For old NIX_PATH/channel style nix files, I was able to pin channels prior to the nixos-rebuild and home-manager through nix-shell magic in the NIX_PATH and running the mentioned commands within the pinned environment. At the same time I used either model to set the NIX_ENV environment variable for the system/user to roughly match the inputs by doing environment = { NIX_PATH = "nixpkgs=${<nixpkgs>}"; } (pseudo code), I’m missing something equivalent for flakes to be able to alter system/user wide pins through my configuration.

Also I am missing a way to delete global entries, as I do not consider them important for me.

Local registry

I have not yet found any hint about whether “local” entries are per user or systemwide.

6 Likes

A doubt I have is if I use flakes to manage my system and use nixos-unstable as input, when I tell flakes to update it will update nixos-unstable to the master of the github repo or it will update to the last version that succeeded to build correctly in Hydra?

As I understand the process, the branches represent what you would get from the channel URLs. As the branch will only be updated after it’s corresponding “pre” branch has been successfully evaluated and tested.

For the unstable channels that’s staging and staging-next, AFAIK, for the numbered stable those are prefixed with “release” rather than nixos/nixpkgs.

But again, this is only from my understanding.

I saw that nixos-unstable branch head is in the same commit of nixos-unstable channel.

Today I realised another thing:

nix flake check does not accept an attribute

If one has more than a single check in a flake, it seems not to be possible to run it selectively.

In a playground project of mine I have basically 3 checks (pseudoecode and simplified):

checks = {
  nix-format = runTest "nixpkgs-fmt --check {,**/}*.nix";
  rust-format = runTest "cargo fmt -- --check";
  build = self.packages.${system}.theProject;
};

Now I can run them using nix flake check, which will do all the checks. Though the build check is quite costly, therefore I’d prefer to only run it every once in a while, while the other 2 could be part of my commit hooks, though:

nix flake check .#nix-format
error: --- Error ----------------------------------------------------------------------------------------------------------------- nix
unexpected fragment 'nix-format' in flake reference '.#nix-format'
1 Like

You can nix build a check

nix build .#checks.x86_64-linux.nix-format

but it would be more convenient, if like nix build with packages, it could handle the arch for you.

Thanks, that helps indeed, though it would be nice to have it in the interface of nix flake check.

1 Like

YIL (the Y stands for “yesterday”), that one can indeed pin entries in the registry declaratively!

  1. I had to make sure that the module system will see the inputs (not sure if this way was idiomatic though, but it works for now: https://github.com/NobbZ/nixos-config/commit/7e4d3747b28ca90830267dcd2dfac2dfda12ad2a
  2. set nix.registry.<name>.flake to whatever one wants to pin for that registry entry: https://github.com/NobbZ/nixos-config/commit/bda3cef8f4b98413a0aa892fd33cfb6489f7db6e
1 Like

I use flakes to build my system, home-manager and now some dev-env.

I agree with all the comments on global registry:

  • strange content (projects and branches)
  • no way to remove unused stuff

Indeed I use the nix.registry.<name>.flake option to pin nixpkgs so that all references for my system and users is using the same version.

Basically my organization is:

  • One configuration flake containing:
    • All inputs to upstream branches (nixpkgs, home-manager, …) used to pin everything with the flake.lock
    • A custom package set for local stuff
    • An overlay to help combining all the packages
    • Some nixosModules.* for my custom services and to generalize my configs
    • Some nixosConfigurations.* to handle my system configurations
    • Some homeModules.* to generalize my home configs (following the nixos* pattern)
    • Some homeConfigurations.* to handle home-manager configs

Part of my packages a flake-mgr script to:

  • Update the lock file of all flake inputs
  • Build/Switch homeConfigurations
  • Build/Test/Boot/Switch nixosConfigurations

This handles my system and user persistent configurations.

For my dev-env, I write another flake depending on the first to see my overlay and pin all upstream flakes. I simply defines a devShell.

BTW: My flakes are currently not in a git repository, and it still works.

Pro Tip: when defining a dev-env, dont put your flake.nix file at the root … otherwise all the folder and subfolders will be pushed to the nix-store and caching will be very bad. I use:

  • Main Project
    • .envrc Direnv config using ./nix/flake.nix as a source
    • nix
      • git
      • flake.nix Used to pin deps, and provide CI context
      • flake.lock
    • app1 Git clone of a backend app
      • .git
      • nix Specific env for this app
        • default.nix
      • Cargo.toml
      • src
    • app2 Git clone of a frontend app
      • .git
      • nix Specific env for this app
        • default.nix
      • src
      • package.json
      • package.lock
1 Like

I can’t reproduce…

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/unstable";

  outputs = { self, nixpkgs }:
    let
      pkgs = nixpkgs.packages.x86_64-linux;
    in
      {
        devShell.x86_64-linux = pkgs.mkShell {
          buildInputs = [ pkgs.go_1_14 pkgs.nixpkgs-fmt pkgs.lefthook pkgs.docker-compose pkgs.entr ];
        };
      };
}

This is my .flake.nix at the company project:

nix develop
warning: unknown setting 'extra-sandbox-paths'
error: --- Error ---------------------------------------------------------------------------------------------------------------------------------------------------------------- nix
source tree referenced by 'git+file:///home/nobbz/path/to/repo?ref=branch-name&rev=0e69a288c41b89dec8566b6d109f6c7a9e3fabaf' does not contain a '/flake.nix' file

If though I git add the flake.nix, it will ask for the flake.lock.

So maybe you can use flakes for things that aren’t in git yet, but I’m working with things that are already in git, and I’d prefer to have the dev shell defined through flakes, to not have to switch between niv and flake or even manually updating SHAs in the fetchGitHub or similar call depending on the flake…

Ah yes, if it is inside a git repo, it is forcing the non dirty behavior

1 Like