Flakey-profile: declarative nix profiles as flakes

Currently, to declaratively manage just user-level packages (and nothing else about the machine), the options are:

  • home-manager. It’s quite large and wants to do a lot of things; it winds up eating some seconds of bonus Nix evaluation for most tasks.
  • nix-env[1] can do exactly the desired thing, but then locking the nixpkgs version is kind of annoying, since one would need flake-compat.
  • home-mangler. A fair amount of Rust code to wrangle nix profile into installing declared sets of packages.
  • nix-profile-declarative. This builds a nix-env format manifest by reusing the nix internals; this is really cool and I only found it while writing this post. It could likely be adapted to use flakes. The manifest is not strictly required though; NixOS doesn’t have one.

None of these were not quite what I was envisioning: simply build the profile with Nix flakes and then switch to it. I sought out to build exactly that.

I realized that NixOS/home-manager is using exactly the machinery to necessary to achieve this in nixos-rebuild. The machinery in question is nix-env --set, which is used for generations in NixOS, and allows rollbacks like standard profiles.

Another realization is the cute trick of using more levels under packages.*, as done with nix2container’s nix run .#dockerImage.copyTo <...> and similar.

Combining these ideas together, we can make declarative flake profiles in 14 lines of Nix code by leveraging buildEnv and adding switch/rollback shell scripts as attributes on that derivation:

# Creates a profile. Arguments are forwarded to pkgs.buildEnv.
#
# The following attributes are available:
# - .switch: switches to the profile
# - .rollback: rolls back to the previous version of the profile
args @ { pkgs, name ? "flake-profile", ... }:
let
  args' = builtins.removeAttrs args [ "pkgs" ];
  env = pkgs.buildEnv (args' // {
    inherit name;
  });
in
env // {
  switch = pkgs.writeShellScriptBin "switch" ''
    nix-env --set ${env} "$@"
  '';
  rollback = pkgs.writeShellScriptBin "rollback" ''
    nix-env --rollback "$@"
  '';
}

To make a declarative profile, make a flake:

{
  description = "Basic usage of flakey-profile";

  inputs = {
    flakey-profile.url = "github:lf-/flakey-profile";
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils, flakey-profile }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
        };
      in
      {
        # Any extra arguments to mkProfile are forwarded directly to pkgs.buildEnv.
        #
        # Usage:
        # Switch to this flake:
        #   nix run .#profile.switch
        # Revert a profile change:
        #   nix run .#profile.rollback
        # Build, without switching:
        #   nix build .#profile
        packages.profile = flakey-profile.lib.mkProfile {
          inherit pkgs;
          paths = with pkgs; [
            hello # put more packages here
          ];
        };
      });
}

and then you can switch to the profile with: nix run .#profile.switch, and rollback with nix run .#profile.rollback.

The code is here as CC0: GitHub - lf-/flakey-profile: Declarative profiles with nix flakes


Other related posts:


[1]: nix-env --log-format bar-with-logs --install --remove-all --file ./env.nix

9 Likes

That is a very interesting approach!

Why does this work, exactly? What is the value of "$@" here and why?

A few things I’m noticing after looking at the readme (and not trying it out at all):

let
  pkgs = import nixpkgs {
    inherit system;
  };
in

You might want to replace that with pkgs = nixpkgs.legacyPackages.${system};, see 1000 instances of nixpkgs.

Update package versions

Would it be possible to make that a builtin script as well? nix run .#profile.upgrade would very nice in terms of UX.

Non-goals

For this as well, would it be possible to implement nix run .#profile.list? I know it’s not technically required, but when you have a few different profiles, especially when composing them from multiple lists of packages (CLI only, GUI tools, debugging utilities, etc.), it would be quite useful as a debugging tool.

We are in favour of the removal of nix profile from the Nix codebase in lieu of stabilization.

Would you say then that nix-env should stay forever? I feel like nix profile has to replace it at some point as the main interface for managing profiles, and for flakey-profile to work you need some interface like that. I guess you could re-implement it pretty easily, it’s just symlinks after all, but as that functionality is such a basic part of a lot of nix-related tooling, that doesn’t seem feasible.

  • Do similar things to home-manager for services or other things. This is just a replacement for nix-env and nix profile and nothing more.

Very reasonable, having a defined scope is a good idea.

To use it, see the template in templates/default, or included below, or run:

nix flake init -t github:lf-/flakey-profile#templates.default

nix flake init -t github:lf-/flakey-profile is sufficient here :slight_smile:


Overall, I think this is a cool approach and definitely has its merits. What I would be interested in is how much of the package-management/profiles docs could be removed/replaced with this?

Could we make nix profile just be a codification of the “numbered symlink” paradigm used to implement profile, and handle everything else within something like flakey-profile?

2 Likes

I’m quite intrigued by all this, because it proves a claim I was making repeatedly in the past: much of the stuff Nix does in a very cumbersome way can be done quite elegantly on top of it instead.

I like the general direction of this, but it needs more thought – a bit or a lot depending on where you want to take it. @ericson2314 described the “numbered symlink” paradigm as a rudimentary version control system. It’s nice and simple at first glance, but it could be too simple and is in fact still broken: rollback to parent · Issue #311 · NixOS/nix · GitHub

The implication of the linked issue is that profile versions need branching, and then “numbered symlinks” are not expressive enough. I was briefly contemplating underpinning generation management with proper version control like Git. That could be anything between a great way to do one thing well and complete overkill. Letting end users do their own version control by default may be too much to ask, and if you bork your profile there may be no tools left to do that. So probably Nix has to have those batteries included, it may be a defining feature of built-in profiles.

What I’m still uneasy about are profiles themselves. They probably don’t need to be as complicated as they are. Over the course of the year, @ericson2314, @roberth, @tomberek, and me discussed ideas such as to let derivations declared in Nixpkgs handle the package and environment management, where facilities for that – including a schema for where to put what kind of thing in a package attribute set – could be centralised and augmented by a dedicated CLI maintained as part of Nixpkgs. Then all Nix would have to provide is something along the lines of nix run [<qualifier>]<name> and let the /bin/<name>, which is in the store object this resolves to, handle the rest. Maybe this is an avenue for simplifying the interface Nix offers for profile management, thus simplifying Nix itself. Maybe profile generations could simply link to an externally defined store object directly?

Related:

4 Likes

Why does this work, exactly? What is the value of "$@" here and why?

It’s interpreted by the shell, not Nix, and it just dumps the args to the shell script verbatim into the args of Nix.

You might want to replace that with pkgs = nixpkgs.legacyPackages.${system};

Fair point, and that’s not how I use flakes: I don’t much like the composition mechanisms of flakes so I mostly ignore them and just use overlays instead, but I definitely understand the idea of keeping every flake’s own internal package set tampering to itself.

For this as well, would it be possible to implement nix run .#profile.list?

I don’t know why you would want multiple profiles (and this is definitely not something this tool has any particular intentional support for (you could do nix run .#profile.switch -- --profile blah, probably)), but: do you mean list the packages in the profile? I guess that might be easier to do directly than by generating a manifest, but at some level you might just want to generate a manifest there.

Would you say then that nix-env should stay forever?

Yes, since we have an indefinite backwards compatibility guarantee for it already. I think nix profile should be removed or severely reduced in scope in lieu of stabilization, since I don’t think the concept of a mutable profile is a good idea in general, but independently of that, I think that the exact ways of implementing it should be library features rather than language features.

Nix should own way fewer undocumented hardcoded schemas for data, and since we mostly can’t remove the ones we have, we should at least stop adding more of them. The flakes CLI as it is parses attributes in the locations that are expected in nixpkgs, nix-shell -p is deeply nixpkgs tied, and so on. I realize that the profile manifest is a slightly different category of object, but I would argue it is not that different.

Could we make nix profile just be a codification of the “numbered symlink” paradigm used to implement profile, and handle everything else within something like flakey-profile?

I think so! Or at least, some kind of versioning system as discussed by @fricklerhandwerk. And I think this is a much better scoping for flakes CLI stabilization, while leaving more space to figure stuff out with way less commitment and barrier to entry to having a go at new ideas for how a profile looks.

3 Likes

They probably don’t need to be as complicated as they are .

Incidentally the linked manifest for the nix profile version is of an obsolete schema. I need not say much more.

I like this!

But I think that there is still room for imperative installation of packages. Sometimes, I at least, just like to do it quick & dirty.

It is also useful in “installation by shell script” environments where you source a bash file that includes all installation instruction: There it is easy to add a flake to your profile (or could be with a better implementation) but not so much finding and editing a general nix file.

I’d like if nix profile install nixpkgs#something had just the semantics of:

  • Checking whether flake nixpkgs with attribute something is already in the profile.

    • If yes, upgrade and replace it.
    • If no, install it.
  • Also warn about duplicate binaries and shadowing any paths.

2 Likes

I know how "$@" works, but I was confused about how it fits into the script. I guess it’s just for passing extra arguments?

Different profiles for different machines. Not multiple profiles on the same machine. My point was that Nix itself allows factoring out duplication, and if you have different profiles, listing the profile contents can be a very useful tool.

Yes that’s what I meant. What do you mean it might be easier to do directly? The only thing I can imagine is nix-store -q --references /nix/store/3h5qgzq....-env, but the output of that is pretty barebones. Or is there a better way?

I do feel something that somewhat works like mutable profiles is a useful entrypoint for new users, and nix profile fills that niche. Though indeed, the limitations start to show quickly. It would be cool if declarative profiles could be made simple enough to be modified with an imperative tool, but whether that is implemented as a part of Nix or as a library built on top of Nix doesn’t matter too much.

What I do think is important is that a change like this does not degrade the OOTB UX of Nix. It must be easy to get set up with a few terminal commands.

I’m definitely in that boat as well, but I’m wondering how often you really need to nix profile install instead of just nix shell for the “quick and dirty” approach.

I’m not quite sure I agree there, and this seems to depend on the context of where that script is run. I can see that running nix profile install with a few packages in a Dockerfile is simpler than having to bundle a flake, but as soon as you have an actual script, I see no issue with also adding a flake and lock file and just installing that. Besides, this idea of implementing this functionality in pure Nix doesn’t really change much about the semantics, except that you can never install multiple environments into one profile.

But maybe I’m not quite understanding your usecase here?

Yeah the current behavior is a bit strange, and there is an issue for fixing it.

2 Likes

I just updated flakey-profile to:

  1. Provide a template of pinning nixpkgs without using any flakes at all (ironic)
  2. Add a profile.pin script that pins nixpkgs or whatever else in flake registries and NIX_PATH, which means I can actually completely throw away the use of channels on my non-NixOS computer, yay!!
2 Likes