How to compose/reuse nix flakes (do I need flake-parts and friends)?

Help
1

I’m building a flake.nix repo (called nullkomma), which I would like to reuse in a bunch of other repos (the “caller” repos). I might also let the caller repos add stuff on top.

It’s very important to me that the caller repos have absolutely minimal boilerplate. In my experience, having a bunch of copy/pasted (or git subtree merge) files causes the caller repos to quickly diverge from the DevOps standard way of doing things (here, nullkomma), requiring manual, painful upgrades.

So ideally, I’d like a flake which looks like this in the caller repos (pseudocode):

{
  description = "some-caller-repo";
  inputs = {
    nullkomma.url = "https://flakehub.com/f/dataheld/nullkomma/0.1.*";
  };
  outputs = {
    nullkomma,
    self
  } {
    nullkomma.do-it-the-normal-way
  }

In addition, I need to copy/paste in the .gitignore and .envrc (for nix-direnv). I can live with that.

However, using just flakes, my flake.nix is considerably longer, repetitive and abstraction-leaking (see below).

My question is: Do I need something like flake-parts or are there nix/flake-native ways of increasing reuse/composability that I’m missing?

In general, what’s the idiomatic/recommended way to achieve reusability & composability with nix flakes? Strong opinions welcome :hugs:.

Here's the `flake.nix` inside the **caller repo**, with comments on what feels repetitive to me.

For better legibility, I’ve cut out most of the stuff and only kept the formatter and check for formatting.

{
  description = "some-caller-repo";

  inputs = {
    # I get that I might need to declare nixpkgs for *this* (caller) flake...
    nixpkgs = {
      url = "https://flakehub.com/f/NixOS/nixpkgs";
      follows = "nullkomma/nixpkgs";
    };
     flake-utils = {
      url = "https://flakehub.com/f/numtide/flake-utils";
      follows = "nullkomma/flake-utils";
    };
    nullkomma.url = "https://flakehub.com/f/dataheld/nullkomma/0.1.*";
  };

  outputs = {
    flake-utils,
    nixpkgs,
    nullkomma,
    self,
    ...
  }: let
    universalOutputs = {
      schemas = nullkomma.schemas;
    };
    systemOutputs = flake-utils.lib.eachDefaultSystem (
      system: let
        pkgs = nixpkgs.legacyPackages.${system};
        # this is where it gets repetetive ...
        nullkommaDevShell = nullkomma.devShells.${system}.default;
        shellInputs = nullkommaDevShell.nativeBuildInputs or nullkommaDevShell.buildInputs or [];
      in {
        checks = {
          formatting = pkgs.stdenv.mkDerivation {
            # this is where it gets bad;
            # I'm building treefmt-nix upstream in nullkomma,
            # but could not think of an easier way to reduce it as a *check* here
            # since that will always require self/src,
            name = "check-format";
            buildInputs = [ pkgs.treefmt ] ++ shellInputs;
            phases = [ "checkPhase" "installPhase" ];
            # this seems bad to me because that `treefmt --ci` is the command
            # is a concern *internal* to nullkomma,
            # and should not leak into the caller repo's flake
            checkPhase = ''
              output=$(treefmt --ci)
            '';
            # all of this boilerplate already exists, presumably,
            # inside treefmt-nix, but I couldn't find a way to reuse it here
            # without just repeating all the treefmt-nix calls from nullkomma
            installPhase = ''
              mkdir -p $out
              echo "Test passed" > $out/result
            '';
          };
        };
        devShells.default = pkgs.mkShell {
          # that seems ok ... but why do I even need it in the first place?
          inputsFrom = [ nullkomma.devShells.${system}.default ];
        };
        formatter = nullkomma.formatter.${system};
      }
    );
  in
    universalOutputs // systemOutputs;
}

For completeness sake, this is the upstream nullkomma flake.nix:

{
  description = "nullkomma";

  inputs = {
    nixpkgs.url = "https://flakehub.com/f/NixOS/nixpkgs/0.2411.*";
    # keep-sorted start
    fh.url = "https://flakehub.com/f/DeterminateSystems/fh/0.1.*";
    flake-checker.url = "https://flakehub.com/f/DeterminateSystems/flake-checker/0.2.*";
    flake-iter.url = "https://flakehub.com/f/DeterminateSystems/flake-iter/0.1.*";
    flake-schemas.url = "https://flakehub.com/f/DeterminateSystems/flake-schemas/0.1.*";
    flake-utils.url = "https://flakehub.com/f/numtide/flake-utils/0.1.*";
    format.url = "path:./format";
    # keep-sorted end
  };

  outputs = {
    nixpkgs,
    # keep-sorted start
    fh,
    flake-checker,
    flake-iter,
    flake-schemas,
    flake-utils,
    format,
    # keep-sorted end
    self,
    ...
  }: let
    universalOutputs = {
      schemas = flake-schemas.schemas;
    };
    systemOutputs = flake-utils.lib.eachDefaultSystem (
      system: let
        pkgs = nixpkgs.legacyPackages.${system};
      in {
        checks = {
          format-check = format.packages.${system}.format-check self;
          flake-check = flake-checker.packages.${system}.default;
        };
        devShells.default = pkgs.mkShell {
          packages = [
            # keep-sorted start
            fh.packages.${system}.default
            flake-checker.packages.${system}.default
            flake-iter.packages.${system}.default
            format.packages.${system}.default
            pkgs.git
            pkgs.gnumake
            pkgs.nixd
            # keep-sorted end
          ];
        };
        formatter = format.packages.${system}.default;
      }
    );
  in
    universalOutputs // systemOutputs;
}
1 Like
2

I mean, nix is a functional programming language. Have you considered using… functions?

You can really easily just have a lib.checks output in your nullkomma flake, e.g. for the checks:

lib.checks = inputs: {
  formatting = pkgs.stdenv.mkDerivation {
    name = "check-format";
    buildInputs = [ pkgs.treefmt ] ++ inputs;
    phases = [ "checkPhase" "installPhase" ];
    checkPhase = ''
      output=$(treefmt --ci)
    '';
    installPhase = ''
      mkdir -p $out
      echo "Test passed" > $out/result
    '';
  };
};

Then in your caller:

checks = { 
  # Some local checks
} // (nullkomma.lib.checks shellInputs);

Apply same pattern wherever you need to pass some local context to something that is otherwise best defined in nullkomma, or want to replace a few lines of code with one where you can’t with normal flake outputs.

Flake-parts may make you happy since it basically is the above pattern but with some bells and whistles. Personally I’ve found it infuriatingly magic for something that is frankly quite simple if you can program, but I can see it being useful if you need to compose with third party code.

Personally, I’ve yet to find a use case for composing with third party code in the context of flakes, and I don’t think yours is one given you just have one library and don’t need to keep to any standards but your own. You might however appreciate the magic and pre-defined structure if you don’t care too much about the trade-off in transparency, and given the number of dependencies you have it might actually provide some value.

Tangential, since this is more about the general case than the specific implementation, and I trust that you’ve tried this, but:

Yeah, why do you need that? Can’t you just:

devShells.default = nullkomma.devShells.${system}.default;

As an aside, be aware that flakehub is controversial, it’s a proprietary thing and the promised open standardization hasn’t done anything resembling surfacing in the year since it was announced.

3 Likes
3

You can get away with standard nix constructs, but flake.parts makes composing much more convenient IMHO. Kinda comparable to attrset update vs nix modules merging values. I personally like flake.parts and use it quite a lot.

Shameless plug: my post on writing custom flake.parts modules.

3 Likes
4

:man_facepalming: thank you so much for taking the time to set me straight. Hard to explain why I didn’t think of the lib output …

1 Like
5

Thanks for pointing me to your post! This kind of worked example is super helpful.
I’m still doing baby steps in flake-parts, so it’ll be a while until I catch up to your level :slightly_smiling_face: