Flakelight: A new modular flake framework

accelbread · August 31, 2023, 12:09am

Hi, I have been working on a framework for making writing flakes simpler with less boilerplate.

I initially developed it for use in my projects, as existing flake libraries did not meet my expectations, though I intend this to be generally usable.

I would appreciate any feedback, especially on where documentation is lacking.

It supports setting the standard flake outputs as module config, handles generating package sets and per-system attributes, and supports loading nix files from a directory automatically. Packages are set as functions which will be called with callPackage, just like packages are written in nixpkgs.

Why another flake framework?

I’d started out with using flake-utils for its eachSystem function. Pulling in flake-utils everywhere ended up more overhead than necessary, so replaced it with a two-line simpler version of eachSystem I copied into each of my flakes. This was good for a while, but I found that the boilerplate in my flakes was kind of high, and there was a lot of common code to set up things like devShells and checks.

Thus, I ended up giving flake-parts a try since it seemed like it would fit my needs. The modular part seemed cool, but lacked what I wanted in the end, though there were a lot of parts I liked.

Reasons why I did not end up using flake-parts:

Added significant complexity on top of base flakes, especially with how it handled its perSystem stuff using separate modules, and which one needed to understand to use it in a way that would fit my needs (did not seem fixable without breaking compatibility)
Its overlays outputs just add the packages built using flake’s dependencies to the overlay, instead of building those packages with dependencies from the package set the overlay was applied to; while that type of overlay is sometimes useful, its definitely not what I would expect or want from the default
Setting overlays on the package set required setting _module.args.pkgs which was boilerplate
It also output empty attributes which at the time showed up in nix flake show (nix flake show ignores them now, but I’d still rather not have empty attributes when I’m exploring the flake with eval or repl…)
Lacked modules that would populate flakes automatically (fixable by writing modules)

Now, to clarify, I’m not saying I think flake-parts is bad, but its not for me, so I decided to make my own thing. Especially, since if I ended up contributing to flake-parts, it would be best to have a proof-of-concept of ideas I wanted to implement. I ended up with an interface that deviated from flake-parts in compatibility breaking ways, so decided to just stick with mine.

For a more complex usage example, can see my config flake, which uses most flake outputs and where everything is loaded automatically from the nix directory: GitHub - accelbread/config-flake: Personal nix flake [MIRROR] · GitHub

matthewcroughan · September 16, 2023, 1:22pm

In what ways is this different from flake.parts? I’ve read your explanation, but maybe it’d be easier to bullet point what is different, rather than what was disliked about flake parts, since it does have a lot of similiarities.

accelbread · September 20, 2023, 3:45am

Here are some of the differences:

Per-system attributes

Flake-parts has perSystem modules which contain per-system options, while Flakelight has regular options which can be functions that are passed the set of packages.

For example:

Flake-parts:

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixpkgs-unstable";
    flake-parts.url = "github:hercules-ci/flake-parts";
  };
  outputs = inputs@{ flake-parts, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      systems = [ "x86_64-linux" "aarch64-linux" ];
      perSystem = { pkgs, ... }: {
        apps.default.program = "${pkgs.hello}/bin/hello";
      };
    };
}

Flakelight:

{
  inputs.flakelight.url = "github:accelbread/flakelight";
  outputs = { flakelight, ... }:
    flakelight ./. {
      app = pkgs: "${pkgs.hello}/bin/hello";
    };
}

Shortcuts for default

There are options to set default outputs; in the above example, app sets apps.default. This lowers boilerplate for flakes that just set default outputs. You can still set other app outputs or directly use apps.default.

Overlays actually use your deps

If a dependency flake using Flake-parts has a package A that depends on package B in its overlay, and a user applies that overlay to their nixpkgs, A will use B from the dependency flake’s nixpkgs. If that flake were using Flakelight, A would use B from the user’s nixpkgs, like how overlays usually work.

If a user actually wanted to use the flake’s packages with it’s dependencies, they can already do that with _: prev: flake.packages.${prev.system}.

Autoload files from ./nix

Flakelight options can be automatically read from files in ./nix. For example you could have files for ./nix/packages/a.nix and ./nix/packages/b.nix to define packages a and b, or just have a ./nix/packages.nix that defines both.

The directory to load from can also be changed.

Using overlays from other flakes

Flake-parts:

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixpkgs-unstable";
    flake-parts.url = "github:hercules-ci/flake-parts";
  };
  outputs = inputs@{ flake-parts, emacs-overlay, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      systems = [ "x86_64-linux" "aarch64-linux" ];
      perSystem = { pkgs, system, ... }: {
        _module.args.pkgs = import nixpkgs { 
            inherit system;
            overlays = [ emacs-overlay.overlays.default ]; };
        };
    };
}

Flakelight:

{
  inputs.flakelight.url = "github:accelbread/flakelight";
  outputs = { flakelight, emacs-overlay, ... }:
    flakelight ./. {
      withOverlays = [ emacs-overlay.overlays.default ];
    };
}

Inputs

In Flake-parts, inputs is a separate parameter. In Flakelight, it is a normal option. Also, defaults for all used dependencies are provided, so you can leave it unset.

Src

Flakelight takes an src argument that your modules can use to automatically set up packages and attributes. For example, Flakelight-rust derives flake outputs from src + /Cargo.toml. As another example, if your repo has a .editorconfig file, an editorconfig check will be added.

Packages are defined like in nixpkgs

{
  inputs.flakelight.url = "github:accelbread/flakelight";
  outputs = { flakelight, ... }:
    flakelight ./. {
      package = { stdenv, cmake, ninja }:
        stdenv.mkDerivation {
          pname = "pkg1";
          version = "0.0.1";
          src = ./.;
          nativeBuildInputs = [ cmake ninja ];
        };
    };
}

Alternatively using autoloaded files:

flake.nix:

{
  inputs.flakelight.url = "github:accelbread/flakelight";
  outputs = { flakelight, ... }: flakelight ./. { };
}

nix/package.nix:

{ stdenv, cmake, ninja, src }:
stdenv.mkDerivation {
  pname = "pkg1";
  version = "0.0.1";
  inherit src;
  nativeBuildInputs = [ cmake ninja ];
};

With Flake-parts, this would be:

{
  inputs = {
    nixpkgs.url = "nixpkgs/nixpkgs-unstable";
    flake-parts.url = "github:hercules-ci/flake-parts";
  };
  outputs = inputs@{ flake-parts, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      systems = [ "x86_64-linux" "aarch64-linux" ];
      perSystem = { pkgs, ... }: {
        packages.default =
          let
            inherit (pkgs) stdenv cmake ninja;
          in
          stdenv.mkDerivation {
            pname = "pkg1";
            version = "0.0.1";
            src = ./.;
            nativeBuildInputs = [ cmake ninja ];
          };
    };
  };
}

Flakelight’s src parameter also lets other files and modules easily access the flake’s root dir, which an external module could not in Flake-parts. With Flakelight, since packages are functions, it can also automatically generate a correct overlay.

No empty outputs

Flake-parts creates empty outputs; for example if you don’t have any packages, it will create a packages output that is an empty set. Flakelight does not do this.

Multi-language formatter

Flakelight has options to define formatters for different file types, that can be set by different modules. All configured formatters will be combined to provide a formatter.${system} that formats each configured file type with its corresponding formatter.

Designed to enable modules to do most of the work

For example, here is a flake for a Rust package:

{
  inputs = {
    flakelight.url = "github:accelbread/flakelight";
    flakelight-rust.url = "github:accelbread/flakelight-rust";
  };
  outputs = { flakelight, flakelight-rust, ... }: flakelight ./. {
    imports = [ flakelight-rust.flakelightModules.default ];
  };
}

It exports the following:

Per-system attributes for default systems (x86_64-linux and aarch64-linux)
packages.${system}.default attributes for each system
overlays.default providing an overlay with the package (built with the applied pkg set’s dependencies)
devShells.${system}.default that provides rust-analyzer, cargo, clippy, rustc, and rustfmt as well as sets RUST_SRC_PATH
checks.${system}.${check} attributes for build, test, clippy, and formatting checks
formatter.${system} will also format Rust files with rustfmt.

The above flake can also be written equivalently as:

{
  inputs.flakelight-rust.url = "github:accelbread/flakelight-rust";
  outputs = { flakelight-rust, ... }: flakelight-rust ./. { };
}

adrian-gierakowski · September 20, 2023, 9:38pm

Thanks for the comprehensive summary. Sounds enticing!

Are there any drawback to the solutions you employed which enabled the features which flake-parts doesn’t provide?

tgunnoe · September 20, 2023, 10:26pm

This is cool. the interface looks minimal and clean. Great work!

accelbread · September 20, 2023, 10:49pm

Thanks! I am not aware of any drawbacks.

Well, other than being incompatible with flake-parts.

accelbread · December 19, 2023, 6:05am

Flakelight is now part of nix-community!