How to override `let` variables?

Context: Upon discovering that arch-install-scripts is in nixpkgs (hooray! most of my machine still run Arch at this point), I’ve decided to replace my somewhat-janky Arch-based rescue USB with a custom NixOS rescue image, complete with a lot of the convenience tools I love like ripgrep, fd, neovim, and tools for working with zfs, btrfs, in addition to the usual suspects for a rescue drive. It seems like the NixOS install image is a perfect candidate for a starting place, but I’d like to tweak a few things.

It seems like I’m frequently ending up in this situation of finding an existing bit of code that does almost what I’d want, which in an imperative context would be really easy to sed -i s/foo/bar/, but in nix I struggle to make this change. It’s simple to override a function’s inputs, but what I usually end up wanting to do is override one of the variables defined in a letin block.

As a very simple example, I’d like to take the static set timeout=10* in the EFI section of iso-image.nix (part of efiDir, which is part of a letin) and decrease it for snappier boot time. I’d also like to change some of the names and titles of boot targets, but we’ll stick with this timeout for demonstration purposes.

One obvious / simple approach is to just copy the whole piece of iso-image.nix, modify, and import that instead. The obvious disadvantage here would be potentially missing out on updates over time, or becoming out-of-sync with the rest of nixpkgs. It also seems a bit heavy handed compared to sed 's/\(timeout=\)[0-9]\+/\15/'.

I thought it might be more “the nix way” to split out the efi part of isoImage.config.isoImage.contents, which I think I could then could use in mkDerivation and put a modified version in $out and use that instead of iso-image.nix, or perhaps as some kind of override:

{ modulesPath, config, lib, pkgs, ... }: {
  # currently just pulls out `efi` then puts it back
  config.isoImage.contents =
    let
      isoImage = import "${modulesPath}/installer/cd-dvd/iso-image.nix" { inherit config lib pkgs; };
      contents = isoImage.config.isoImage.contents;
      efi = lib.findSingle (item: (builtins.match "/EFI" item.target != null)) (abort "no EFI match") (abort "multiple EFI matches") contents;
      rest = lib.remove efi contents;
    in
    rest ++ [ efi ];
}

However, I get the feeling that there’s probably an even better or more “nix-like” approach for this type of minor tweak that I’m missing.

  • If you wanted to make an image that for the sake of argument was otherwise identical to the default iso-image.nix, but with set timeout=2 for EFI, what approach would you take?
  • If you also wanted a few other minor changes (modify titles, perhaps remove some boot targets) would you be likely to take the same approach?
  • Given the general problem of “I don’t want to reinvent the wheel, I hope not to have to modify a bunch of the dependencies or dependents, I just want a minor change to a bit of existing nix code” (perhaps tuning a setting on a config file that for whatever reason isn’t exposed as an input) should I be reading more about overlays, mkDerivation / overrideDerivation + patches, perhaps makeWrapper, something else entirely, or is it just so situation-dependent that there’s no best “try this first” approach?

Thanks for any thoughts and discussion. After a few false starts with nix over the last few years, I think it’s finally sticking.

* BTW, ignoring the seconds vs decisecond discrepancy, does anyone know why this EFI timeout is a static 10 instead of using the same value used to configure the syslinux timeout? Do you think a PR addressing this would be welcome?

1 Like

This looks to me like just a simple oversight in iso-image.nix. There is a boot.loader.timeout option that should be used here but it’s not. I would just make a PR to interpolate that in as ${toString config.boot.loader.timeout}.

That’s what I figured, nixos/installer/cd-dvd/iso-image: honor EFI boot timeout by n8henrie · Pull Request #204669 · NixOS/nixpkgs · GitHub

Any thoughts on the rest of the Q? For example, if this were not a change that were suitable for the general nixpkgs audience?

There is a tradeoff between the depth of customization and the time to evaluate and build, so I think we do whatever is practical in terms of putting things in static let … in blocks or as arguments to function. I don’t know of a rule but if it had an exception it might be booting. If you want something specific then build your boot partition from scratch, I’ve done it before and fortunately it’s not as difficult as the EFI cabal of idiots would like to make it.

2 Likes

In tinkering with this more, I was trying to use pkgs.runCommand to write an importable module, but I keep running into a recursion error.

For example, starting from this:

  imports = [
    "${modulesPath}/profiles/all-hardware.nix"
    "${modulesPath}/profiles/base.nix"
    "${modulesPath}/installer/cd-dvd/iso-image.nix"
];

I can do this:

  imports = [
    "${modulesPath}/profiles/all-hardware.nix"
    "${modulesPath}/profiles/base.nix"
    ./iso-image-foo.nix
];

where iso-image-foo.nix is simply:

{ modulesPath, ... }: {
  imports = [
    "${modulesPath}/installer/cd-dvd/iso-image.nix"
  ];
}

I thought this would basically be the same as:

let
  isoimage = pkgs.runCommand "isoimagerescue.nix" { } ''
    cat <<'EOF' > $out
      {modulesPath, ...}: { imports = [ "''${modulesPath}/installer/cd-dvd/iso-image.nix" ] ; }
    EOF
  '';
in
{
  imports = [
    "${modulesPath}/profiles/all-hardware.nix"
    "${modulesPath}/profiles/base.nix"
    "${isoimage}"
  ];

but I get this infinite recursion error:

warning: Git tree '/home/n8henrie/git/nixos-rescue' is dirty
error: infinite recursion encountered

       at /nix/store/mdhc10773i0mydcl77ss4r1hlcx08lvs-source/lib/modules.nix:467:28:

          466|         builtins.addErrorContext (context name)
          467|           (args.${name} or config._module.args.${name})
             |                            ^
          468|       ) (lib.functionArgs f);
(use '--show-trace' to show detailed location information)

Why doesn’t the runCommand version work? (relevance: as an experiment, I was hoping to try let stuff = builtins.readFile "${original}"; in pks.runCommand {} ''echo -n ${stuff} | sed 's'/foo/bar/g' > $out ''; and then add that to imports).

Looks like the answer may be here: https://discourse.nixos.org/t/infinite-recursion-on-optional-import/8892/5

Namely, imports = [ foo ] is desugared to imports = [ config._module.args.foo ], which means that config results in a circular dependency. I think.

1 Like

As one option for my initial question, it looks like I can just patch nixpkgs itself, using an example like this (which I’m already using to patch nix-darwin like so).

It’s building now, I’ll report back if it worked.

{
  description = "NixOS-based rescue drive";
  inputs.nixpkgs.url = "nixpkgs/nixos-unstable";
  outputs = { self, nixpkgs }:
    let
      system = "x86_64-linux";
      nixos =
        let src = (import nixpkgs { inherit system; }).applyPatches {
          name = "nixos-rescue-patches";
          src = nixpkgs;
          patches = [ ./iso-image.patch ];
        };
        in
        nixpkgs.lib.fix (self: (import "${src}/flake.nix").outputs { inherit self; });
    in
    {
      packages.x86_64-linux = rec {
        default = rescue.config.system.build.isoImage;
        rescue = nixos.lib.nixosSystem {
          inherit system;
          modules = [
            ./configuration.nix
          ];
        };
      };
    };
}

The patching approach worked fine, though it’s not quite as simple as I might have hoped.

I eventually turned it into a function that I import from a utils directory*:

inputs: {
  name,
  system,
  url,
  hash,
  src ? inputs.nixpkgs-stable,
}: let
  pkgs = import src {inherit system;};
  nixpkgs-patched = pkgs.applyPatches {
    inherit name src;
    patches = pkgs.fetchpatch {
      inherit url hash;
    };
  };
in
  (import "${nixpkgs-patched}/flake.nix").outputs {inherit (inputs) self;}

and can call from my main flake as:

patchNixpkgs = import utils/patchNixpkgs.nix inputs;  
nixpkgs = patchNixpkgs {
    inherit system;
    name = "fix-docker-compose";
    url = "https://patch-diff.githubusercontent.com/raw/NixOS/nixpkgs/pull/245782.patch";
    hash = "sha256-KFY/AlZEXROJJzDJz/DghRFtNkjRgdSd/eO8hkAKaE8=";
};

(adding .patch to the end of a PR on GitHub will redirect to that handy patch URL)

Alternatively, my problem with the infinite recursion can be worked around by not putting the runCommand in a module, since the module receives config as an argument, and we would be modifying config by setting values in the module (which is silently desugared to config = { module contents } behind the scenes).

For example, this shows the basic idea:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/4c5307057ad84440363b873658bb86e4c4d38677";
  outputs = {
    self,
    nixpkgs,
  }: let
    system = "x86_64-linux";
    pkgs = import nixpkgs {inherit system;};
    isoimage = pkgs.runCommand "isoimagerescue.nix" {} ''
      cp ${nixpkgs + "/nixos/modules/installer/cd-dvd/iso-image.nix"} $out
      sed -i 's/timeout=10/timeout=99/' $out
    '';
  in {
    nixosConfigurations."foo" = nixpkgs.lib.nixosSystem {
      inherit system;
      modules = [
        (nixpkgs + "/nixos/modules/profiles/base.nix")
        {isoImage.makeEfiBootable = true;}
        "${isoimage}"
      ];
    };
  };
}

Now, when one runs:

$ nix build .#nixosConfigurations.foo.config.system.build.isoImage

you get an error related to the relative path referencing iso9660-image.nix, which makes sense because that path will no longer be found relative to the module in question.

However, you can find the path to our modified module in the error output, which indeed shows that our sed modification worked:

$ nix build .#nixosConfigurations.foo.config.system.build.isoImage --show-trace 2>&1 | grep -F isoimagerescue
       ... while evaluating definitions from `/nix/store/6hj5bxb93plfcm4gdzh2pyzi39j3iv5j-isoimagerescue.nix':
         at /nix/store/6hj5bxb93plfcm4gdzh2pyzi39j3iv5j-isoimagerescue.nix:779:29:
$ grep -F 'timeout=' /nix/store/6hj5bxb93plfcm4gdzh2pyzi39j3iv5j-isoimagerescue.nix
    set timeout=99

So now, if we want to feel really gross, we can use sed to patch the relative paths, leveraging the module’s existing reference to pkgs to point towards the appropriate place in the nix store:

{
  inputs.nixpkgs.url = "github:nixos/nixpkgs/4c5307057ad84440363b873658bb86e4c4d38677";
  outputs = {
    self,
    nixpkgs,
  }: let
    system = "x86_64-linux";
    pkgs = import nixpkgs {inherit system;};
    isoimage = pkgs.runCommand "isoimagerescue.nix" {} ''
      cp ${nixpkgs + "/nixos/modules/installer/cd-dvd/iso-image.nix"} $out
      sed -i $out \
        -e 's/timeout=10/timeout=99/' \
        -e 's#\(pkgs.callPackage \)../../..\([^ ]\+\)#\1(pkgs.path + "/nixos\2")#g'
    '';
  in {
    nixosConfigurations."foo" = nixpkgs.lib.nixosSystem {
      inherit system;
      modules = [
        (nixpkgs + "/nixos/modules/profiles/base.nix")
        {isoImage.makeEfiBootable = true;}
        "${isoimage}"
      ];
    };
  };
}
$ nix build .#nixosConfigurations.foo.config.system.build.isoImage
trace: warning: system.stateVersion is not set, defaulting to 23.05. Read why this matters on https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion.
$ loopdev=$(sudo losetup --find --partscan --show ./result/iso/nixos.iso)
$ mkdir -p /tmp/loop
$ sudo mount "$loopdev" /tmp/loop
mount: /tmp/loop: WARNING: source write-protected, mounted read-only.
$ grep timeout /tmp/loop/EFI/boot/grub.cfg
set timeout=99

This approach is probably not recommended in any modern textbook :laughing:, I think the patch technique above is probably the way to go!

* Not sure if this should use lib.fix like in Support flake references to patches · Issue #3920 · NixOS/nix · GitHub

1 Like