How to make a simple substitution when using `pkgs.writers`?

Help
1

Turning a simple non-nix python script into something nix-friendly can be pretty easy:

# flake.nix
{
  description = "A very basic python script";

  outputs = {
    self,
    nixpkgs,
  }: let
    pkgs = import nixpkgs {system = "aarch64-darwin";};
  in {
    packages.aarch64-darwin.default = pkgs.writers.writePython3Bin "say_hello" {} ./hello.py;
    apps.aarch64-darwin.default = {
      type = "app";
      program = "${self.outputs.packages.aarch64-darwin.default}/bin/say_hello";
    };
  };
}

# hello.py
print("hello, world")

Result:

$ nix run
hello world

However, I’m having trouble making the simplest substitution – without changing hello.py, what’s the best way to have it output hello there n8henrie?

At first, I thought something like this should work:

packages.aarch64-darwin.default =
      (
        pkgs.writers.writePython3Bin "say_hello" {} ./hello.py
      )
      .overrideAttrs (old: {
        patchPhase = "sed -i 's/world/there n8henrie/g' $out" + (old.patchPhase or "");
      });

I had anticipated an error (perhaps I need to use $src instead of $out) but I get no error, and no change in output. I guess this is because phases is undefined? Adding phases = ["patchPhase"]; changes nothing. So I guess this writer doesn’t come from mkDerivation? No, looks like it comes from runCommandLocal which comes from runCommandWith which uses mkDerivation.

So then I thought perhaps I could use pkgs.substitute; but then I ran into build-support/substitute does not allow spaces in arguments · Issue #178438 · NixOS/nixpkgs · GitHub , which I initially thought I could make a PR to fix and eventually gave up.

So then I gave up and just used runCommandLocal to give me access to the substitute function, which then required a builtins.readFile:

pkgs.writers.writePython3Bin "say_hello" { }
      (builtins.readFile (pkgs.runCommandLocal "script-text" {}
        ''
          substitute ${./hello.py} $out \
            --replace 'world' 'there n8henrie'
        ''));

Am I missing something here? If you had a perfectly dandy python script whose content you did not want to change, is there a better way to make a simple subtitution and still enjoy the simplicity of pkgs.writers?

I really was hoping that the overrideAttrs approach was going to work!

2

In pkgs/build-support/writers/scripts.nix we can see where things are coming from:

writePython3Bin → writePython3 → makePythonWriter → makeScriptWriter

As its final argument, makeScriptWriter constructs a big string that – among other things – writes the script content to $out and passes this to runCommandLocal.

From there, pkgs/build-support/trivial-builders/default.nix shows us:

runCommandLocal → runCommandWith, which accepts buildCommand as its final argument and passes this to mkDerivation.

If we look at pkgs/stdenv/generic/setup.sh, we can see:

if [ -n "${buildCommand:-}" ]; then
        eval "$buildCommand"
        return
fi

and therefore none of the phases are run.

This is all addressed in reasonable detail at this related issue: support phases with buildCommand? · Issue #66826 · NixOS/nixpkgs · GitHub

So while the usual phases tricks won’t work, one can always just override buildCommand:

# flake.nix
{
  description = "A very basic python script";

  outputs = {
    self,
    nixpkgs,
  }: let
    system = "aarch64-darwin";
    pkgs = import nixpkgs {inherit system;};
    name = "say_hello";
  in {
    packages.${system}.default =
      (pkgs.writers.writePython3Bin name {} ./hello.py)
      .overrideAttrs (old: {
        buildCommand =
          old.buildCommand
          + ''
            sed -i 's/world/there n8henrie/g' $out/bin/${name}
          '';
      });
    apps.${system}.default = {
      type = "app";
      program = "${self.outputs.packages.${system}.default}/bin/${name}";
    };
  };
}

$ nix run
hello, there n8henrie

Success!

Instead of sed, this also seems to work fine:

substituteInPlace $out/bin/${name} \
    --replace 'world' 'there n8henrie'

It doesn’t look like there’s an obvious way to prepend the substitution step before buildCommand (i.e. "sed ..." + old.buildCommand as opposed to old.buildCommand + "sed..."), because buildCommand runs cat $contentPath > $out, where $contentPath is read-only in the nix store.

1 Like