Am I understanding string contexts correctly?

Help
1

I’m currently working on trying create bootable unattended install mediums for my NixOS configs. In order to make the install mediums perform unattended installs, I created a systemd service that looks something like this:

# This file is dedicated to the public domain using 🅭🄍1.0:
# <https://creativecommons.org/publicdomain/zero/1.0>.
/**
  A NixOS module for performing unattended installs.
*/
{ lib, ... }:
{
  systemd.services.unattended-install.script =
    let
      # In my actual NixOS config, I automatically generate a URL that refers to
      # the flake’s Nix store path. The flake’s Nix store path could
      # theoretically contain bytes that can’t safely be put into a URL’s path,
      # and I wanted to make sure that those bytes get percent-encoded if they’re
      # present.
      #
      # Originally, I was using lib.strings.escapeURL to escape the Nix store
      # path. The documentation for that function says “Escape the `string` so it
      # can be safely placed inside a URL query.” [1] I’m not trying to escape a
      # value so that it can be put in a URL query. I’m trying to escape a value
      # so that it can be put in a URL path. Unfortunately, the list of
      # characters that need to be escaped in URL paths is different than the
      # list of characters that need to be escaped in URL queries [2]. This means
      # that lib.strings.escapeURL is not guaranteed to always work for URL
      # paths.
      #
      # I ended up writing my own function that does percent-encoding. It
      # percent-encodes all bytes, even if they don’t need to be percent-encoded.
      # That way, I don’t have to worry about which byte sequences do or do not
      # need to be percent-encoded.
      #
      # [1]: <https://github.com/NixOS/nixpkgs/blob/d3d2d80a2191a73d1e86456a751b83aa13085d7d/lib/strings.nix#L1042-L1068>
      # [2]: <https://url.spec.whatwg.org/#path-percent-encode-set>
      url = "path:<completely percent-encoded Nix store path>";
    in
    ''
      disko-install \
        --flake ${lib.strings.escapeShellArg url}#exampleNixOSConfig \
        --disk main /dev/vda \
        --write-efi-boot-entries
    '';
}

When I created bootable install mediums that used that module, I ran into a problem. The systemd service was failing because the Nix store path for the flake was missing.

I was very surprised when this happened. The string that contains the URL has string context that refers to the flake’s Nix store path (I used builtins.break and builtins.getContext to verify this). I thought for sure that the string context would make it so that the systemd service depends on the flake’s Nix store path, but it looks like that’s not the case.

I created this shell.nix file in order to test to see how string contexts work:

# This file is dedicated to the public domain using 🅭🄍1.0:
# <https://creativecommons.org/publicdomain/zero/1.0>.
let
  nixpkgsCommit = "3e3afe5174c561dee0df6f2c2b2236990146329f";
  nixpkgsTarball = builtins.fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/${nixpkgsCommit}.tar.gz";
    sha256 = "0dcslr2lwfaclfl4pmbwb3yw27bnvwlqiif394d3d66vyd163dvy";
  };
  pkgs = import nixpkgsTarball { };
  inherit (pkgs) writeShellScriptBin;
  inherit (pkgs.lib.attrsets) getBin;
  inherit (pkgs.lib.meta) getExe;
  inherit (pkgs.lib.strings) addContextFrom escapeShellArg;

  scriptText = ''
    echo 'Hello, world!'
  '';
  exampleScript = writeShellScriptBin "example-script" scriptText;

  scriptTextWithExtraContext = addContextFrom "${pkgs.hello}" scriptText;
  exampleScriptWithExtraContext = writeShellScriptBin "example-script-with-extra-context" scriptTextWithExtraContext;

  nixStoreCommand = escapeShellArg "${getBin pkgs.nix}/bin/nix-store";
  grepCommand = escapeShellArg (getExe pkgs.gnugrep);
in
pkgs.mkShellNoCC {
  name = "shell-for-testing-string-contexts";
  shellHook = ''
    set -o errexit -o nounset -o pipefail

    function depends_on_hello {
      ${nixStoreCommand} --query --requisites "$@" | ${grepCommand} --silent hello
    }

    if depends_on_hello ${escapeShellArg exampleScript.drvPath}
    then
      echo exampleScript’s drv file depends on the hello package.
    else
      echo exampleScript’s drv file does not depend on the hello package.
    fi

    if depends_on_hello ${escapeShellArg exampleScript}
    then
      echo exampleScript depends on the hello package.
    else
      echo exampleScript does not depend on the hello package.
    fi

    if depends_on_hello ${escapeShellArg exampleScriptWithExtraContext.drvPath}
    then
      echo exampleScriptWithExtraContext’s drv file depends on the hello package.
    else
      echo exampleScriptWithExtraContext’s drv file does not depend on the hello package.
    fi

    if depends_on_hello ${escapeShellArg exampleScriptWithExtraContext}
    then
      echo exampleScriptWithExtraContext depends on the hello package.
    else
      echo exampleScriptWithExtraContext does not depend on the hello package.
    fi

    exit
  '';
}

When I run that file with nix-shell <path to that file>, here’s what I get:

exampleScript’s drv file does not depend on the hello package.
exampleScript does not depend on the hello package.
exampleScriptWithExtraContext’s drv file depends on the hello package.
exampleScriptWithExtraContext does not depend on the hello package.

From what I can tell, string contexts only affect the dependencies of .drv files. They do not affect the dependencies of a derivation’s outputs, even if one of the outputs is just a copy of a string that has string context.

Am I understanding string contexts correctly? I ask because this behavior is really surprising to me.

2

Yeah, the dependencies of an output are determined by, essentially, grepping the output for store paths. (More details here.) String contexts won’t save you if the literal store path—not encoded, not compressed—of a dependency isn’t somewhere in your output.

1 Like