Bind mount output of `runCommand` (and other questions of a more general nature)

Help
1

I am creating a NixOS module to run a service obtained from a Docker image in a systemd-nspawn container.

The general idea is:

  1. Pull the image using pkgs.dockerTools.pullImage
  2. Export that image to a tarball with pkgs.dockerTools.exportImage
  3. Extract that tarball into the Nix store with a simple invocation of pkgs.runCommand
  4. Bind mount the extracted tree into /var/lib/machines/<service-name>
  5. Create the appropriate <service-name>.nspawn file
  6. Enable the relevant systemd-nspawn@<service-name>.service

As an example of what I’m thinking of, here’s my attempt at endlessh.nix:

{ config, pkgs, lib, ... }:
let
  cfg = config.containerServices.endlessh;
  endlesshImage = pkgs.dockerTools.exportImage {
    fromImage = pkgs.dockerTools.pullImage {
      imageName = "ghcr.io/linuxserver/endlessh";
      imageDigest = "sha256:1a80087704a1fffb750b5a07ef296d2ff21f8524d4d206d896cad521096c8a75";
      finalImageTag = "dfe44eb2-ls72";
      sha256 = "";  # will handle this once things are working
    };
  };
  endlesshTree = pkgs.runCommand {
    name = "endlessh";
    buildCommand = ''
      mkdir "$out"
      tar --directory="$out" --extract --file="${endlesshImage}"
    '';
  };
in
{
  options = {
    containerServices.endlessh = {
      enable = lib.mkEnableOption (lib.mdDoc "Endlessh is an SSH tarpit that very slowly sends an endless, random SSH banner.");
    };
  };

  config = lib.mkIf cfg.enable {
    fileSystems."/var/lib/machines/endlessh" = {
      device = "${endlesshTree}";
      options = [ "bind" ];
    };

    systemd.nspawn.endlessh = {
      enable = true;
      execConfig = { /*snip*/ };
      filesConfig = { /*snip*/ };
      networkConfig = { /*snip*/ };
    };

    systemd.services."systemd-nspawn@endlessh.service".enable = true;
  };
}

It would be used in configuration.nix like:

{ config, pkgs, lib, ... }:
{
  imports = [ ./endlessh.nix ];
  containerServices.endlessh.enable = true;
  ...
}

This unfortunately doesn’t work: nixos-rebuild build fails with the error cannot coerce a function to a string in the definition of the device for the bind mount. I have tried "${endlesshTree}", applying "${builtins.path endlesshTree}", and also changing to pkgs.callPackage pkgs.runCommand ... to no avail.

This leaves me with a few questions:

  1. How can I get the output path of the runCommand invocation in my configuration? Is a bind mount even the best way to make part of the Nix store available under /var?
  2. Would it be possible to refactor this so, for example, the image tag/digest could be overridden with an option like containerServices.endlessh.imageDigest = "..."? In its current form it seems like this would be difficult since it would be affecting things that were evaluated above it. (This is in trying to be analogous to the common services.<name>.package option.)
  3. Is this approach overall sound, or is there a better way to get an unpacked Docker image into the store? (Is this an Import From Derivation in a bad way?)
  4. Finally, an optimization question: if my understanding is correct, the Docker image will be present in the store 3 times: once in whatever form pullImage gets it, once in the tarball from exportImage, and once after being unpacked. Is there a reasonable way to avoid this without having to fully reimplement parts of dockerTools?

Thanks for your help!

1 Like
2
  1. buildCommand is a string passed to runCommand
    runCommand "endlessh" {} ''mkdir "$out"; tar --directory="$out" --extract --file="${endlesshImage}"''
  2. There is no such thing as “evaulated above” in nix :wink: , they are all evaluated on demand.
    1. Something wants evaluate ``fileSystems.“/var/lib/machines/endlessh”`
      • if no one need it, evaluantion “stops” here, and nothing else will be evaluated.
    2. It requires ${endlessTree} to be evaluated
    3. It requires ${endlesshImage} to be evaluated
      • If you add ${cfg.imageDigest} in your image build, it will evaluate before,
    4. It build your image
  3. No clue :slight_smile:
  4. Yes, but if you left no reference to the previous steps, when you run nix store gc, others will be collected.
3

Heck yeah! I was just reading the Nix Pill on functions but I didn’t really consider the difference between an argument set and “normal” arguments. Thanks!

1 Like