"Exec format error" `dockerTools.streamLayeredImage` creates an image that can't be loaded

Help
1

Hello!

In mboyea/nixflymc I build a Nix package into a Docker container at src/server-image.nix:

{ dockerTools, name, version, packages, apps }: dockerTools.streamLayeredImage {
  name = "${name}-server-image";
  tag = "${version}";
  contents = [ packages.server ];
  config = {
    Cmd = [ apps.server.program ];
    ExposedPorts = {
      "25565/tcp" = {};
    };
  };
}

I run the container using nix run .#serverContainer (or podman run docker-archive:/dev/stdin).

The container loads and then throws this error:

{"msg":"exec container process `/nix/store/ii4fzmg0m5zaixdsv0wv3w4myw2jgq9m-nixflymc-server-0.0.0/bin/nixflymc-server`: Exec format error","level":"error","time":"2024-10-31T22:02:38.880181Z"}

Then it exits.

However, if I call the Nix package directly with nix run .#server the software works as expected.

Why?

Steps to Reproduce

  • git clone https://github.com/mboyea/nixflymc
  • cd nixflymc
  • nix run .#server
  • Observe that this starts a Minecraft server
  • nix run .#serverContainer
  • Observe that this does not start a Minecraft server

Additional Information

Thank you for your time!

I appreciate this community <3

2

The container doesn’t seem to have a usefull command, what dies apps.server.program evaluate to?

Is there a reason for not using getExe?

3

There’s no particular reason to not use lib.getExe.

I will use it now; I did not know about it before.

With that change:

Cmd = [ "${lib.getExe packages.server}" ];

The same behavior is present.

4

I just verified that it isn’t an issue with calling the script.

Currently, src/server.nix looks like:

{ stdenv, name, version, system, nix-minecraft } : stdenv.mkDerivation rec {
  pname = "${name}-server";
  inherit version;
  src = nix-minecraft.packages.${system}.vanilla-server;
  installPhase = ''
    mkdir -p $out/bin $out/lib
    cp -r -v $src $out/lib/minecraft-server

    echo "echo 'eula=TRUE' > eula.txt" > $out/bin/${pname}

    cat >> $out/bin/${pname} << EOF
      $out/lib/minecraft-server/bin/minecraft-server
    EOF
    chmod +x $out/bin/${pname}
  '';
  meta.mainProgram = "${pname}";
}

But if I replace it with:

{ pkgs, stdenv, name, version, system, nix-minecraft } : pkgs.hello

then the Docker container works as expected, printing “Hello, world!”

So there must be something wrong with the server itself, just only when packaged into the Docker image.

5

You don’t seem to prefix the script with a shebang line - guessing that’s the problem? Maybe easiest to use one of the shell script helpers?

6

Well no unfortunately, assuming you mean the mkDerivation installPhase in src/server.nix.

  1. server.nix works fine when run directly by nix run .#server so the script itself must work.
  2. installPhase is wrapped by mkDerivation to include other bash lines like set -euo pipefail, so a shebang wouldn’t do anything because it’s not at the top of the execution (kernel would never see it).
  3. I went ahead and tried to add the shebang as you suggested, and it made no difference.
1 Like
7

In bed on phone so can’t fiddle with it, unfortunately. I meant the $out/bin/${pname} script might need a shebang, but I agree, I can’t see why it would work outside the container…

8

Gah, older versions of nix seem to use execvp() for nix run, which will try to run a script with /bin/sh if there’s no shebang. Guessing your environment has that link and the docker image doesn’t?

9

I think you’re onto something…!

I’ve compiled a “distroless” Docker image, which means it doesn’t include an operating system, and by extension, perhaps it doesn’t include a shell by default. Maybe if I were to manually include a shell in the image, and also a shebang, the program could run.

Of course it can’t run without a shell…
I’ll post back here with what I find.

10

So JUST adding the shebang in src/server.nix like you suggested was enough to make it work!

{ stdenv, name, version, system, nix-minecraft } : stdenv.mkDerivation rec {
  pname = "${name}-server";
  inherit version;
  src = nix-minecraft.packages.${system}.vanilla-server;
  installPhase = ''
    mkdir -p $out/bin $out/lib
    cp -r -v $src $out/lib/minecraft-server

    cat > $out/bin/${pname} << EOF
    #!/bin/bash
    echo 'eula=TRUE' > eula.txt
    $out/lib/minecraft-server/bin/minecraft-server
    EOF

    chmod +x $out/bin/${pname}
  '';
  meta.mainProgram = "${pname}";
}

I really thought I’d have to use a base image to introduce the bash shell like in pkgs/build-support/docker/examples.nix:

{ pkgs, name, version, packages, lib }: let
  bash = pkgs.dockerTools.buildImage {
    name = "bash";
    tag = "latest";
    copyToRoot = pkgs.buildEnv {
      name = "image-root";
      paths = [ pkgs.bashInteractive ];
      pathsToLink = [ "/bin" ];
    };
  };
in pkgs.dockerTools.streamLayeredImage {
  name = "${name}-server-image";
  tag = "${version}";
  fromImage = bash;
  contents = [
    packages.server
  ];
  config = {
    Cmd = [ "${lib.getExe packages.server}" ];
    ExposedPorts = {
      "25565/tcp" = {};
    };
  };
}

But it turns out that fromImage = bash; isn’t necessary, and it does suffice to just add the shebang by itself. Perhaps bash is included with the base image unlike Docker’s “scratch” image, or maybe it detects the dependency. Either way, I’m pretty impressed!

Thank you for your help @srd424

1 Like
11

Hello.
Posting back here because I now have more information.

The standard when writing a derivation that uses cat to generate a runtime script is to use the shebang #!/${pkgs.runtimeShell}.

Rather than use the shebang #!/bin/bash, please use #!/${pkgs.runtimeShell}. This enables packages which consume your software to override the shell.