"{kvm} is required" error building a Docker image using "runAsRoot"

Help
1

I’m attempting to use the following recipe to build a Docker image:

{ pkgs ? import <nixpkgs> { }
, pkgsLinux ? import <nixpkgs> { system = "x86_64-linux"; }
}:
pkgs.dockerTools.buildImage {
  name = "delme";
  tag = "latest";
  contents = pkgs.buildEnv {
    name = "image-root";
    paths = with pkgsLinux; [
      bashInteractive
      coreutils
      curl
    ];
    pathsToLink = [ "/bin" ];
  };
  runAsRoot = "mkdir -p /data";
  config = {
    Cmd = [ "${pkgsLinux.bashInteractive}/bin/bash" ];
  };
}

I hit the following error on nix-build:

error: a 'x86_64-linux' with features {kvm} is required to build '/nix/store/zf5rifg7gyszd9cxx8xmxmzfcg4ia772-docker-layer-delme.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, nixos-test}

The error only occurs if the runAsRoot attribute is provided, otherwise the Docker image builds correctly. Hoping someone can advise as to what the error means, and how I might be able to fix it. Thanks!

2

The runAsRoot setting uses kvm (kernel VM implementation) to run the commands, rather than doing it through containers.

I don’t really understand why either, to be honest, would be much better to use something like bubblewrap.

This requires enabling your CPU’s virtualization features in BIOS if you’re on bare metal. If you’re building on cloud infra you might be out of luck, or at the very least need to look into virtualization options…

An alternative might be to use Entrypoint and manage your directory setup/switching to the target user there.

3

I’m sorry to ask this, but are you sure you really need to do something as root? I observed users often rely on runAsRoot while it would actually not be required.

If you really need runAsRoot, i would be interested in your use case!

4

Not at all, it’s a great question - if there’s a better way to achieve the same goal, I’m very happy to to learn about it!

My use case is adding a non-root user to the container, and creating a /tmp directory:

   runAsRoot = ''
      #!${pkgs.runtimeShell}
      ${pkgs.dockerTools.shadowSetup}
      groupadd --gid 1000 -r jenkins
      useradd -m --uid=1000 -r -g jenkins jenkins
      mkdir /tmp
      chmod 1777 /tmp
    '';
1 Like
5

With a big of digging, I found this post, which suggested adding:

system-features = kvm

to /etc/nix/nix.conf. That enabled the build to work without an error (although I got the following logs - it looks like it’s using QEMU TCG emulation to run something?)

Could not access KVM kernel module: No such file or directory
qemu-kvm: failed to initialize kvm: No such file or directory
qemu-kvm: falling back to tcg
cSeaBIOS (version rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org)


iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+1FF90FC0+1FEF0FC0 CA00



Booting from ROM...
Probing EDD (edd=off to disable)... ocloading kernel modules...
[    2.928094] Invalid ELF header magic: != ELF
[    3.186787] Invalid ELF header magic: != ELF
[    3.261119] Invalid ELF header magic: != ELF
[    3.324290] Invalid ELF header magic: != ELF
[    4.836923] Invalid ELF header magic: != ELF
[    4.908298] Invalid ELF header magic: != ELF
[    5.039657] Invalid ELF header magic: != ELF
[    5.115107] Invalid ELF header magic: != ELF
[    5.186883] Invalid ELF header magic: != ELF
[    5.259467] Invalid ELF header magic: != ELF
[    5.421870] Invalid ELF header magic: != ELF
[    5.490063] Invalid ELF header magic: != ELF
[    5.551640] Invalid ELF header magic: != ELF
[    5.621662] Invalid ELF header magic: != ELF
[    5.690720] Invalid ELF header magic: != ELF
[    6.290835] Invalid ELF header magic: != ELF
[    6.442829] Invalid ELF header magic: != ELF
[    6.578055] Invalid ELF header magic: != ELF
[    6.681639] Invalid ELF header magic: != ELF
[    6.751195] Invalid ELF header magic: != ELF

For future Googling in case it helps someone else - I could also do the same the workaround in a Github Cachix action as follows:

    - name: Install Nix
      uses: cachix/install-nix-action@v18
      with:
        # Needed for "runAsRoot" capability
        extra_nix_config: |
          system-features = kvm
6

Regarding the user, maybe defining the uid:gid as image configuration(Config.User=...) could be enough.

Regarding /tmp permissions, i don’t remember if it is possible with dockerTools (without runAsRoot) but nix2container provides a way to set permissions on files.

7

For my use case, I wanted a “real” user with a home dir etc. Is runAsRoot discouraged? I was using this recipe.

8

Is runAsRoot discouraged?

No, but it’s often boring to need KVM :wink:

1 Like
9

You can make a “real”* nonroot user by creating /etc/passwd, /etc/group, /etc/shadow in a layer, no runAsRoot required

This example is pretty complicated but it’s complete:

&

Or just do something like this replacing youruser and 12345 with whatever

echo "youruser:x:12345:12345:youruser:/home/youruser:/bin/false" > $out/etc/passwd
echo "youruser:x:12345:" > $out/etc/group
echo "youruser:!:1::::::" > $out/etc/shadow

I typically just do passwd and group really, I’ve not looked into the benefit of including shadow

add a root:x:0:0....etc to that too if you want

*(UID is determined at container creation so it doesn’t really matter if the container FS has that user listed, it will just say it’s unknown. The main thing you want to watch out for is host collisions if you’re not using user namespacing)

λ docker run -u 12345 -it nixos/nix
bash-5.1$ id -u
12345
bash-5.1$ whoami
whoami: cannot find name for user ID 12345
bash-5.1$ cat /etc/passwd | grep "12345"
bash-5.1$
10

Hm, I wonder what the effect of using qemu-tcg is. I’d dig into that before recommending/using it. If it’s still properly virtualized, sure, but I suspect it’s just running things on the host, and might sneak in nonreproducible state.

Or just go with any of the recommendations here, it’ll significantly speed up your builds to skip runAsRoot.

11

This example is pretty complicated but it’s complete:

Thanks - I have to admit I’m very new to Nix, and this is the first use case I’m really trying it out with, so I apologise in advance that I don’t really know what I’m doing! In particular, I don’t know how to transplant that example to my use case - I don’t suppose you could indicate how I could change my example from above to e.g. write a line to /etc/passwd without using runAsRoot? Would be much appreciated!

12 

{ pkgs ? import <nixpkgs> { }
, pkgsLinux ? import <nixpkgs> { system = "x86_64-linux"; }
}:
pkgs.dockerTools.buildImage {
  name = "delme";
  tag = "latest";
  contents = [
    (pkgs.buildEnv {
      name = "image-root";
      paths = with pkgsLinux; [
        bashInteractive
        coreutils
        curl
      ];
      pathsToLink = [ "/bin" ];
    })
    (pkgs.runCommand "user" ''
      echo "hello" > $out/etc/file
    '')
  ];
  config = {
    Cmd = [ "${pkgsLinux.bashInteractive}/bin/bash" ];
  };
}

just eyeballing it, something like this should work

13

I get an error unfortunately, not sure what it’s indicating:

$ nix-build delme.nix
trace: warning: in docker image delme: The contents parameter is deprecated. Change to copyToRoot if the contents are designed to be copied to the root filesystem, such as when you use `buildEnv` or similar between contents and your packages. Use copyToRoot = buildEnv { ... }; or similar if you intend to add packages to /bin.
error: cannot coerce a function to a string

       at /nix/store/37nl1pnk837xd0033qw7jrq7i18q36vn-nixpkgs/nixpkgs/pkgs/build-support/docker/default.nix:350:9:

          349|         inherit baseJson extraCommands;
          350|         contents = copyToRoot;
             |         ^
          351|         nativeBuildInputs = [ jshon rsync tarsum ];
(use '--show-trace' to show detailed location information)
14

ok

Well either you could drop the pathsToLink and use 1 env. or do this:

{ pkgs ? import <nixpkgs> { }
, pkgsLinux ? import <nixpkgs> { system = "x86_64-linux"; }
}:
pkgs.dockerTools.buildImage {
  name = "delme";
  tag = "latest";
  copyToRoot = pkgs.buildEnv {
    name = "image-root";
    paths = with pkgsLinux; [
      (pkgs.buildEnv {
        name = "image-root";
        paths = with pkgsLinux; [
          bashInteractive
          coreutils
          curl
        ];
        pathsToLink = [ "/bin" ];
      })
      (pkgs.runCommand "user" ''
        echo "hello" > $out/etc/file
      '')
    ];
  };
  config = {
    Cmd = [ "${pkgsLinux.bashInteractive}/bin/bash" ];
  };
}
15

Thanks - unfortunately I still get an error with that recipe:

$ nix-build delme.nix
trace: warning: in docker image delme: The contents parameter is deprecated. Change to copyToRoot if the contents are designed to be copied to the root filesystem, such as when you use `buildEnv` or similar between contents and your packages. Use copyToRoot = buildEnv { ... }; or similar if you intend to add packages to /bin.
error: cannot convert a function to JSON

       at /nix/store/37nl1pnk837xd0033qw7jrq7i18q36vn-nixpkgs/nixpkgs/pkgs/build-support/trivial-builders.nix:78:8:

           77|     # TODO(@Artturin): enable strictDeps always
           78|     }: buildCommand:
             |        ^
           79|     stdenv.mkDerivation ({
(use '--show-trace' to show detailed location information)
16

replace contents with copytoroot

17

I get the same error.

18 
{ pkgs ? import <nixpkgs> { }
, pkgsLinux ? import <nixpkgs> { system = "x86_64-linux"; }
}:
pkgs.dockerTools.buildImage {
  name = "delme";
  tag = "latest";
  copyToRoot = pkgs.buildEnv {
    name = "image-root";
    paths = with pkgsLinux; [
      (pkgs.buildEnv {
        name = "image-root";
        paths = with pkgsLinux; [
          bashInteractive
          coreutils
          curl
        ];
        pathsToLink = [ "/bin" ];
      })
      (pkgs.runCommand "user" { } ''
        mkdir -p $out/etc
        echo "hello" > $out/etc/file
      '')
    ];
  };
  config = {
    Cmd = [ "${pkgsLinux.bashInteractive}/bin/bash" ];
  };
}
19

Many thanks - that does now work for me :+1:

I think I’ll need to go off and read up a bit to understand exactly what’s going on in this recipe, but I had one conceptual question - what’s the difference between making changes to a Docker image using pkgs.runCommand and runAsRoot? They ostensibly would seem similar - both making changes to the image through a series of commands that need root privileges. One wants to use KVM and the other doesn’t?

20

anything that goes in copyToRoot is going to be built in nix world as normal. runAsRoot builds a VM, then runs that VM in order to safely run these commands as “root” even if you’re not “root” on the host & then exports the output.

We could try replace some of that with a rootless container but that would likely then depend on user namespaces being enabled for your kernel, whereas with nix we can enforce having KVM as a required feature.

https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-system-features

Maybe we could add a system feature for user namespaces but I’ve not seen any discussion around that yet

1 Like