Overriding the config.toml used by nvidia-container-runtime

gautaz · January 14, 2024, 11:48am

Hello,

I am trying to understand an issue related to using nvidia-container-runtime in rootless docker (see Docker rootless with nvidia support).

I need to enable the debug log of the nvidia-container-toolkit as described here: Troubleshooting — NVIDIA Container Toolkit 1.14.2 documentation

I have located the config.toml to change:

❯ cat /nix/store/frmvd1w17nd622mk7vvj4qay0lph9amx-container-toolkit-container-toolkit-1.9.0/etc/nvidia-container-runtime/config.toml
disable-require = false
#swarm-resource = "DOCKER_RESOURCE_GPU"

[nvidia-container-cli]
#root = "/run/nvidia/driver"
#path = "/usr/bin/nvidia-container-cli"
environment = []
#debug = "/var/log/nvidia-container-runtime-hook.log"
ldcache = "/tmp/ld.so.cache"
load-kmods = true
#no-cgroups = false
#user = "root:video"
ldconfig = "@/nix/store/bh4lz3c2n3qfbm2hhwjhnqcaxcjs2sm8-glibc-2.38-27-bin/bin/ldconfig"

And, from my understanding, the contents of this file come from this part of the nixpkgs:

github.com

NixOS/nixpkgs/blob/b1ddf062c781713af177253aadef6ad884ca2b79/pkgs/applications/virtualization/nvidia-container-toolkit/packages.nix#L24


      
          # The config is only exposed as an attrset so that the user may reach the
          # deafult values, for inspectability purposes.
          dockerConfig = {
            disable-require = false;
            #swarm-resource = "DOCKER_RESOURCE_GPU"
          
          
  nvidia-container-cli = {
              #root = "/run/nvidia/driver";
              #path = "/usr/bin/nvidia-container-cli";
              environment = [ ];
              #debug = "/var/log/nvidia-container-runtime-hook.log";
              ldcache = "/tmp/ld.so.cache";
              load-kmods = true;
              #no-cgroups = false;
              #user = "root:video";
              ldconfig = "@@glibcbin@/bin/ldconfig";
            };
          };
          nvidia-container-toolkit-docker = self.callPackage ./. {
            containerRuntimePath = "${docker}/libexec/docker/docker";
            configTemplate = self.dockerConfig;

I have seen a similar override made by @eadwu here:

But it seems to go quite further than just modifying the line I intend to.

As I am still quite uncomfortable with nix, would someone be kind enough to share a simple way to override this configuration file the nix way?

I guess it comes to using overriding but I am unable to wrap my head around this…