Nixos-container permission problem

Help
1

Hi, I’m currently trying to set up ollama inside a nixos-container with GPU acceleration.

When I configure ollama on my machine without a container everything is working perfectly fine.
If i wrap the same configuration with containers.ollama = { ... it’s not working.
After investigating quite a bit, I’ve hit the wall of permission errors that I can’t get past

in essence, the ollama service is trying to acces my GPU under /sys/class/kfd/kfd/topology/nodes/1/properties as the ollama user (inside the container)
i’ve passed /sys/class/kfd (and all other relevant paths) into the container via bindMounts
If I log into the container and run the service start script as root the service is coming up fine (with GPU accel)

After adding ExecStartPre=cat /sys/class/kfd/kfd/topology/nodes/1/properties via runtime overrides i can see that the ollama user can not access the file (but root can)

here is my .nix file with everything I’ve tried:

{
  pkgs,
  lib,
  ...
}:
{
  nixpkgs.config.rocmSupport = true;
  containers.ollama = {
    autoStart = false;
    privateNetwork = false;

    restartIfChanged = true;
    timeoutStartSec = "5min";

    bindMounts = {
      "/var/lib/ollama" = {
        hostPath = "/data/ollama/";
        isReadOnly = false;
      };

      "/dev/dri" = {
        hostPath = "/dev/dri";
        isReadOnly = false;
      };

      "/dev/kfd" = {
        hostPath = "/dev/kfd";
        isReadOnly = false;
      };

      "/sys/class/kfd/kfd/topology/nodes" = {
        hostPath = "/sys/class/kfd/kfd/topology/nodes";
        isReadOnly = false;
      };

      "/sys/class/drm" = {
        hostPath = "/sys/class/drm";
        isReadOnly = false;
      };

      "/sys/module/amdgpu" = {
        hostPath = "/sys/module/amdgpu";
        isReadOnly = false;
      };

    };

    forwardPorts = {
      containerPort = 8080;
      hostPort = 80;
      protocol = "tcp";
    };

    config = {
      system.stateVersion = "unstable";
      nixpkgs.config.rocmSupport = true;
      services = {
        ollama = {
          enable = true;

          home = "/var/lib/ollama";
          loadModels = [
            "deepseek-r1:14b"
            "deepseek-coder-v2"
          ];

          acceleration = "rocm";

          # https://github.com/NixOS/nixpkgs/issues/308206
          # https://rocm.docs.amd.com/en/latest/reference/gpu-arch-specs.html
          rocmOverrideGfx = "11.0.1"; # gfx1101

          host = "127.0.0.1";
          port = 11434;
          openFirewall = true;
        };

        nextjs-ollama-llm-ui = {
          enable = true;
          port = 8080;
          ollamaUrl = "http://127.0.0.1:11434";
        };
      };

      boot.kernelModules = [ "amdgpu" ];
      hardware.amdgpu.opencl.enable = true;
      hardware.graphics = {
        enable = true;
        enable32Bit = true;
        extraPackages = [ pkgs.rocmPackages.clr.icd ];
      };

      environment.systemPackages = with pkgs; [
        rocmPackages.clr.icd
      ];

      systemd.tmpfiles.rules = [
        "L+    /opt/rocm   -    -    -     -    ${pkgs.rocmPackages.clr}"
        "L+    /opt/rocm   -    -    -     -    ${pkgs.rocmPackages.rocblas}"
      ];

    };
  };
}

I’m on the nixos-unstable branch where this PR has been merged

thanks in advance :+1:

1 Like