So your issue has to do with Docker rootless. Currently, Docker rootless does not copy the CDI specs into the namespace where dockerd is executed. I have opened a PR for Docker here: Dockerd rootless: make {/etc,/var/run}/cdi available by ereslibre · Pull Request #48541 · moby/moby · GitHub.
I also created a nixpkgs PR here: dockerd rootless: include patch to read /etc/cdi and /var/run/cdi by ereslibre · Pull Request #344005 · NixOS/nixpkgs · GitHub, but now is outdated, given I did change the moby PR and the hash does not match anymore. In any case, you can probably place a nixpkgs overlay to include that patch, or:
- Use rootful docker
- Use podman (not rootful) (
nix run nixpkgs#podman -- run --rm -it --device=nvidia.com/gpu=all ubuntu:latest nvidia-smi -L)
However, eventually, when Moby merges my PR, there’s a new Moby release --or we include the patch ourselves–, and we package the new version–or the current version with the patch once it gets accepted in moby/moby–, your current configuration should work out of the box.