torch+CUDA on nix without nixOS

[self_destruct_button]

I’m hoping to manage data science environments on a nvidia GPU equipped system running ubuntu with nix, a task I’ve been doing with conda up to now. Specifically I’m trying to get torch.cuda.is_available() to report true. In troubleshooting this issue I’ve identified the following issues:

• For nixpkgs CUDA support is enabled in a few ad hoc ways, from a global config options cudaSupport to override keys passed to individual packages to the use of distinguished packages like torchWithCuda.
• Binaries compiled with CUDA support require the use of a community cache, which I’ve done, to avoid compiling e.g. torch.
• Being on a non-nixOS system the graphics driver is not part of the nix environment which leads to the use of tools like nixGL and finnagling e.g. LD_LIBRARY_PATH to include system openGL paths and any nix-installed CUDA toolchains.

Most advice I’ve seen boils down to declaring a shell more or less like this one, but torch.cuda.is_available() isn’t what I need it to be no matter what I try it seems.

I guess I’ve got two questions:

1. Is the flake linked above not really the right recipe and should I be trying something else? The point was to use something idiomatic and share the approach with some colleagues, so using nix to deploy venvs doesn’t qualify; I can already do that without nix. Also, I don’t really know what the underlying issue even is or how to begin troubleshooting it.
2. Is this generally just a waste of time? Recent discussions are short on praise with experienced users admitting that nix is an obstacle more often than not and that they just use conda or venvs for ad hoc package use. nixGL seems to be abandoned (and doesn’t solve my problem), and I haven’t seen more recent initiatives to address the issue of running graphics accelerated applications on non-nixOS systems.

[waffle8946]

Nix on NixOS is fine, but nix on other distros historically required some schemes like nixGL which just doesn’t work well. Allegedly there are some fixes coming that make nixgl unnecessary:

Updating to mesa 25.0.2

So, unstable has somewhat recently shaken up how mesa is used in NixOS. Before these changes, it was technically unsafe to mix channels, because packages would be linked to the version of libgbm provided by the mesa in nixpkgs, and the drivers provided by the host system would require the libgbm from the host’s version of mesa. It was often the case that this would work out, but it certainly wasn’t guaranteed. The recent change is that the libgbm linked to by applications now is only a stub tha…

But I’ve not tried this in years, so I can’t comment on the recent changes.

Other options are discussed elsewhere:

CUDA app in non-NixOS Linux

I recently solved the problem of launching a CUDA app with nix run in Ubuntu and Fedora and I 'm going to post this code here because I haven’t seen this technique posted anywhere. I have a manufacia3 flake package which is built with poetry2nix and uses the CUDA version of torch. This code declares a manufacia3-cuda flake package which is a wrapper script that uses nixglhost to discover the system CUDA driver paths at runtime. packages.manufacia3-cuda = pkgs.writeShellApplication { …

Runtime issues on different flake package channels

This problem is exactly why I generally recommend that people do not mix channels. It’s unsupported and it causes these kinds of issues. But as a complete moonshot, there is a project called libcapsule, which is not packaged in nixpkgs, but you can explore it yourself; see libcapsule: init at 0.20240806.0; libglcapsule: init; libGL: point to libglcapsule and make supporting changes by Pandapip1 · Pull Request #337995 · NixOS/nixpkgs · GitHub for details.

Home-manager and graphic drivers on non-NixOS desktop

I’ve not had good results with nixGL on foreign distros personally. If possible I’d use kitty from your distro’s package manager. EDIT: there’s also GitHub - numtide/nix-gl-host: Run OpenGL/Cuda programs built with Nix, on all Linux distributions., I have no experience with it though.

Best of luck. You might also want to post your actual code if you want feedback on it.

[danieldk]

We use Nix with CUDA a lot and on non-NixOS systems we generally do this:

Nix CUDA on non-NixOS systems

@self_destruct_button

Is the flake linked above not really the right recipe and should I be trying something else?

No, because CUDA needs the CUDA driver library from the host operating system (which may be a different version than in Nix).

[self_destruct_button]

@danieldk

We use Nix with CUDA a lot and on non-NixOS systems we generally do this:

Nix CUDA on non-NixOS systems

@self_destruct_button

Is the flake linked above not really the right recipe and should I be trying something else?

No, because CUDA needs the CUDA driver library from the host operating system (which may be a different version than in Nix).

Your article was very helpful, thank you for writing and sharing it. I guess there’s no substitute (in this case) for understanding the underlying dynamic linking picture.

I was under the impression that the GL driver was correctly linked in the shell definition I linked, but of course, nothing is present at /run/opengl-driver on non-nixOS, which I might’ve checked beforehand. Setting LD_PRELOAD as your article suggests seems to have solved the problem.

I’m wondering why nixGL didn’t do this for me automatically. I’ll post a flake for posterity when I settle on one.

Thank you both for your assistance.