CUDA 12.8 support in nixpkgs

[SergeK]

@lucassong3000

And i thought there is a dedicated CUDA team of nix to be doing this, no?

Much of the dedicated team is busy securing resources to fix more CUDA issues in Nixpkgs

@lucassong3000

    export LD_LIBRARY_PATH=${pkgs.linuxPackages.nvidia_x11}/lib

Generally speaking, pkgs.linuxPackages from a random Nixpkgs instance pkgs is not compatible with the running kernel. The userspace driver libcuda.so has to match the version string in the kernel module (.ko) exactly, otherwise the driver short circuits into an error. You never want to reference pkgs.linuxPackages.nvidia_x11, except as config.boot.kernelPackages.nvidia_x11 in a NixOS configuration. On NixOS CUDA programs load the kernel-compatible driver from /run/opengl-driver/lib/libcuda.so (grep Nixpkgs for usages of autoAddDriverRunpath). Outside NixOS the current status quo is to use wrappers (nixglhost or nixGL).

@lucassong3000

    export LD_LIBRARY_PATH=${pkgs.cudaPackages.cuda_cudart}/lib64:${pkgs.cudaPackages.cuda_cudart}/lib:$LD_LIBRARY_PATH
    export LD_LIBRARY_PATH=${stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH

Note that LD_LIBRARY_PATH takes priority over DT_RUNPATH. Setting LD_LIBRARY_PATH means you disable the deterministic/“static” dynamic loading as configured by Nixpkgs, and force the program instead to load a potentially incompatible library from elsewhere.

@lucassong3000

PyTorch and Numba use the CUDA Runtime API in a more abstracted way:

To best of my knowledge they do not.

@lucassong3000

• They don’t directly initialize CUDA devices like our raw CUDA C code

To the best of my knowledge they do, they either dlopen("cudart", ...) or even directly dlopen("cuda", ...) (not sure why, possibly linking pieces of static cudart); please link if there’s something I’m not aware of.

@lucassong3000

• They include version compatibility layers

To the best of my knowledge they do not, they rely on the same compatibility guarantees promised by nvidia: cudart version at runtime >= cudart at build time && cudart at runtime <= cudaDriverGetVersion() && libcuda.so version == nvidia.ko (or equivalent) version unless using cuda_compat. EDIT(2025-02-24): the original message had the wrong sign for cudaRuntimeGetVersion vs cudaDriverGetVersion

@lucassong3000

you will have some sort of unknown cuda errors, that is mostly caused by cuda version mismatch

Likely caused by LD_LIBRARY_PATH but hard to tell without logs

@lucassong3000

worse yet, the CUDA wiki wrote by whoever is just large incompetent and very annoying with little useful information to say the least.

The wiki article hasn’t been updated in years, recent contributors have been mostly putting stuff in the Nixpkgs manual (which would still benefit from more work).

@TLATER

The latest currently available is 12.6, you could do the legwork of adding the new version upstream.

There’s instructions for updating the package set in the Nixpkgs manual (admittedly the current process is more involved and cumbersome than it should be)

@lucassong3000

I dont know how.

This can be helped: it takes time, search, engagement with other users and contributors. Among other places, conversations in https://matrix.to/#/#cuda:nixos.org may be relevant