CUDA 12.8 support in nixpkgs

Development
35

wow extremely interesting results on nixos with latest cuda 12.8 :hugs:


alice7@nixos ~/.d/t/cuda-puzzles> nvcc -I$CUDA_PATH/include -L$CUDA_PATH/targets/x86_64-linux/lib -lcudart -lcublas -Wno-deprecated-gpu-targets benchmark_matrix_mul.cu -o benchmark
alice7@nixos ~/.d/t/cuda-puzzles> ./benchmark
Starting CPU matrix multiplication...
CPU compute took 6.407872 seconds
Starting GPU matrix multiplication with cuBLAS...
GPU compute (cuBLAS) took 0.031666 seconds

Performance Comparison:
CPU time: 6.407872 seconds
GPU time: 0.031666 seconds
Speedup (CPU/GPU): 202.36x
Verification passed!
2 Likes
36

:+1: Note that upstream we no longer use the run-file based release out of licensing, closure size, and “build” time considerations: Nixpkgs Reference Manual

1 Like
38

may I ask if not download from the source website, where should i obtain the original source cuda12.8 package before building it from the first place?

39

Per Nixpkgs manual:

    Updating CUDA redistributables
    <https://nixos.org/manual/nixpkgs/unstable/#updating-cuda-redistributables>

Go to NVIDIA’s index of CUDA redistributables:
https://developer.download.nvidia.com/compute/cuda/redist/
<https://developer.download.nvidia.com/compute/cuda/redist/>

Thank you

1 Like
40

You don’t need such a long and verbose flake.nix. Try this flake.nix:

# flake.nix, run with `nix develop`
{
  description = "CUDA development environment";
  outputs = {
    self,
    nixpkgs,
  }: let
    system = "x86_64-linux";
    pkgs = import nixpkgs {
      inherit system;
      config.allowUnfree = true;
      config.cudaSupport = true;
      config.cudaVersion = "12";
    };
  in {
    # alejandra is a nix formatter with a beautiful output
    formatter."${system}" = nixpkgs.legacyPackages.${system}.alejandra;
    devShells.${system}.default = pkgs.mkShell {
      buildInputs = with pkgs; [
        ffmpeg
        fmt.dev
        cudaPackages.cuda_cudart
        cudatoolkit
        linuxPackages.nvidia_x11
        cudaPackages.cudnn
        libGLU
        libGL
        xorg.libXi
        xorg.libXmu
        freeglut
        xorg.libXext
        xorg.libX11
        xorg.libXv
        xorg.libXrandr
        zlib
        ncurses5
        stdenv.cc
        binutils
        uv
      ];

      shellHook = ''
        export LD_LIBRARY_PATH="${pkgs.linuxPackages.nvidia_x11}/lib:$LD_LIBRARY_PATH"
        export CUDA_PATH=${pkgs.cudatoolkit}
        export EXTRA_LDFLAGS="-L/lib -L${pkgs.linuxPackages.nvidia_x11}/lib"
        export EXTRA_CCFLAGS="-I/usr/include"
        export CMAKE_PREFIX_PATH="${pkgs.fmt.dev}:$CMAKE_PREFIX_PATH"
        export PKG_CONFIG_PATH="${pkgs.fmt.dev}/lib/pkgconfig:$PKG_CONFIG_PATH"
      '';
    };
  };
}
2 Likes
41

In addition, if you use packages instead of buildInputs, you can skip the shellHook almost entirely.

… besides:

You also don’t need to add alejandra, and its use should probably be discouraged, since NixOS upstreams now have an actual formatter. Using alejandra means going against ecosystem conventions.

Go ahead if you’ve been using it, and switch to it if you really like its output better, but don’t recommend it to new users by default, certainly not without that caveat.

2 Likes
42

Thanks for your nice advice!

How to use packages, what should I change in the flake.nix?

43

Change that to:

devShells.${system}.default = pkgs.mkShell {
      packages = with pkgs; [
        ffmpeg
        fmt.dev
        cudaPackages.cuda_cudart

Generally, when using mkShell you should never use buildInputs and just specify packages.

Under the hood, packages is translated to nativeBuildInputs, which will run for example the pkg-config hook which automatically configures the pkg-config environment variable - then you can remove that line from your shellHook because it will already be set correctly.

There likely are hooks for most of the other variables you set, but you’ll have to try what can and cannot be removed, I don’t know the hooks by heart :wink:

1 Like