Cuda from nixkgs in non-NixOs case

Help
1

Hello. I’m a newbie and I’m super confused about how even to formulate my question. A third of my confusion comes from not fully understanding how to properly configure things, and the other two thirds is about nvidia’s dependency infrastructure.

I’m trying to make cuda from nixpkgs work in ubuntu. As far as I understand it gets built against the nvidia_x11 package. But it has a driver version different from the version installed in my system. This wiki entry Nvidia - NixOS Wiki links to GitHub - nix-community/nixGL: A wrapper tool for nix OpenGL application [maintainer=@guibou].

I figured out how to add it to my shell.nix as a build input, but I need guidance how to configure cuda to use the driver provided by nixGL. Could you show me from what I could start?

with import <nixpkgs> {
     config.allowUnfree = true;
     config.cudaSupport = true;
     config.cudnnSupport = true;
};

let
  nixGL = fetchTarball "https://github.com/guibou/nixGL/archive/master.tar.gz";
  myNixGL = (import "${nixGL}/default.nix" {
          nvidiaVersion = "440.64.00";
          nvidiaHash = "08yln39l82fi5hmn06xxi3sl6zb4fgshhhq3m42ksiglradjd0ah";
          inherit pkgs;
  }).nixGLNvidia;
  pythonEnv = python37.withPackages (ps: with ps; [
            ## nothing here yet
  ]);
in mkShell {
  buildInputs = [
    python37
    pythonEnv
    linuxPackages.nvidia_x11
    cudatoolkit_10_1
    cudnn_cudatoolkit_10_1
    
    ## TODO: configure everything to use the driver version mentioned above
    
  ];

  shellHook = ''
    export CUDA_HOME=${pkgs.cudatoolkit_10_1}
    export PATH=${pkgs.cudatoolkit_10_1}/nsight-compute-2019.4.0:$(nixGLNvidia printenv PATH):${myNixGL}/bin
    export LD_LIBRARY_PATH=$(nixGLNvidia printenv LD_LIBRARY_PATH)
  '';
}
1 Like
2

I append it to my LD_LIBRARY_PATH as so:

  mkShell {
    name = "tf-dev-environment";

    nativeBuildInputs = [
      pythonEnv
    ];

    shellHook = ''
      export LD_LIBRARY_PATH=${pkgs.libGL}/lib:${pkgs.libGLU}/lib:${pkgs.freeglut}/lib:${pkgs.xorg.libX11}/lib:${pkgs.stdenv.cc.cc.lib}/lib:${pkgs.cudatoolkit_10_1}/lib:${pkgs.cudnn_cudatoolkit_10_1}/lib:${pkgs.cudatoolkit_10_1.lib}/lib:$LD_LIBRARY_PATH
      export LD_LIBRARY_PATH=$(nixGLNvidia printenv LD_LIBRARY_PATH):$LD_LIBRARY_PATH
    '';

https://github.com/guibou/nixGL/issues/46

3

Do I understand correctly that $(nixGLNvidia printenv LD_LIBRARY_PATH) works when it’s installed with nix-env -iA, and to make it work in shell.nix without installing it we should use something like this?

export LD_LIBRARY_PATH=$(${myNixGL}/bin/nixGLNvidia printenv LD_LIBRARY_PATH)
4

And I’m not sure how to address the following. My system has nvidia-440.64 installed while cuda in nixpkgs is built with 440.86. What should I do?

  • Pin nixpkgs to a commit with nvidia_x11 version lower than 440.86
  • Make an overlay to configure nvidia_x11 to use the desired version
  • Use nixGL here — I can’t figure out how

@guibou, could you help me with the third option?

1 Like
5

Sure. Could you open a bug in nixGL tracker so I can track that.

The problem is that I don’t have pure nvidia cuda hardware available. I’ll try with my hybrid GPU configuration and perhaps I’ll have results.

6

I’ve opened Cuda support · Issue #50 · nix-community/nixGL · GitHub to track cuda support.

1 Like
7

Ah, ok. It’s twice as embarrassing now, haha, as I can’t fully comprehend both the documentation of nix and the limitations of the nixGL utility. Thanks for opening the ticket. Can I help anyhow? I could at least pay for an azure nc6 instance, it comes with an nvidia gpu, and costs around $0.50 per hour.

1 Like
8

Let me link you post here:

9

This should be fixed now that cudnn: init cudnn_8_3 at 8.3.0 by samuela · Pull Request #158218 · NixOS/nixpkgs · GitHub has landed.