I’m trying to launch a machine learning algorithm on ubuntu 22.04. I’ve spent literally dozens of hours specifying all the dependencies in pyproject.toml and then porting it to nix with poetry2nix, with specifying different overrides, etc. You can have a look at what I’ve done here. After a really tiring work I managed to actually run the program without python reporting missing dependencies and nix build failing. But it occurred python libraries (tensorflow, torch, ray) cannot find the GPU. System reports:
> inxi -G
Graphics: Device-1: NVIDIA TU106 [GeForce RTX 2070] driver: nvidia v: 545.23.08
Display: server: X.org 1.21.1.4 driver: loaded: nvidia unloaded: fbdev,modesetting,nouveau,vesa tty: 181x42
Now, you can take a look at what derivations I use in overrides, PATH, CUDA_PATH and LD_LIBRARY_PATH. Essentially, I use nixes’ versions of… pretty much everything. Now, I can provide my nvidia_x11 package to the environment (LD_LIBRARY_PATH), then e.g. torch complains:
In [2]: torch.cuda.get_device_name(0)
/nix/store/k8r8pkh3hcpsny433px8g2npfqnwbzp1-python3-3.10.13-env/lib/python3.10/site-packages/torch/cuda/__init__.py:611: UserWarning: Can't initialize NVML
warnings.warn("Can't initialize NVML")
Out[2]: 'NVIDIA GeForce RTX 2070'
And tensorflow says I just don’t have a gpu (thx tensorflow, you’re reeeeally helping). The problem here seems to be in mismatched versions
Feels like providing drivers other than that installed on the system is dumb indeed. But if I drop the nvidia_x11 altogether, torch says I don’t have nvidia drivers at all (but they are obviously installed).
All in all, I just don’t know what to do with this. What can be done to make nix work with this? Of course, I can drop all the poetry stuff and use packages bundled with nix, but that kinda kills all the point of using such versatile tool as poetry.