Fix-python: make Python "just run" on NixOS

Following my dated post: How to use Python on NixOS without Nix

I’ve been using the same script for more than a year now, so I turned it into a small reusable tool.

It simply sets the RPATH of all binaries in your project’s venv with packages from Nix. It’s kind of ad-hoc, but it’s simple and flexible.

Any feedback is very much welcome!

15 Likes

Hi Guillaume…slight contretemps

[marcus@nixos:~/Programming/Python/Morsels]$ fix-python --venv virt_env --libs .nix/libs.nix
sh: ligne 1: file : commande introuvable
sh: ligne 1: file : commande introuvable
sh: ligne 1: file : commande introuvable
Found 1 binary files
Patching file:
/nix/var/nix/profiles/default/bin/fix-python: ligne 97: patchelf : commande introuvable
/nix/var/nix/profiles/default/bin/fix-python: ligne 100: patchelf : commande introuvable
/nix/var/nix/profiles/default/bin/fix-python: ligne 101: patchelf : commande introuvable

Hi! Glad to see some feedback!

Woops my bad, I forgot to mention that patchelf is needed in the path…

Before running fix-python, you should do

nix shell nixpkgs#patchelf

Let me know how it goes! :smile:

Hi Guillaume. Je rame toujours

[marcus@nixos:~/Programming/Python/Morsels]$ fix-python --venv virt_env --libs .nix/libs.nix
sh: ligne 1: file : commande introuvable
sh: ligne 1: file : commande introuvable
sh: ligne 1: file : commande introuvable
Found 1 binary files
Patching file:
patchelf: getting info about ‘’: No such file or directory
patchelf: getting info about ‘’: No such file or directory
patchelf: getting info about ‘’: No such file or directory

Thanks for the feedback :+1:

I’ve made it so the tool now automatically loads its runtime dependencies (file and patchelf) if they are missing (c9c8e8a).

Hope it helps!

Basically nix-ld but locally?

Indeed :+1: nix-ld can be used to solve these sorts of issues.

I would love to see a comparison from an external PoV of the pros and cons between my approach and nix-ld. :smile:

why not just using buildFHSEnv with set of packages you need to build your pip requirements, and work inside of it the most usual venv way?

why not just

Well, you can “just do” that :slight_smile: I’m fine with buildFHSEnv being a better solution for you, or even in general.

What led me to not use buildFHSEnv was that it is not so accessible for less experienced NixOS users. As a new user, you started using NixOS to write foo.bar.enable = true which is great… then suddenly need to learn a whole lot about Nix to get something as simple as pip install numpy to work. My experience is that not everyone wants such headaches.

What fix-python brings IMO is: use Python on NixOS without deep diving into Nix.

I can imagine that people would then start using some other alternatives (e.g. nix-ld, buildFHSEnv) that suits their own configuration/workflow better. But now I can say to other people “you can easily use your regular Python workflow on NixOS”. :smile:

2 Likes

didn’t mean to impose “my way” in any manner, sorry.

I’ve myself started using nixos relatively recently and tried out some different ways of making python development environments.
When I finally discovered buildFHSEnv - it just felt like one of those “aha!” moments. It solved 2 problems straight away: using python “the usual venv way”, and isolating dev env from the rest of the system, since buildFHSEnv uses bwrap, so you can straight away unshare pid, mount and other namespaces, and bind new home so that a “bad” pip package cannot read your real home, and you yourself cannot accidently delete anything important

2 Likes

No worries mate, no offense was taken :smile:
I’m happy to hear that you found a solution that fits your needs :ok_hand: indeed fix-python can’t do all that.

Nice work! I think it will come in very handy to me. Can you have both python packages from nixpkgs and regular packages installed via pip for example? Lets say I want to use the nixpkgs tensorflow and use pybullet from pip.

Thank you for the interest and feedback!
Mixing packages from a site-packages built with Nix and from a venv is not in the scope at the moment, though it sounds interesting.
I had made some sort of template that achieved that: GitHub - tweag/python-nix-flake-template: Bootstrap a reproducible yet flexible Python development environment using Nix