Noob: shell with python and packages

Help
1

Hi,

I am a newbie who just started using nixos (unstable). I followed advice from this article but I am not having any luck creating even the most simple python shell. For example, I tried following shell.nix

{ pkgs ? import <nixpkgs> {} }:
let
  my-python = pkgs.python3;
  python-with-my-packages = my-python.withPackages (p: with p; [
    oscrypto
  ]);
in
pkgs.mkShell {
  buildInputs = [
    python-with-my-packages
  ];
  shellHook = ''
    PYTHONPATH=${python-with-my-packages}/${python-with-my-packages.sitePackages}
  '';
}

After I run nix-shell I do get a python interpreter with oscrypto package installed, but I get an exception

python -c 'from oscrypto import asymmetric'                                                                                           10.0.2.15 via ❄️  impure (nix-shell)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/nix/store/na2ysz3s2cfx5m60h4h5fa0k6hczgxkw-python3-3.9.10-env/lib/python3.9/site-packages/oscrypto/asymmetric.py", line 19, in <module>
    from ._asymmetric import _unwrap_private_key_info
  File "/nix/store/na2ysz3s2cfx5m60h4h5fa0k6hczgxkw-python3-3.9.10-env/lib/python3.9/site-packages/oscrypto/_asymmetric.py", line 27, in <module>
    from .kdf import pbkdf1, pbkdf2, pkcs12_kdf
  File "/nix/store/na2ysz3s2cfx5m60h4h5fa0k6hczgxkw-python3-3.9.10-env/lib/python3.9/site-packages/oscrypto/kdf.py", line 9, in <module>
    from .util import rand_bytes
  File "/nix/store/na2ysz3s2cfx5m60h4h5fa0k6hczgxkw-python3-3.9.10-env/lib/python3.9/site-packages/oscrypto/util.py", line 14, in <module>
    from ._openssl.util import rand_bytes
  File "/nix/store/na2ysz3s2cfx5m60h4h5fa0k6hczgxkw-python3-3.9.10-env/lib/python3.9/site-packages/oscrypto/_openssl/util.py", line 6, in <module>
    from ._libcrypto import libcrypto, libcrypto_version_info, handle_openssl_error
  File "/nix/store/na2ysz3s2cfx5m60h4h5fa0k6hczgxkw-python3-3.9.10-env/lib/python3.9/site-packages/oscrypto/_openssl/_libcrypto.py", line 15, in <module>
    from ._libcrypto_ctypes import (
  File "/nix/store/na2ysz3s2cfx5m60h4h5fa0k6hczgxkw-python3-3.9.10-env/lib/python3.9/site-packages/oscrypto/_openssl/_libcrypto_ctypes.py", line 28, in <module>
    raise LibraryNotFoundError('The library libcrypto could not be found')
oscrypto.errors.LibraryNotFoundError: The library libcrypto could not be found

However, if I use nix-shell -p python39Packages.oscrypto, I don’t get the above error. I am now totally confused about what am I doing wrong. My ultimate goal is to replace a traditional python virtualenv that I normally use to develop several python packages.

Thank you.

2

The problem here is a missing c library. Try adding these packages to your buildInputs: https://github.com/NixOS/nixpkgs/blob/ed02c2ba0384b2800db41333045a6fb781f12aac/pkgs/development/python-modules/oscrypto/default.nix#L24

In theory propagatedBuildInputs means that that should not be necessary, but I guess the way python packages are propagated to the env using withPackages breaks that? It’s a bit odd…

I don’t think setting PYTHONPATH is required, by the way, since that’s the entire point of using withPackages. withPackages works by setting the path using makeWrapper, it may have unintended consequences if you override that.

No offense to the wiki contributors, but it’s sadly often inaccurate or even misleading, the official docs are generally better. This is the python section, which has a fair bit of detail, and also a snippet like yours: Nixpkgs 23.11 manual | Nix & NixOS

You may also be interested in GitHub - DavHau/mach-nix: Create highly reproducible python environments

1 Like
3

I think the tools like poetry2nix and mach2nix take care of things like this, but the LD_LIBRARY_PATH isn’t being set in your mkShell, so it’s not finding the library.

4

If that’s the case, the solution would be:

{ pkgs ? import <nixpkgs> {} }:
let
  inherit (pkgs.lib) makeLibraryPath;
  my-python = pkgs.python3;
  python-with-my-packages = my-python.withPackages (p: with p; [
    oscrypto
  ]);
in
pkgs.mkShell {
  buildInputs = [
    python-with-my-packages
  ];
  shellHook = ''
    LD_LIBRARY_PATH=${makeLibraryPath [
      pkgs.openssl
      pkgs.asn1crypto
    ]}
  '';
}

I still fail to see how this can be necessary though. LD_LIBRARY_PATH should be populated with the contents of propagatedBuildInputs, or at least buildInputs if that fails because of the makeWrapper indirection. If either suggestion fixes this, maybe we should open an issue?