Adding non-python dependencies to poetry2nix application

Help
1

Hello there,

I am writing a python application which also has a runtime dependency on redis. I am using poetry2nix to build the application and package this up however I am not able to figure out how or whats the best way to add non-python dependencies to a poetry project (using mkPoetryApplication)?

Any guidance on this would be really appreciated!

Thanks,
Bhavesh.

1 Like
2

Here there is written that non listed attributes are passed to buildPythonApplication (mkPoetryApplication wraps it). So you could try passing

propagatedBuildInputs = [ pkgs.redis ];
3

Thanks @aciceri , that does work however the problem is that these packages are not installed when I install the python app. When I use nix-store -qR the resulting package, I can see that it has a dependency on redis as expected.

However, when I install the package using nix profile install (I am using nix flakes), the redis package is not made available in the path.

Is there something else which needs to be done to make redis available in the path?

EDIT: I would like to point out that I do get access to these in the dev shell environment. So perhaps there are gaps in my understanding of nix. When I enter the dev-shell using nix develop, I can see redis is added to the path. Both the mkPoetryApplication and mkShell are using propagatedBuildInputs as per @aciceri 's reply above.

4

I think I understand what I was missing then. Please correct me if I am wrong, but adding a package as dependency doesn’t necessarily mean its made available at runtime or is added to the path.

I had to use makeWrapper to add these packages to the path so they are available to my python application. Is it the recommended way of achieving this?

5

I think makeWrapper is a fine solution.

Essentially it means you are passing the redis dependency through the PATH environment variable, but you could also decide to pass it in a different way:

  • through a config file for your application
  • by replacing the redis path in the code directly at build time
  • by using another environment variable

The reason to do it like that is that it’s more hermetic, only the code which uses redis will see it, while using PATH/makeWrapper any subprocess of the application will see redis.

6

An example of how to do this with a flake:

{
  description = "Application packaged using poetry2nix";

  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    poetry2nix = {
      url = "github:nix-community/poetry2nix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, flake-utils, poetry2nix }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # see https://github.com/nix-community/poetry2nix/tree/master#api for more functions and examples.
        pkgs = nixpkgs.legacyPackages.${system};
        inherit (poetry2nix.lib.mkPoetry2Nix { inherit pkgs; }) mkPoetryApplication;
      in
      {
        packages = {
          smartp = mkPoetryApplication {
            projectDir = self;
            nativeBuildInputs = [ pkgs.makeWrapper ];
            propogatedBuildInputs = [ pkgs.util-linux ];
            postInstall = ''
              wrapProgram "$out/bin/smartp" \
                --prefix PATH : ${nixpkgs.lib.makeBinPath [ pkgs.util-linux ]}
            '';
          };
          default = self.packages.${system}.smartp;
        };

        devShells.default = pkgs.mkShell {
          inputsFrom = [ self.packages.${system}.smartp ];
          packages = [
            pkgs.poetry
            ];
        };
      });
}

For now smartp is a simple script that just calls subprocess.run a few times and prints the output:

import subprocess

def main():
    out = subprocess.run(["whereis", "lsblk"], capture_output=True)
    print(out)
    out = subprocess.run(["which", "lsblk"], capture_output=True)
    print(out)
    out = subprocess.run(["lsblk"], capture_output=True)
    print(out)

Running ./result/bin/smartp after building gives:

CompletedProcess(args=['whereis', 'lsblk'], returncode=0, stdout=b'lsblk: /usr/bin/lsblk /nix/store/4h02i6q6hfvp5avk97lympf43zvnk2d0-util-linux-2.39.2-bin/bin/lsblk /usr/share/man/man8/lsblk.8.gz\n', stderr=b'')
CompletedProcess(args=['which', 'lsblk'], returncode=0, stdout=b'/nix/store/4h02i6q6hfvp5avk97lympf43zvnk2d0-util-linux-2.39.2-bin/bin/lsblk\n', stderr=b'')
CompletedProcess(args=['lsblk'], returncode=0, stdout=b'NAME MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS\nsda    8:0    0 389.8M  1 disk \nsdb    8:16   0     4G  0 disk [SWAP]\nsdc    8:32   0     1T  0 disk /snap\n                               /mnt/wslg/distro\n                               /\n', stderr=b'')