Calling Python from Rust

Help
1

My goal: Create a flake.nix with a Rust binary that will import a Python module called osc (I want to call Python code from Rust).

My problem: Resulting Rust binary does not see a Python module. Even though it is included in the buildInputs section. What’s even weirder, a devShell has an identical buildInputs section and imports osc perfectly:

Console output 
[username@host]$ nix develop
[devShell@host]$ python -c "import osc"
[devShell@host]$ nix run
["/nix/store/rgmy7k7c7yrv2f67ijdjawc66kazwrkh-python3-3.11.11", "/nix/store/rgmy7k7c7yrv2f67ijdjawc66kazwrkh-python3-3.11.11"]

Succesfully imported osc!

[devShell@host]$ exit
[username@host]$ nix run
["/nix/store/rgmy7k7c7yrv2f67ijdjawc66kazwrkh-python3-3.11.11", "/usr"]

thread 'main' panicked at src/main.rs:17:6:
called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'ModuleNotFoundError'>, value: ModuleNotFoundError("No module named 'osc'"), traceback: None }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
My flake.nix file 
{
  description = "";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-24.11";
    nixpkgs-unstable.url = "nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs =
    {
      self,
      nixpkgs,
      nixpkgs-unstable,
      flake-utils,
      ...
    }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        pkgs-unstable = nixpkgs-unstable.legacyPackages.${system};

        python = pkgs.python311;
        pythonEnv = python.withPackages (
          ps: with ps; [
            osc
          ]
        );
        buildInputs = [
            python
            pythonEnv
        ];
      in
      rec {
        formatter = pkgs.nixfmt-rfc-style;

        devShells.default = pkgs.mkShell {
          inherit buildInputs;
        };

        packages.default = pkgs-unstable.rustPlatform.buildRustPackage rec {
          pname = "devaur";
          version = "0.1.0";

          cargoLock.lockFile = ./Cargo.lock;
          src = ./.;

          inherit buildInputs;

          nativeBuildInputs = with pkgs; [
            python
          ];
        };
      }
    );
}
My src/main.rs 
use pyo3::prelude::*;

#[tokio::main]
async fn main() {
    Python::with_gil(|py| {
        let site = PyModule::import(py, "site")?;

        let prefixes = site.getattr("PREFIXES")?.extract::<Vec<String>>()?;

        println!("{:?}", prefixes);

        let _ = PyModule::import(py, "osc")?;
        println!("\nSuccesfully imported osc!\n");

        Ok::<(), PyErr>(())
    })
    .unwrap();
}
My Cargo.toml 
[package]
name = "devaur"
description = ""
version = "0.1.0"
edition = "2024"

[dependencies]
tokio = { version = "1.44.1", features = ["full"] }
pyo3 = { version = "0.24.0", features = ["auto-initialize"] }

I am completely lost. From what I understand, osc might be present during the runtime of the Rust binary. Please help :​(

2

I don’t have experience doing something like that, but you should get rid of your python binding and only use pythonEnv.

I also know some C bindings need rustPlatform.bindgenHook and/or pkg-config in nativeBuildInputs, again, not sure if it’s relevant for python, or if there even are .pc files for that case.

3

Very interesting… I removed python and left only pythonEnv. Bahaviour of the devShell didn’t change, but rust binary not doesn’t see python3.11 at all! Although it did see lolcat!

So buildInputs do work. It’s only a withPackages function that acts like this.

4

Some progress! I just changed

python = pkgs.python311;
pythonEnv = python.withPackages (
  ps: with ps; [
    osc
  ]
);
buildInputs = with pkgs; [
  pythonEnv
  lolcat
];

into this

buildInputs = with pkgs; [
  python311.withPackages
  (
    ps: with ps; [
      osc
    ]
  )
  lolcat
];

And managed to get an error message!

error: Dependency is not of a valid type: element 1 of buildInputs for devaur
5

You’re going backward; you’re missing parentheses.

6

I don’t get it… I’ve completely removed python from any of the buildInputs, but the binary still has access to system’s python (3.10 in my case). Same thing with calling nix develop. Should’t these things be pure? Why even have an --impure flag then?

I’m either missing something or did something wrong.

I’ve just called nix’s garbage collector and re-downloaded all of the inputs. Deleted flake.lock multiple times. Nothing seems to help the situation. My ~/.config/nix/nix.conf has only one line — experimental-features = nix-command flakes

I really want to use Nix, but it just does not want to work :snowflake: :broken_heart:

My flake.nix 
{
  description = "";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-24.11";
    nixpkgs-unstable.url = "nixpkgs/nixos-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs =
    {
      nixpkgs,
      nixpkgs-unstable,
      flake-utils,
      ...
    }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
        pkgs-unstable = nixpkgs-unstable.legacyPackages.${system};
      in
      rec {
        formatter = pkgs.nixfmt-rfc-style;

        packages.default = pkgs-unstable.rustPlatform.buildRustPackage rec {
          pname = "devaur";
          version = "0.1.0";

          cargoLock.lockFile = ./Cargo.lock;
          src = ./.;
        };
      }
    );
}
7

Solved :white_check_mark:

The issue was that buildInputs do make python and its packaged available during runtime. Buy the path does not get modified automatically, you should tell Nix where to look for the available Python environment. You can do it like this:

pythonEnv = python.withPackages (
  ps: with ps; [
    osc
  ]
);

buildInputs = with pkgs; [
  pythonEnv
];

nativeBuildInputs = with pkgs; [
  makeWrapper
];

postFixup = ''
  wrapProgram $out/bin/devaur \
    --prefix PATH : ${lib.makeBinPath [ pythonEnv ]}
'';

Now everything works correctly.