Nix-shell isn't executing its Python/poetry command and doesn't exit

Help
1

Hi,

Edit: current NixOS stable here. My config: GitHub - novoid/nixos-config: NixOS + flakes + home-manager with xfce, zsh, tmux, ... (host jackson)

I’m trying to write a shell script wrapper that deals with Python dependencies (using poetry) and calls a series of Python commands.

However, even the first Python call isn’t executed and it doesn’t exit. You can see everything from my minimal example on:

https://paste.debian.net/1343505/

I tried to strip down the issue to its core. So I even replaced the actual python call with a simple “python --version”.

As you can see, when invoked interactively (zsh), it processes the poetry install command, starts the nix-shell but doesn’t execute the poetry run command. I end up in the interactive nix-shell until I press Ctrl-D to kill the mostcurrent shell. Only after that, the final echo-commands get executed.

What I want: I want to execute the shell script, it sets up poetry, runs the poetry run command with the python command and exits to continue with the rest of the shell script.

What is my mistake here?

1 Like
2

AFAIK buildFHSUserEnv breaks the --run functionality in nix-shell :weary:

2 Likes
3

What you can at least do is specify the profile:

{ pkgs ? import <nixpkgs> { } }:
(pkgs.buildFHSEnv {
  name = "poetry-env";
  multiPkgs = pkgs: (with pkgs; [ python3 poetry libz ]);
  profile = ''
    set -x
    which python
    python --version
    which poetry
    poetry --version
    exit
  '';
}).env

❯ nix-shell
++ which python
/usr/bin/python
++ python --version
Python 3.12.7
++ which poetry
/usr/bin/poetry
++ poetry --version
Poetry (version 1.8.4)
++ exit
1 Like
4

Ha, got it!

{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell { # a normal nix shell, no FHS magic yet
  buildInputs = [
    # this puts an executable 'poetry-env' into the PATH
    # which executed by itself yeets you into the FHS environment,
    # but it also accepts a -c 'command-to-run' flag
    (pkgs.buildFHSEnv {
      name = "poetry-env";
      multiPkgs = pkgs: (with pkgs; [ python3 poetry libz ]);
    })
  ];
}

❯ nix-shell --run "poetry-env -c 'python --version'"
Python 3.12.7

It’s one additional annoying quoting layer, but I think this is as close as you can get.

Update: standalone nix-shell shebang version

Slightly uglier version but better for multiple commands, with nix-shell shebang:

run.sh:

#!/usr/bin/env nix-shell
#!nix-shell -i bash -E '{ pkgs ? import <nixpkgs> { } }: pkgs.mkShell { buildInputs = [ (pkgs.buildFHSEnv { name = "poetry-env"; multiPkgs = pkgs: (with pkgs; [ python3 poetry libz ]); }) ]; }'
poetry-env -c '
set -x
which python
python -V
poetry --version
'

❯ ./run.sh
+ which python
/usr/bin/python
+ python -V
Python 3.12.7
+ poetry --version
Poetry (version 1.8.4)

Update: standalone nix-shell shebang version together with above shell.nix

run.sh:

#!/usr/bin/env nix-shell
#!nix-shell -i bash
poetry-env -c '
set -x
which python
python -V
poetry --version
'
2 Likes
5

Thank you very much for your input here!

With your help, I was able to make the minimum example run after adapting the nix file and renaming it to “default.nix”.

Then I started to adapt my more complex shell script example, which is using shell variables and more CLI parameters.

Somehow, there seems to be an issue with a missing dependency.

I hope that I was able to extend the minimum example so that you can duplicate the error I get:

Example invocation with the dependency error visible:

2del % ./test.sh 
+ which python
/usr/bin/python
+ python -V
Python 3.12.7
+ poetry --version
Poetry (version 1.8.4)
+ echo '=========> example Python call:'
=========> example Python call:
+ python minimal.py
Traceback (most recent call last):
  File "/home/vk/tmp/2del/2del/minimal.py", line 1, in <module>
    from jinja2 import Environment, FileSystemLoader
ModuleNotFoundError: No module named 'jinja2'
+ echo '=========> end'
=========> end
2del %

All the required files

test.sh

The slightly extended test script that now calls the minimal.py script:

#!/usr/bin/env nix-shell
#!nix-shell -i bash
poetry-env -c "
set -x
which python
python -V
poetry --version
# poetry install  ## optionally enabled or not, doesn't matter
echo \"=========> example Python call:\"
python minimal.py
echo \"=========> end\"
"

I also tried to invoke poetry install within the poetry-env section with not having an effect on the error:

2del % ./test.sh       
+ which python
/usr/bin/python
+ python -V
Python 3.12.7
+ poetry --version
Poetry (version 1.8.4)
+ poetry install
Installing dependencies from lock file

No dependencies to install or update

Installing the current project: gematik_afo_vergleich (0.1.0)
Warning: The current project could not be installed: No file/folder found for package gematik-afo-vergleich
If you do not want to install the current project use --no-root.
If you want to use Poetry only for dependency management but not for packaging, you can disable package mode by setting package-mode = false in your pyproject.toml file.
In a future version of Poetry this warning will become an error!
+ echo '=========> example Python call:'
=========> example Python call:
+ python minimal.py
Traceback (most recent call last):
  File "/home/vk/tmp/2del/2del/minimal.py", line 1, in <module>
    from jinja2 import Environment, FileSystemLoader
ModuleNotFoundError: No module named 'jinja2'
+ echo '=========> end'
=========> end
2del %

Moving the poetry install above the poetry-env call and execute it beforehand didn’t resolve the issue as well.

default.nix

This is exactly your version from above:

{ pkgs ? import <nixpkgs> { } }:
pkgs.mkShell { # a normal nix shell, no FHS magic yet
  buildInputs = [
    # this puts an executable 'poetry-env' into the PATH
    # which executed by itself yeets you into the FHS environment,
    # but it also accepts a -c 'command-to-run' flag
    (pkgs.buildFHSEnv {
      name = "poetry-env";
      multiPkgs = pkgs: (with pkgs; [ python3 poetry libz ]);
    })
  ];
}

minimal.py

This example Python script is loading jinja2:

from jinja2 import Environment, FileSystemLoader
print('Testoutput')

pyproject.toml

Here, the jinja2 dependency is defined:

[tool.poetry]
name = "foo"
version = "0.1.0"
description = "foo"
authors = ["foo"]

[tool.poetry.dependencies]
python = "^3.10"
jinja2 = "^3.1.4"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
6

You will need to run your Python code through poetry with poetry run python minimal.py. Otherwise it’s just the nix-provided Python environment without any packages. That’s how poetry works.

7

Ah, I see.

I thought that poetry is already present with the poetry-env but now I realize that this assumption was stupid of mine.

Thanks. The minimal example with jinja now works, I’ll continue to adapt my “large” real-world example and report back if this finally works (which I assume for now).

1 Like
8

The extraInstallCommands and especially runScript might also be useful. :thinking: