Can not use the Python Package which is built from GitHub Repo

Help
1

I need to use the Python package which is call argparse but it isn’t available on NixOS Packages, so I need to build it from the source.

Here, it’s my shell.nix

{ pkgs ? import <nixpkgs> { } }:
let
  my-python = pkgs.python3;
  python-with-my-packages = my-python.withPackages (p: with p; [
    pip
    pytest
    pybind11
    numdifftools
  ]);

  argparse = pkgs.python3Packages.buildPythonPackage rec {
    name = "argparse";
    src = builtins.fetchGit {
      url = "https://github.com/ThomasWaldmann/argparse.git";
      ref = "refs/heads/master";
    };
  };
in
pkgs.mkShell {
  buildInputs = [
    python-with-my-packages
    argparse
  ];
  shellHook = ''
    PYTHONPATH=${python-with-my-packages}/${python-with-my-packages.sitePackages}
  '';
}

And I try to verify it and can’t see the Package argparse

$ nix-shell shell.nix --command 'pip list'

So I try to find it in the /nix/store/

$ find /nix/store -name "*argparse" 
/nix/store/gp7r5bmynf71hkgdzkdz7xjx3jcw1smp-python3.9-argparse
/nix/store/9r7nkmwwyny6pk690k5lwyk3vh1z8a1y-python3.9-argparse
/nix/store/c1n2qddrmf94bk756x9l9pya0g2vqas0-python3.9-argparse

I find it is built successfully but the nix-shell doesn’t know the path.

$ tree /nix/store/gp7r5bmynf71hkgdzkdz7xjx3jcw1smp-python3.9-argparse 
/nix/store/gp7r5bmynf71hkgdzkdz7xjx3jcw1smp-python3.9-argparse
├── lib
│   └── python3.9
│       └── site-packages
│           ├── argparse-1.4.0.dist-info
│           │   ├── direct_url.json
│           │   ├── INSTALLER
│           │   ├── LICENSE.txt
│           │   ├── METADATA
│           │   ├── RECORD
│           │   ├── REQUESTED
│           │   ├── top_level.txt
│           │   └── WHEEL
│           ├── argparse.py
│           └── __pycache__
│               └── argparse.cpython-39.pyc
└── nix-support
    └── propagated-build-inputs

So, I want to ask how to tell the nix-shell in my shell.nix file ?

Thank you for your reading.

2

Have you tried just starting python and importing argparse? Unless you’re talking about some other module with the same name, it’s one of the modules included with python: argparse — Parser for command-line options, arguments and sub-commands — Python 3.9.14 documentation

3

If I just starting python and importing argparse, I can get the version of it.

$ nix-shell shell.nix 
[nix-shell] $ python
Python 3.9.13 
>>> import argparse
>>> print(argparse.__version__)
1.1

But, If I use the following shell.nix, which include a package call mugrade, it’s a GitHub Repo and it uses the argparse.

{ pkgs ? import <nixpkgs> { } }:
let
  my-python = pkgs.python3;
  python-with-my-packages = my-python.withPackages (p: with p; [
    pip
    pytest
    pybind11
    numdifftools
  ]);

  argparse = pkgs.python3Packages.buildPythonPackage rec {
    name = "argparse";
    src = builtins.fetchGit {
      url = "https://github.com/ThomasWaldmann/argparse.git";
      ref = "refs/heads/master";
    };
  };

  mugrade = pkgs.python3Packages.buildPythonPackage rec {
    name = "mugrade";
    src = builtins.fetchGit {
      url = "https://github.com/dlsyscourse/mugrade.git";
      ref = "refs/heads/main";
    };
  };
in
pkgs.mkShell {
  buildInputs = [
    python-with-my-packages
    argparse
    mugrade
  ];
  shellHook = ''
    PYTHONPATH=${python-with-my-packages}/${python-with-my-packages.sitePackages}
  '';
}

$ nix-shell shell.nix

After execute the command, the error messages as below

ERROR: Could not find a version that satisfies the requirement argparse>=1.1 (from mugrade) (from versions: none)
ERROR: No matching distribution found for argparse>=1.1

error: builder for '/nix/store/wyvxgs8lspbwz8zhkx507njzni94cgsh-python3.9-mugrade.drv' failed with exit code 1;
       last 10 log lines:
       > adding 'mugrade-1.2.dist-info/RECORD'
       > removing build/bdist.linux-x86_64/wheel
       > Finished executing setuptoolsBuildPhase
       > installing
       > Executing pipInstallPhase
       > /build/source/dist /build/source
       > Processing ./mugrade-1.2-py3-none-any.whl
       > ERROR: Could not find a version that satisfies the requirement argparse>=1.1 (from mugrade) (from versions: none)
       > ERROR: No matching distribution found for argparse>=1.1
       >
       For full logs, run 'nix log /nix/store/wyvxgs8lspbwz8zhkx507njzni94cgsh-python3.9-mugrade.drv'.

$ nix log /nix/store/wyvxgs8lspbwz8zhkx507njzni94cgsh-python3.9-mugrade.drv
error: experimental Nix feature 'nix-command' is disabled; use '--extra-experimental-features nix-command' to override

So, I think I need to handle the path of argparse.

4

You can probably force this to work by just making sure both of the python packages you’re adding expressions for are also included in the withPackages environment.

That said, I suspect GitHub - dlsyscourse/mugrade: mugrade scripts for 10-714 has made an error in specifying argparse as a separate dependency since none of their required python versions would’ve come without it. Given the cautions in the README at GitHub - ThomasWaldmann/argparse: python argparse, pypi version (moved from google code), I would recommend trying to patch argparse out of mugrade’s setup.py (you can probably create a prePatch that uses substituteInPlace to snip it out).