buildPythonPackage: No matching distribution found for requests (from incapsula-cracker-py3==0.1.8.1)

Help
1

Today(yesterday) I switched to NixOS from Fedora. Over the last few months I’ve read/learned a lot.
I’m doing a slow transition using the NixOS Gnome image. Now I’m trying to convert/fix all my little scripts (bash,python).

I have a python script which uses incapsula-cracker(https://github.com/ziplokk1/incapsula-cracker-py3). Because it’s not inside nix, I can’t use it like it did with some other python packages(eg. feedparser, argcomplete,…).

In configuration.nix : environment.systemPackages = with pkgs; [

			(python38.withPackages(ps: with ps; [
				argcomplete
				colorama
				dateutil
				feedparser
				maxminddb
				numpy
				python-slugify
				beautifulsoup4
				incapsula-cracker-py3  # THIS ERROR LINE 271:5
			]))

running “nixos-rebuild switch” errors:

error: undefined variable ‘incapsula-cracker-py3’ at /etc/nixos/configuration.nix:271:5

Next I tried to fetch it from pypi

			(let
				my_incapsula = python38.pkgs.buildPythonPackage rec {
					pname = "incapsula-cracker-py3";
					version = "0.1.8.1";

			    	src = python38.pkgs.fetchPypi {
					    inherit pname version;
					    sha256 = "60079f6602a3e4ef21d68e7bc99b52e378ae1d0e55135b05de01a241d71d1fe7";
					};

			      	buildInputs = [ python38.pkgs.beautifulsoup4 ];

			      	doCheck = false;

			      	meta = {
					    homepage = "https://github.com/ziplokk1/incapsula-cracker-py3";
					    description = "A way to bypass incapsula robot checks when using requests.";
			      	};
			    };

			  	in python38.withPackages (ps: [
				  	ps.argcomplete
					ps.colorama
					ps.dateutil
					ps.feedparser
					ps.maxminddb
					ps.numpy
					ps.python-slugify
					ps.toolz

				  	my_incapsula
			  	])
			)

This also errors:

Finished executing setuptoolsBuildPhase
installing
Executing pipInstallPhase
/build/incapsula-cracker-py3-0.1.8.1/dist /build/incapsula-cracker-py3-0.1.8.1
Processing ./incapsula_cracker_py3-0.1.8.1-py3-none-any.whl
ERROR: Could not find a version that satisfies the requirement requests (from incapsula-cracker-py3==0.1.8.1) (from versions: none)
ERROR: No matching distribution found for requests (from incapsula-cracker-py3==0.1.8.1)
builder for '/nix/store/swwkyprl8jz0cwhhbi2my9h2x8smzkp7-python3.8-incapsula-cracker-py3-0.1.8.1.drv' failed with exit code 1
cannot build derivation '/nix/store/gnvp0123r02i7gnqxwxh5iffnfcbzb9w-python3-3.8.5-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/8ykipkd044p23sf2kzssl0p2zl70im8r-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/hk7y1njmnvjkrn7g8qcfm7w3kfmkqgss-nixos-system-sl-think-20.09.1889.58f9c4c7d3a.drv': 1 dependencies couldn't be built
error: build of '/nix/store/hk7y1njmnvjkrn7g8qcfm7w3kfmkqgss-nixos-system-sl-think-20.09.1889.58f9c4c7d3a.drv' failed

Can anyone please steer me in the right direction?

2 Likes
2

This is a missing dependency error, it means that your pythonPackage requires requests.

You can shorten a bit your attribute:

propagatedBuildInputs = with python3.pkgs; [ beautifulsoup4 requests ];

python3 == python38 for the current release, and this way, when we switch to 3.9, the transition will be smooth

3

ty

I tried:

propagatedBuildInputs = with python3.pkgs; [ beautifulsoup4 requests six ];
and
propagatedBuildInputs = with python3.pkgs; [ beautifulsoup4 requests six bs4 ];

but now it errors on bs4(bs4 · PyPI), which is a “dummy package”.

Executing pipInstallPhase
/build/incapsula-cracker-py3-0.1.8.1/dist /build/incapsula-cracker-py3-0.1.8.1
Processing ./incapsula_cracker_py3-0.1.8.1-py3-none-any.whl
Requirement already satisfied: requests in /nix/store/qg81j8h5f6fki2ghkxajvfaxk09wxf2m-python3.8-requests-2.24.0/lib/python3.8/site-packages (from incapsula-cracker-py3==0.1.8.1) (2.24.0)
ERROR: Could not find a version that satisfies the requirement bs4 (from incapsula-cracker-py3==0.1.8.1) (from versions: none)
ERROR: No matching distribution found for bs4 (from incapsula-cracker-py3==0.1.8.1)
builder for '/nix/store/fpjskgz7a102ah3ml1sqj4mhvgxss6qm-python3.8-incapsula-cracker-py3-0.1.8.1.drv' failed with exit code 1
cannot build derivation '/nix/store/m0p7rs3dgx1fc9gkb7agmgi55sd28ycr-python3-3.8.5-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/bpn9ph6gvsyz5vapviprg9l38xnxr1pk-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/rdypxqbfwr5465g63nvn8677xw2rvv8s-nixos-system-sl-think-20.09.1889.58f9c4c7d3a.drv': 1 dependencies couldn't be built
error: build of '/nix/store/rdypxqbfwr5465g63nvn8677xw2rvv8s-nixos-system-sl-think-20.09.1889.58f9c4c7d3a.drv' failed

and

$ sudo nixos-rebuild switch
building Nix...
building the system configuration...
error: undefined variable 'bs4' at /etc/nixos/starting-configuration.nix:272:79
(use '--show-trace' to show detailed location information)
4

Great … the setup.py file is asking for the wrong dependency…

prePatch = ''
  substituteInPlace setup.py \
    --replace "bs4" "beautifulsoup4"
'';

This should do the trick

5

Ty @freezeboy It works

future reference:

			(let
				my_incapsula = python3.pkgs.buildPythonPackage rec {
					pname = "incapsula-cracker-py3";
					version = "0.1.8.1";

			    	src = python3.pkgs.fetchPypi {
					    inherit pname version;
					    sha256 = "60079f6602a3e4ef21d68e7bc99b52e378ae1d0e55135b05de01a241d71d1fe7";
					};

					propagatedBuildInputs = with python3.pkgs; [ beautifulsoup4 requests six ];

					prePatch = ''
					  substituteInPlace setup.py \
					    --replace "bs4" "beautifulsoup4"
					'';

			      	doCheck = false;

			      	meta = {
					    homepage = "https://github.com/ziplokk1/incapsula-cracker-py3";
					    description = "A way to bypass incapsula robot checks when using requests.";
			      	};
			    };

			  	in python3.withPackages (ps: [
				  	ps.argcomplete
					ps.colorama
					ps.dateutil
					ps.feedparser
					ps.maxminddb
					ps.numpy
					ps.python-slugify
					ps.toolz
					ps.beautifulsoup4

				  	my_incapsula
			  	])
			)
1 Like
6

Next level will be to submit it as an official nixpkgs package :wink:

7

For hassle free dependency management you might want to try mach-nix:

let
  mach-nix = import (builtins.fetchGit {
    url = "https://github.com/DavHau/mach-nix/";
    ref = "refs/tags/3.0.2";
  }) {};
in
mach-nix.mkPython rec {
  requirements = ''
    argcomplete
    colorama
    python-dateutil
    feedparser
    maxminddb
    numpy
    python-slugify
    toolz
    beautifulsoup4
    incapsula-cracker-py3
  '';
}
1 Like