FetchPyPi fails on many packets due to dots in path

Help
1

The fetchPyPi function of nixpkg is using the deprecated PEP427 format.

This causes multiple packages to fail with 404 errors when trying to fetch wheels.

According to Names and normalization - Python Packaging User Guide the the package maintainer can choose a variety of names.

I’m not sure how to solve this except to overwrite the packet URL directly (is this a possiblity? my nixfoo is quiet weak)…
trying multiple formats would I think also work? If nobody has a quick workaround I’ll guess I open a issue but maybe there is a quick way of getting it to work

Packets where I got stuck:

  • satpy’s trollimage
  • matplotlib
2

What did you try, including code and commands, and what is the exact, full error?

3

What I did:
create shell.nix

{ pkgs ? import <nixpkgs> {} }:                                                                                   
                                                                                                                  
pkgs.mkShell {                                                                                                    
  nativeBuildInputs = with pkgs; [                                                                                
    (                                                                                                             
    let                                                                                                           
      pyEnv = p: with p; [                                                                                        
      numpy      
      pyyaml                                                                                                      
      pooch                                                                                                       
      xarray                                                                                                      
      matplotlib                                                                                                  
      pillow                                                                                                      
      platformdirs                                                                                                
      dask
      (                                                                                                         
        buildPythonPackage rec {                                                                                  
         pname = "satpy";                                                                                         
         version = "0.53.0";                                                                                      
         format = "wheel";                                                                                        
                                                                                                                  
         src = fetchPypi {                                                                                      
           inherit pname version format;                                                                        
           python = "py3";                                                                                      
           dist = "py3";                                                                                        
           sha256 = "abea68dcfb27793bca9073abb4927fb92c5c39735390aa6235664d1fbad848e2";                         
         };                                                                                                     
         doCheck = false;                                                                                       
        }                                                                                                       
        )       
(                                                                                                       
        buildPythonPackage rec {                                                                                
         pname = "trollsift";                                                                                   
         version = "0.5.3";                                                                                     
         format = "wheel";                                                                                      
                                                                                                                
         src = fetchPypi {                                                                                      
           inherit pname version format;                                                                        
           python = "py3";                                                                                      
           dist = "py3";                                                                                        
           sha256 = "sha256-MS/Y0UtcS/xqVRYlpoJ97ZU6vi7pUPRLhhlP4OjcZbY=";                                      
         };                                                                                                   
         doCheck = false;                                                                                     
        }                                                                                                     
        )                                                                                                     
                                                                                                              
        (                                                                                                     
        buildPythonPackage rec {                                                                              
         pname = "donfig";                                                                                    
         version = "0.8.1.post1";                                                                             
         format = "wheel";                                                                                    
                                                                                                              
         src = fetchPypi {                                                                                    
           inherit pname version format;                                                                      
           python = "py3";                                                                                    
           dist = "py3";                                                                                      
           sha256 = "sha256-KjF1znSgYQn/kwfZCiMPgSFcusmnUfTRxhlGRLggT50=";                                    
         };                                                                                                   
         doCheck = false;                                                                                     
        }                                                                                                     
        )  
   (                                                                                                     
        buildPythonPackage rec {                                                                              
         pname = "trollimage";                                                                                
         version = "1.26.0";                                                                                 
         format = "wheel";                                                                                   
                                                                                                 
         src = fetchPypi {                                                                                   
           inherit pname version format;                                                                     
           dist = "cp311";                                                                                   
           python = "cp311";                                                                                 
           abi = "cp311-manylinux_2_17_x86_64";                                                              
           platform = "manylinux2014_x86_64";                                                                
           sha256 = "";                                                                                      
         };                      
       doCheck=false;
)

This will result in a 404 error when building the direnv.

direnv: loading ~/tmp/foo/.envrc                                                                                                                                                                                                            
cat warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
these 3 derivations will be built:
  /nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv
  /nix/store/8zwpr3ryr28r5i72gsjxvdc8247jb7lp-python3.11-trollimage-1.26.0.drv
  /nix/store/zckp5h97b1acm0s31zd9rngchnq46j98-python3-3.11.10-env.drv
building '/nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv'...

trying https://files.pythonhosted.org/packages/cp311/t/trollimage/trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
error: cannot download trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl from any mirror
error: builder for '/nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv' failed with exit code 1;
       last 7 log lines:
       >
       > trying https://files.pythonhosted.org/packages/cp311/t/trollimage/trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (22) The requested URL returned error: 404
       > error: cannot download trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl from any mirror
       For full logs, run 'nix-store -l /nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv'.
error: 1 dependencies of derivation '/nix/store/8zwpr3ryr28r5i72gsjxvdc8247jb7lp-python3.11-trollimage-1.26.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/zckp5h97b1acm0s31zd9rngchnq46j98-python3-3.11.10-env.drv' failed to build

When looking at the Trollimage pypi page you will find that the packages are not named
trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl
but
trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

According to the aboved named packaging user guide this is valid, according to pep427 used by fetchPyPi it is not found.

2 Likes
4

It was reported here fetchpypi fail with curl 404 · Issue #369009 · NixOS/nixpkgs · GitHub

1 Like