FetchPyPi fails on many packets due to dots in path

The fetchPyPi function of nixpkg is using the deprecated PEP427 format.

This causes multiple packages to fail with 404 errors when trying to fetch wheels.

According to Names and normalization - Python Packaging User Guide the the package maintainer can choose a variety of names.

I’m not sure how to solve this except to overwrite the packet URL directly (is this a possiblity? my nixfoo is quiet weak)…
trying multiple formats would I think also work? If nobody has a quick workaround I’ll guess I open a issue but maybe there is a quick way of getting it to work

Packets where I got stuck:

  • satpy’s trollimage
  • matplotlib

What did you try, including code and commands, and what is the exact, full error?

What I did:
create shell.nix

{ pkgs ? import <nixpkgs> {} }:                                                                                   
                                                                                                                  
pkgs.mkShell {                                                                                                    
  nativeBuildInputs = with pkgs; [                                                                                
    (                                                                                                             
    let                                                                                                           
      pyEnv = p: with p; [                                                                                        
      numpy      
      pyyaml                                                                                                      
      pooch                                                                                                       
      xarray                                                                                                      
      matplotlib                                                                                                  
      pillow                                                                                                      
      platformdirs                                                                                                
      dask
      (                                                                                                         
        buildPythonPackage rec {                                                                                  
         pname = "satpy";                                                                                         
         version = "0.53.0";                                                                                      
         format = "wheel";                                                                                        
                                                                                                                  
         src = fetchPypi {                                                                                      
           inherit pname version format;                                                                        
           python = "py3";                                                                                      
           dist = "py3";                                                                                        
           sha256 = "abea68dcfb27793bca9073abb4927fb92c5c39735390aa6235664d1fbad848e2";                         
         };                                                                                                     
         doCheck = false;                                                                                       
        }                                                                                                       
        )       
(                                                                                                       
        buildPythonPackage rec {                                                                                
         pname = "trollsift";                                                                                   
         version = "0.5.3";                                                                                     
         format = "wheel";                                                                                      
                                                                                                                
         src = fetchPypi {                                                                                      
           inherit pname version format;                                                                        
           python = "py3";                                                                                      
           dist = "py3";                                                                                        
           sha256 = "sha256-MS/Y0UtcS/xqVRYlpoJ97ZU6vi7pUPRLhhlP4OjcZbY=";                                      
         };                                                                                                   
         doCheck = false;                                                                                     
        }                                                                                                     
        )                                                                                                     
                                                                                                              
        (                                                                                                     
        buildPythonPackage rec {                                                                              
         pname = "donfig";                                                                                    
         version = "0.8.1.post1";                                                                             
         format = "wheel";                                                                                    
                                                                                                              
         src = fetchPypi {                                                                                    
           inherit pname version format;                                                                      
           python = "py3";                                                                                    
           dist = "py3";                                                                                      
           sha256 = "sha256-KjF1znSgYQn/kwfZCiMPgSFcusmnUfTRxhlGRLggT50=";                                    
         };                                                                                                   
         doCheck = false;                                                                                     
        }                                                                                                     
        )  
   (                                                                                                     
        buildPythonPackage rec {                                                                              
         pname = "trollimage";                                                                                
         version = "1.26.0";                                                                                 
         format = "wheel";                                                                                   
                                                                                                 
         src = fetchPypi {                                                                                   
           inherit pname version format;                                                                     
           dist = "cp311";                                                                                   
           python = "cp311";                                                                                 
           abi = "cp311-manylinux_2_17_x86_64";                                                              
           platform = "manylinux2014_x86_64";                                                                
           sha256 = "";                                                                                      
         };                      
       doCheck=false;
)

This will result in a 404 error when building the direnv.

direnv: loading ~/tmp/foo/.envrc                                                                                                                                                                                                            
cat warning: found empty hash, assuming 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
these 3 derivations will be built:
  /nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv
  /nix/store/8zwpr3ryr28r5i72gsjxvdc8247jb7lp-python3.11-trollimage-1.26.0.drv
  /nix/store/zckp5h97b1acm0s31zd9rngchnq46j98-python3-3.11.10-env.drv
building '/nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv'...

trying https://files.pythonhosted.org/packages/cp311/t/trollimage/trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
error: cannot download trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl from any mirror
error: builder for '/nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv' failed with exit code 1;
       last 7 log lines:
       >
       > trying https://files.pythonhosted.org/packages/cp311/t/trollimage/trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl
       >   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
       >                                  Dload  Upload   Total   Spent    Left  Speed
       >   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
       > curl: (22) The requested URL returned error: 404
       > error: cannot download trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl from any mirror
       For full logs, run 'nix-store -l /nix/store/w6bz35cjkxk73gcdmh53w0rj4zd23n4m-trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl.drv'.
error: 1 dependencies of derivation '/nix/store/8zwpr3ryr28r5i72gsjxvdc8247jb7lp-python3.11-trollimage-1.26.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/zckp5h97b1acm0s31zd9rngchnq46j98-python3-3.11.10-env.drv' failed to build

When looking at the Trollimage pypi page you will find that the packages are not named
trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64-manylinux2014_x86_64.whl
but
trollimage-1.26.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

According to the aboved named packaging user guide this is valid, according to pep427 used by fetchPyPi it is not found.

2 Likes

It was reported here fetchpypi fail with curl 404 · Issue #369009 · NixOS/nixpkgs · GitHub

1 Like