Overlay with overridden fetchurl to access resources

Help
1

Hi there, I want to set a netrcPhase for all fetchers specifying credentialed access to resources. To do this, I have tried defining an overlay as follows:

final: prev: 
let
    newfetch = inputs: prev.fetchurl (inputs//{ netrcPhase = "...";});
in
{
    fetchurl = newfetch;
}

Unfortunately this gives me an infinite recursion error. I think this is because fetchurl has a functor attribute but I am not sure. Any help is much appreciated!

EDIT: The netrc file needs to be generated on the fly using temporary credentials, which is why I want to do it this way.

2

It’s probably a result of what’s inside the netrcPhase depending on fetchurl somehow, directly or indirectly.

Try it with a simple string with nothing antiquoted in it and see if you still get infrec.

4

The following gives me infrec:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs";
  };

  outputs =
    inputs:
    let
      system = "x86_64-linux";
      overlay = final: prev: {
        foo = pkgs.hello;
        fetchurl =
          args:
          prev.fetchurl (
            args
            // {
              netrcPhase = ''
                cat > netrc <<EOF
                machine foobar
                        login FOO
                        password BAR
                EOF
              '';
            }
          );
      };
      pkgs = (inputs.nixpkgs.legacyPackages.${system}.extend overlay);
    in
    {
      packages.${system}.default = pkgs.fetchurl {
        url = "https://github.com/developit/unfetch/archive/refs/tags/isomorphic-4.0.2.tar.gz";
        hash = "sha256-Qwp5O4055CoHFx+ifJE0gcVlfeHyrg0H4Isp6XC0pkk=";
      };

    };
}
5

I’ve found the issue:

Note the dependence on netrcPhase being set. fetchurl (without cacert) is required in order to build cacert, so you get infrec.

A functional workaround should be to override cacert to use the unmodified fetchurl. (Or maybe not… it doesn’t use it directly. It uses fetchFromGitHub, which uses fetchzip, which uses fetchurl.)

1 Like
6

That’s rather surprising! I thought the lazy evaluation of nix was implemented partly to overcome exactly these problems. Is there any possible way to override the netrcPhase arg in fetchurl? Maybe I should put a PR in to promote these parameters to the top level where they can be overridden.

7

A simple, if somewhat wasteful, solution is to override fetchurl to use the cacert package from a separate nixpkgs instantiation. This evaluates without infrec in the repl for me:

(pkgs.extend (final: prev: {
  fetchurl = args: (prev.fetchurl.override {
    inherit (pkgs) cacert;
  }) (args // { netrcPhase = "foobar"; });
})).fetchurl {
  url="https://example.com/foo";
}
8

Following your example, I came up with the following:

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs";
  };

  outputs =
    inputs:
    let
      system = "x86_64-linux";
      pkgsOld = (inputs.nixpkgs.legacyPackages.${system});
      overlay = final: prev: {
        fetchurl =
          args:
          (prev.fetchurl.override {
            inherit (pkgsOld) cacert;
          })
            (
              args
              // {

                netrcPhase = ''
                  cat > netrc <<EOF
                  machine foobar
                          login FOO
                          password BAR
                  EOF
                '';
              }
            );
      };
      pkgs = (inputs.nixpkgs.legacyPackages.${system}.extend overlay);
    in
    {
      packages.${system}.default = pkgs.fetchurl {
        url = "https://github.com/developit/unfetch/archive/refs/tags/isomorphic-4.0.2.tar.gz";
        hash = "sha256-Qwp5O4055CoHFx+ifJE0gcVlfeHyrg0H4Isp6XC0pkk=";
      };

    };
}

which avoids the infrec error. TYSM!

9

I updated the following to add our use case: Enterprise - NixOS Wiki
and created an issue on nixpkgs fetchurl: infinite recursion when used in an overlay · Issue #428266 · NixOS/nixpkgs · GitHub

1 Like
10

That’s the old, unmaintained wiki, please update the article at Enterprise - NixOS Wiki

2 Likes