Where is the fetch information stored in a derivation?

Help
1

Hi

Still working on How to get a list of all URL used in nixpkgs? to be able to download sources to build derivation locally.

I’m trying to find a workaround to download patches because of https://github.com/NixOS/nixpkgs/issues/186831 . I’m a bit stuck right now, I can’t figure where I can get the “fetcher” used for a derivation.

In the nix file, I can see fetchpatch used to fetch “most” of the patches, but I’d like to get this information from the patch derivation itself (or is it in the package requiring the patch? can’t find either in those)

:pray:t3:

1 Like
2

fetchpatch is just a wrapper around fetchurl that does some normalization in the postFetch phase. To get the URL of the original files you can make use of this:

drv: lib.concatLists (
  builtins.map (patch: patch.urls) (
    builtins.filter (patch: lib.isDerivation patch) (drv.patches or [])
  )
)

However, you still need to replicate the normalization done in postFetch which also depends on arguments passed to fetchpatch! This makes what you have in mind very difficult. Because the output hash pins the normalized patch, just downloading the original file and importing it to the store won’t help you here.

2 Likes
3

Thanks! You are confirming my worries :sweat_smile:

I started to write some boiler plate to my perl script downloading the tarballs, to run postFetch on the file, so far I still get a different hash value but this could be possible… I forked the find-tarballs.nix file into a find-patches.nix that only return files with a postFetch step.

I’m currently a bit puzzled about the hash difference because it seems I’m running the exact same postFetch :woman_shrugging:t3:

Here is the code for curious :slight_smile:

    if ($val->{postFetch}) {
         my $TMPFILE = `mktemp /tmp/patch.XXXXXXXXXXXXXXX`;
         chomp($TMPFILE);

         open(OUT, ">","/tmp/download_patch.sh");
         print OUT "#!/usr/bin/env nix-shell\n";
         print OUT "#! nix-shell -i bash -p patchutils\n";
         print OUT "curl -L -o $TMPFILE $url\n";
         print OUT "out=\"$TMPFILE\"\n";
         print OUT "$val->{postFetch}\n";
         `sed 's,.*/lsdiff.*,lsdiff \\\\,g ; s,.*/filterdiff.*,filterdiff \\\\,g' -i /tmp/script.sh`;
         `chmod +x /tmp/download_patch.sh`;
         `/tmp/download_patch.sh "$url" "$TMPFILE"`;
         my $return = `nix-instantiate --eval -E 'builtins.fetchurl { url = "file://$TMPFILE" ; sha256 = "$val->{hash}"; }'`;
         print $return;
    } else {
        my $return = `nix-instantiate --eval -E 'builtins.fetchurl { url = $url ; sha256 = "$val->{hash}"; }'`;
    }
1 Like
4

I got a bit further, most patches download fine now. My issue was to use a different patchutils version, in fetchurl the version is pinned, see https://github.com/NixOS/nixpkgs/issues/25154

5

It’s now working well, I had to use export LC_ALL=C in the script to prevent the diff hunks to have a different alphabetical order :raised_hands: