I would like to write a shell / nix script that will build a list of URLs from all derivations that use fetchFromGithub. Basically, for every package in <nixpkgs> that it’s src is given by this function, I want to print the URL of the archive. I would like to avoid using raw and unreliable git grep / sed style commands within the nixpkgs repo.
If you are intereseted in the final purpose of this script, read the discussion of me and @ryantm had:
https://github.com/ryantm/nixpkgs-update/issues/122
@ryantm suggested using OfBorg’s ofborg/src/outpaths.nix as a reference but TBH that’s way above my level to understand.
I was hoping someone can give me a quick start as for this part of my small but ambitious project.
Thanks.
3 Likes
For finding all top-level derivations, you can probably do something like lib.filterAttrs (name: lib.isDerivation) pkgs which will get you all derivations (as an attrset; use builtins.filter lib.isDerivation (builtins.attrValues pkgs) if you just want the derivations themselves).
This won’t recurse into package sets of course. These have recurseForDerivations = true so you can probably come up with some recursive traversal that checks for that.
As for detecting fetchFromGitHub, I’m not quite sure offhand how to do that, but maybe you don’t have to; maybe you can get away with just saying something like drv: lib.strings.hasPrefix "https://github.com/" (drv.src.meta.homepage or ""), since apparently fetchFromGitHub sets meta.homepage to the repo URL. This would have the side-effect of also working for any derivations whose src is something else that happens to set meta.homepage to a github URL.
3 Likes
Thanks @lilyball for your comment, I’m kind of getting an idea of what it’ll be like. I’d really like to dive in and test these ideas. I’m sorry if I sound like a complete Noob but could you give me a minimal working example of a script that will use lib.filterAttrs? I guess it’ll be a #!/usr/bin/env nix-shell script that I’ll have to run inside a nixpkgs clone?
My knowledge in the Nix language is poor, I just know how to package basic packages and I think I know the basic syntax. I’d be most obliged even if you’ll hand me some pointers at tutorials / examples that do stuff like this.
Tutorials I’ve found when Google searching seemed too basic and only about the general syntax while reading nix code in projects like e.g node2nix seem to complicated for a starter like me.
I actually don’t typically write Nix scripts like this so I’m not sure what the best setup is for a standalone script. The simplest thing to do is probably just to write a foo.nix file that you manually invoke with nix-shell foo.nix, and have it start with something like
{ pkgs ? import <nipxkgs> {} }:
This construct means running it directly will just import your nixpkgs channel, but you can wrap it in another script that provides a separate package set (e.g. a nixpkgs git checkout) if you want, or even use the --arg flag to nix-shell to do it, like
> cd myNixpkgsCheckout
> nix-shell --arg pkgs 'import ./. {}' /path/to/foo.nix