`nix flake show`, not showing

Help
1

Hello all,

I have a situation where I have a list of package names, and I need to go through this list, and build each package, before ultimately adding these all to the nix flake packages attrset. Some of these packages have dependencies on one another, so the list starts with the package with the fewest dependencies,and proceeds through to the package with the most:

    getAllDerivations = pkgs: packageNameList:
      let
        fold_f = acc: pkgName:
          let
            thisPair = pkgs.lib.attrsets.nameValuePair
              pkgName
              (acc.accPkgs.callPackage ./${pkgName}/${pkgName}.nix {});
            thisAttr = builtins.listToAttrs [ thisPair ];
            newPkgs = acc.accPkgs.extend (self: super: thisAttr);
            newAttrs = acc.accAttrs // thisAttr;
          in
            { accPkgs = newPkgs; accAttrs = newAttrs; };
        all_folded = builtins.foldl'
          fold_f
          { accPkgs = pkgs; accAttrs = {}; }
          packageNameList;
        folded_pkgs_only = all_folded.accPkgs;
      in
        folded_pkgs_only;

I can then get a full attrset of packages by calling:

getAllDerivations 
     pkgs 
     [ "pkg_a" "pkg_b_which_depends_on_a" "pkg_c_which_depends_on_a_and_b" ]

This all works nicely, except that nix flake show does not work here - it shows

error: Please be informed that this pseudo-package is not the only part of
       Nixpkgs that fails to evaluate. You should not evaluate entire Nixpkgs
       without some special measures to handle failing packages, like those taken
       by Hydra.

I’m aware that nix flake show has trouble sometimes with packages that use code generation, like cabal2nix, but as far as I can see, my code above doesn’t use any code generation.

This isn’t exactly a deal-breaker for my use case, since the packages all work fine, but the flake is being used within the organization and it’s actually quite useful for people to be able to do nix flake show to see what’s available.

Could anyone give me some guidance as to how to alter this code to have nix flake show work, or give me some rough idea of what idiom I’ve used here that has stopped it from working?

2

You’re returning the entire nixpkgs attrset from your flake as packages, it seems. Did you mean to be returning all_folded.accAttrs instead of all_folded.accPkgs? nixpkgs uses legacyPackages precisely because the structure of pkgs doesn’t fit the nix flake schema’s requirements for packages.

I’m also a bit concerned by how you’re structuring that fold, from the perspective of overlays. I would be much more comfortable if you accumulated the overlays first, got a final pkgs value, then extract all your own attrs from it. And ideally, you wouldn’t repeatedly extend, as it’s a fairly costly operation. Accumulate a single big overlay, then extend once, I’d say.

3

Yes… you’re absolutely right - I did mean to return accAttrs rather than accPkgs there.

With the overlays/packages, if I understand you right, you’d recommend looping such that no overlays are involved, and the callPackage line therefore looks something like:
(pkgs.callPackage ./${pkgName}/${pkgName}.nix acc.accAttrs);

4

Well, that would not allow your packages to depend on one another, so no. More like:

    getAllDerivations = pkgs: packageNameList:
      pkgs.lib.getAttrs packageNameList (pkgs.extend (self: super: 
        builtins.foldl' (acc: name: acc // {
          ${name} = self.callPackage ./${pkgName}/${pkgName}.nix {};
        }) {} packageNameList
      ));
5

I think that both of these approaches (accumulating the overlays first, or accumulating the packages first) work. For example, this seems OK:

    getDerivations = pkgs: packageNameList:
      let
        fold_f = accAttrs: pkgName:
          let
            thisPair = pkgs.lib.attrsets.nameValuePair
              pkgName
              (pkgs.callPackage ./${pkgName}/${pkgName}.nix accAttrs);
            thisAttr = builtins.listToAttrs [ thisPair ];
          in
            accAttrs // thisAttr;
      in
        builtins.foldl'
          fold_f
          {}
          packageNameList;

    overlay = self: super: getDerivations super [ "pkg_a" ... ];
6

Well that one avoids needing overlays at all, which is even better, really. It does unnecessarily force the list to be topologically sorted, though. If you passed the final result to callPackage instead of the intermediate accAttrs, that restriction would be lifted.

Also, you’ve gone through a rather roundabout method to create { ${pkgName} = pkgs.callPackage ./${pkgName}/${pkgName}.nix accAttrs; }. nameValuePair and listToAttrs are just complicating things here.

You should probably also use self, not super, for the overlay. It technically makes no difference which one you use, since callPackage will find dependencies through self regardless, but super gives the false impression that you’re truly using the “previous” version of pkgs (that’s not really what super is).

7

Thank you, that’s helpful advice. { ${keyName} = value } seems like a good idiom.