Can't reference any attributes of `super` in toplevel of overlay?

Help
1

I can’t figure out if this is intentional, or if it’s just a bug. In a simple overlay, I can’t force evaluation of any super attributes at the top level of my overlay without causing infinite recursion. By way of example:

with builtins;
import <nixpkgs> {
  overlays = [(self: super:
    # trace "at toplevel, hello is type ${typeOf super.hello}"
    {
      hello = trace "in attribute, hello is type ${typeOf super.hello}" super.hello;
    }
  )];
}

Evaluating with nix-instantiate --eval test.nix --show-trace -A hello.name gives me what I’d expect:

trace: in attribute, hello is type set
"hello-2.10"

But if I uncomment the line to force evaluation of super.hello before this overlay function returns anything at all, I get an infinite recursion error:

error: while evaluating anonymous function at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/impure.nix:15:1, called from /home/tim/dev/app-customisations/nix/test.nix:2:1:
while evaluating anonymous function at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/default.nix:20:1, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/impure.nix:82:1:
while evaluating anonymous function at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:42:1, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/default.nix:100:10:
while evaluating 'dfold' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:60:27, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:117:4:
while evaluating 'go' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:63:18, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:72:13:
while evaluating 'folder' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:89:33, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:68:18:
while evaluating 'allPackages' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/default.nix:90:17, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/stdenv/booter.nix:101:10:
while evaluating anonymous function at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/stage.nix:12:1, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/default.nix:90:26:
while evaluating 'fix' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/lib/fixed-points.nix:19:9, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/stage.nix:213:3:
while evaluating 'extends' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/lib/fixed-points.nix:66:24, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/lib/fixed-points.nix:19:20:
while evaluating 'extends' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/lib/fixed-points.nix:66:24, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/lib/fixed-points.nix:66:42:
while evaluating anonymous function at /home/tim/dev/app-customisations/nix/test.nix:3:22, called from /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/lib/fixed-points.nix:66:67:
while evaluating the attribute 'hello' at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/all-packages.nix:17587:3:
infinite recursion encountered, at /nix/store/yspk8pi98sm9npdvb9riz3m3rvmwxfxw-nixpkgs-19.09pre170739.1a95a36dbf1/nixpkgs/pkgs/top-level/all-packages.nix:11:6

Is this a bug or does it make sense? I would expect that self.* would probably fail like this, but shouldn’t super be fully defined before my overlay is called?

2

For what it’s worth, referencing lib and other static attributes works, it’s only anything related to packages (this includes packages themselves, but even just getting the type of super.callPackage (a function) causes the same recursion issue.

3

The reason is because hello takes dependencies from self, so your overlay needs to be evaluated for any changes to hello’s dependencies to be known, but in order to evaluate it it needs to know what hello is because of the trace, for which it needs to evaluate your overlay, and so on.

4

So just to get things straight:

super.hello is the super definition of hello, so it’ll have all the derivation attributes set by super, and no overrides that might appear in my overlay.

However due to callPackage taking its args from the final result of all overlays (i.e. self), super.hello is really “the super definition of hello with all its dependencies taken from self”.

Is that accurate? That sort of makes sense, though I wish there was some way to break this linkage for cases when I want to say “get a pacakge from super, and use it to produce self”. I could import <nixgpks> {} fresh and use that, but that seems inelegant.

5

You could maybe take the super package and then overrideAttrs to modify it.

Or more generally, if you write { hello = …; }, anything in the definition of hello there should be evaluated lazily, so as long as hello does not somehow depend on itself, that should be fine. You just can’t evaluate hello eagerly.