Nixlang: how do you find all uses of a declaration?

[amjoseph]

@sternenseemann

Not just that: If you had a hypothetical strict Nix evaluator that could evaluate a nixpkgs worth of code instantly, it wouldn’t be able to evaluate nixpkgs, because it relies on laziness for building package sets.

I was not proposing to change the nix binary to use strict evaluation while leaving all the nixpkgs code exactly the way it is. Obviously that would break lots of things in many different ways.

Let me be more explicit; I should have said:

@amjoseph

My understanding here is that the reason nixlang chose lazy evaluation semantics is that it lets you avoid having to parse every single file in all of nixpkgs simply in order to evaluate one expression.

Back to your comment:

@sternenseemann

it wouldn’t be able to evaluate nixpkgs, because it relies on laziness for building package sets.

Yeah, I’m still somewhat undecided about this.

On one hand, I’ve spent the last week digging around in the internals of make-derivation.nix and am totally impressed with the way it approaches cross-compilation (specified using natural deduction no less!). The nested fixpoints-within-fixpoints (at least three levels deep, maybe more) involved are clearly an excellent showcase for a situation where a lazy-by-default language is a big win. It’s also exceptionally clean, organized, and well-documented code (this applies to pkgs/stdenv/* in general). Trying to rewrite this in ML with thunks would produce a fragile mess; the flow control is implicit and ought to be implicit.

On the other hand I spent most of this morning fighting with nixpkgs trying to make it not “encounter infinite recursion” when curl depends on file (this is a longstanding upstream dependency via ./configure which nixpkgs has been ignoring). This experience has made me less enthusiastic about the lazy-by-default situation. The boilerplate “Note: this package is used for bootstrapping fetchurl” slapped onto so many packages is a sign that maybe implicit laziness is causing headaches here.

@sternenseemann

you really have to think of it as more of a design decision (at least at this point): We are actively relying on this property

Yes, I know that it’s a design decision. And I know that it won’t be changing anytime soon.

But I think it’s helpful to understand what the truly important uses of this decision are. Here’s my list so far:

• Lazy parsing of .nix files, so we don’t need 24,000 calls to fopen() for every eval
• Internally within stdenv, especially related to bootstrapping. This includes dependencies between multiple invocations of nixpkgsFun (stages.nix, cross-compilation, etc).
• Between packages at top-level.nix – i.e. between the individual packages of a single invocation of nixpkgsFun

The first point is sort of unique to nixlang; most lazy languages (eg Haskell) don’t share this property.

I’m sold on laziness for the second point.

I’m not entirely sold on the last point. Leaving the inter-package dependency graph implicit is definitely tempting, but there are a lot of footguns in there like membership in the “fetchpatch clique” which is expressed in prose documentation rather than in code. When things go wrong it can be really hard to figure out where the infinite recursion is coming from.

None of this is going to change in the next decade, but understanding the “essential uses” of laziness is helpful to me in understanding the “zen of nix”.

@sternenseemann

This property already causes trouble in some stdenv bootstrapping related portions of nixpkgs

Is there an example use case for this? I get the undefined // { x = 3; } example, but do people actually write code like this?

@sternenseemann

CI only exrpessions are to be found in the linked ofborg repository. Expressions that define Hydra jobsets are usually tracked in the nixpkgs repository, indeed. By convention these are called release*.nix .

Ah, thank you so much. I assumed that release*.nix was just for making releases. Now that I understand that it is really more of a “this is what hydra does.nix” it makes more sense. Thanks!

@sternenseemann

One concret step towards this could be to make cross the normal entry point

I absolutely agree with this, 100%.