Nix derivation madness

7 Likes

This behavior has always felt sketchy to me because there’s no evidence that the “recipe” of the derivation even really produces the fixed output.

It’s always felt like each derivation referencing the output should need to get built somewhere and at least a signed “note” added to the store that this derivation really does produce this fixed output.

You seem to be thinking more like CA derivations than the usual FOD.

The amount of churn that we are creating in derivations was unbeknownst to me.

It’s not really “churn” - if the derivation must change because the inputs changed?

I still don’t understand why the output path is the same, but I didn’t actually read the whole nix thesis. Actually, I guess it makes sense since the path is output-addressed.

This part can be expanded is I suspect there’s no reason why the path for the derivation itself can’t be modulo either I think.

Right now it’s a CA style calculation but it’s confusing have so many now…

Interesting ! Thanks for the post and for the discovery of the word unbeknownst ! :slight_smile:

How could the drv be the same? The contents are different. It’s not like the store path itself which just cares about outputHash (and name).

The calculation does not have to be content addressable for the drv path.
It can also be modulo the fixed-output-derivations as well I think.

I don’t understand, can you say that in simpler terms without using “modulo”?
I still don’t understand why a file with different contents should have the same path (in your view).

I feel your pain with “modulo” – you should read the academic draft @Ericson2314 put up.
:smile_cat:

When calculating the outPath for a derivation we already replace out FOD with their outPath in the drv so that the path doesn’t change needlessly. (This is what gives FOD super powers for not doing mass rebuilds).

For all other inputs to a derivation the .drv path is used in the calculation of the outPath.

Now the thing I wanted to highlight, is that .drv themselves have an outPath.
(The path of the .drv file itself not the produced artifact)

This path is usually calculated as a content-addressed hash so it changes depending on all inputs which I tried to demonstrate in my blog. This causes a lot of drv “churn”.

I was contemplating, could we not also replace the inputDrvs when calculating the store path of the .drv file itself also to avoid churn.
(I suspect yes)

1 Like

I added a new graph to the blog to help (I hope – feedback welcome)

3 Likes

Yes! Almost like a weaker form of CA derivations in terms of the exposed interface (but the details of the build process would be pretty different).