A glossary is necessary both for new users dealing with the terminology and intent of conventional phrases as well as for those writing the documentation so that terminology can be kept consistent and well defined throughout.
Case in point is that following the completion of the “hello.nix” packaging tutorial I did not “see” a package. Specifically, instead of one nix file there were two, default.nix and hello.nix. Alternatively, I suppose the directory tree of ./result/ may constitute the package though I did not expect a directory tree.
Upon reflection, I think the package should probably have been ./hello/default.nix or hello.nix without the effort to conform it to the current convention in Nixpkgs which has its own unrelated reasons. In particular, the use of -A hello made no sense at this point since default.nix already refers to hello.nix.
What exactly do you mean by that? As described in the note on what we mean by “package”, the expression in hello.nixis the package (recipe).
Indeed, it’s also the package (artifact). Admittedly the terminology is not great. @tomberek proposed uniformly calling the expression a recipe in his 2024 FOSDEM talk, and I agree with that, but doing this consistently will be quite an undertaking. Why did you not expect it to be a directory tree? Maybe we can improve the introductory note to reduce the surprise.
Well, yes, there’s this cumbersome aspect about Nixpkgs conventions, but the reasons for separating recipe expressions from their collections are not that arbitrary: it simply helps with organising code. We decided to follow Nixpkgs conventions because that’s what allows you a somewhat seamless ramp-up to upstream your own packages eventually. There are a few (not necessarily exclusive) alternatives:
Stuff everything into one Nix file that only returns the derivation
This could potentially be better for a tutorial, because it involves fewer steps. But it doesn’t allow adding a second package, as we do in the very next section. You’d have to do a bit of refactoring then, which means unproductive, cognitively taxing steps in a learning situation that already binds most available attention.
Don’t introduce an additional file and run nix-build -E 'with import <nixpkgs> {}; callPackage ./hello.nix {}'
This is a common pattern for doing things quickly, and definitely worth mentioning at some point, but not necessarily here. It has two downsides: It’s a cumbersome invocation, and you’ll almost certainly have to download all dependencies again once you pin Nixpkgs in a file.
Make a separate tutorial just for hello and start from scratch for the next thing.
But eventually we’ll have to introduce the callPackage pattern anyway. In any case, having smaller tutorials is probably better.
Explain in more detail why we’re doing it this way.
This is not necessarily the job for tutorials, but could be done in a lightweight way here. Splitting things up into smaller chunks would also decrease the cost of adding even more information to a single tutorial.
Reorganise the introduction around using nix-init.
That may get people started a lot faster, but it also introduces yet another layer of magic, which defies a lot of the purpose of skill-building with the tutorials. I think we should strive get learners operational quickly, but also equip them with an understanding of what’s actually happening on the way, as far as possible.
@gdennie how did you perceive it? What would have helped you to avoid confusion?
Thanks for a great response and a quick one as well.
“Stuff everything into a single Nix file that only returns the derivation” seems a much better the way to go.
Presently the relationship between nix-build and package is confused my the intercession of default.nix. As such, its presence implies that default.nix is the actual package definition that is merely assisted for convenience by hello.nix. However, since default.nix is a keyword, the arbitrarily named containing directory must therefore be the package. In particular, as a self contained unit this directory can be relocated, archived, and its contents reconstituted by simply executing nix-build, no assistance required.
As far as Nix is concerned, a package is simply a data type, like an array in another programming language. You might provide an array to another programming language by deserializing it from a file, or by evaluating an expression, or by any number of other methods; once it’s been computed, it’s an array regardless. Same for packages in Nix. One file, two files, relocatable directory, Git repository, expression passed to the nix-build command—none of that matters. If the result of the computation is an attribute set containing the right attributes, you made a package.
Maybe we ought to be calling a directory containing a default.nix a ‘project’ or something, and say that a project can define multiple packages. (But that might be confusing because Nixpkgs contains very many ‘projects’.)
And default.nix isn’t a keyword or anything like that; it’s just a naming convention like index.html for websites.
I wouldn’t exactly agree with that, and to me this description mystifies more than it explains. In fact, the glossary entry for package pretty much says everything there is to say about packages. It’s a very use-case-oriented notion, and only for historical reasons entangled with how the Nix-language-side of derivations is implemented.
The fact alone that “package” can mean two different things is already problematic. It’s either an attribute set with a superset of attributes of “the result of builtins.derivation applied to some argument”, or the directory tree that is produced by realising the “store path of an output of a derivation”. In the nix.dev tutorial we actually use a third meaning, namely “an expression that evaluates to a package (attribute set)”. @rhendric notably these are not interchangeable: It’s always expression → attribute set → directory tree. (Although you can read from a computed file to obtain an expression, which we call “import from derivation”, but that’s something else again.)
We’re still in the process of untangling all that, so the confusion is definitely warranted in my opinion.
Sure, this is somewhat imprecise (though not so different from calling both the term [ 1 2 3 ] and the value to which that term evaluates a ‘list’, and this punning doesn’t usually cause much confusion). But I actually think OP was getting tangled up in a fourth concept, involving the source on-disk representation of the expressions that evaluate to attribute sets that get realized to directory trees.
It’s pretty common for programming languages to define organizational constructs based on their on-disk representations—a ‘module’ might be defined to be a single file with a certain syntax, and you can’t have multiple modules in a file, or represent a module in any way without creating a file, or manipulate a module at run time. I want to be clear that a Nix package is not like that.
That glossary is rather anemic and does not clarify what a package attribute set is, despite the term being a link.
The use of the term “keyword” was to indicate that default.nix has specially meaning to the operators/commands in Nix and is required for certain functionality and should be avoided otherwise to prevent confusion.
It would be very useful to have an enumeration of the varied expected attributes sets “structures” produced from or expected by the various functions/command. For instance, the attribute set, Derivation, produced by the function mkDerivation.
Such an enumeration would be a great addition to the documentation, tutorial assist, and aide to future development and on-boarding.
I can’t tell if you saw the definition of ‘package attribute set’, immediately below ‘package’, and consider it to be too vague to be useful; or if you missed it because clicking the link probably didn’t scroll the page.
I actually missed the definition of package attribute set occurring beneath and followed the link…
Still, the definition is rather vague. It seems to be saying that a package attribute set is merely an attribute set with the attribute and value pair, type = "derivation". I am now wondering if all attribute sets come with this type attribute with a string value announcing their type.
With respect to the special nature of, default.nix,
magic values should generally not be treated like regular values since their presence triggers behavior.
Where did you get this idea?
Sorry, I do not have a source, perhaps there isn’t one. Nonetheless, while keywords are generally language structuring such as if, then, for, etc; in this particular case I would also consider the filename, default.nix, to be a kind of keyword transforming its containing directory into something visible to Nix operators.
I’m trying to dissuade you that default.nix is magic. Believing that default.nix is magic might help you out when following tutorials but is likely to get in the way of understanding what actually happens when you ask Nix to do things for you. The name may be special, but think of the ways that http://.../index.html, 127.0.0.1, and /dev/null are special. Any program that works with these things will find that they can be accessed just like any other URL, IP address, or Unix file path. They don’t represent any sort of special syntax; they probably don’t get highlighted specially by any text editor; and if you want to, say, mount your devtmpfs to some other location and access the null device from there instead, that’ll work just the same (though programs expecting it to be at /dev might not). They’re only special because of the ecosystem that surrounds them, not because they have unusual behavior. default.nix is, in this way, special but not magic. It doesn’t unlock any functionality you couldn’t get by referencing a file directly. It’s just the name the Nix tools look for when told to import a directory instead of a file. It lets you make your paths a little bit shorter. That’s all.
I do understand that default.nix is not special and It’s just the name the Nix tools look for when told to import a directory instead of a file. However, its presence consequently makes a directory path function like a path to a nix file as far as nix operators are concerned.
In any event, I don’t have a big issue. It is merely my preliminary categorization/model and subject to evolution.
