@hsjobeki what you’re working on with RFC 145 and all its implications is a very big, heroic endeavor, and I have a lot of respect for that. So the following is my way of trying to be constructive.
Ideally that would be the case, but in practice I don’t find that to be true: Things that are not documented (or badly documented, or hard to find) effectively don’t exist for all but the few brave souls that dig into the source code. Conversely, things being simply put side by side can leave an impression they are somehow equivalent and it doesn’t matter what you do with them. We can use these signals to our advantage and nudge people to do things that make their lives and our long-term maintenance easier.
In any case, we probably only disagree on superficial aspects. Let’s set them aside until more of your work materialises upstream.
The “Nix ecosystem” is not well-defined, and my gut feel is that it’s futile to pretend it is or to try forcing it.
I deem it more fruitful to instead strengthen separation of concerns, because it promises to
- Make it easier for new users to understand the system (i.e. build a correct mental model of it) and hence use it more effectively, in a smaller amount of time
- Make maintenance more tractable. When problems and their solution mechanisms are not tightly scoped, we tend to get into system-wide architectural debates that cost time but don’t turn out concrete improvements.
tl;dr: Please go wild on the Nixpkgs manual
I don’t feel like I have any authority over the Nixpkgs manual, as I barely work on it and don’t know it very well, so my opinions on that subject are not well-informed and largely a matter of taste. If I were maintaining it, I wouldn’t want to duplicate documentation that’s in the responsibility of Nix maintainers; it’s a source of more work and confusion for users. Noting which library functions are aliases to builtins (or could be, since they’re often conditional on the Nix version) would be really helpful @iFreilicht, but not a concern for this rendering feature - that just has to be written into each function’s docs.
But again, I’m not the one to tell anyone what to do. Feel free to propose changes as you see fit and get buy-in from whoever maintains that code!
However I do maintain nix.dev and the Nix manual, and here I have a clear vision:
-
In nix.dev don’t lump together things that live in different components.
The goal is to communicate very clearly the distinction between each of the major ecosystem components, primarily to ease onboarding and help people decide what to focus on and where to go when they want to learn more.
Also nix.dev should mostly serve as guidance, i.e. be a thin wrapper around actual documentation. Anything else simply doesn’t scale. We the documentation team, and I personally, are already constantly being drawn into more responsibilities. If we don’t set boundaries we won’t finish anything and won’t be able to help anyone.
Therefore I’d say: No, put function documentation in the reference manual for the component where they are defined. We’ll gladly provide pointers to that from nix.dev.
around -
In the Nix manual make sure that Nix, its constituent parts, and their maturity and support status are represented correctly. It should be the single source of truth on everything that is specific to Nix.
Therefore I’d frown upon
builtinsdocumentation being worked on somewhere downstream. Please just make PRs to Nix. I know it’s not fun or easy most of the time, but it’s getting better literally by the day.Somewhat related: We won’t get far with the RFC 145 doc comments syntax for builtins, as those are implemented in C++. We’re lacking the infrastructure to produce automated
builtinsinterface documentation, and I’d appreciate help in that area a lot.
This is always an option and you’re free to put effort into whatever you like.
tl;dr: I still recommend incremental changes to upstream instead
I don’t think more new kinds of anything is beneficial to the broader ecosystem though, because:
- The bigger the new kind of thing grows,
- the larger the diff to upstream,
- the more upstream appears to be deficient or unmaintained,
- the more (especially new) users perceive the ecosystem to be an uncoordinated mess,
- the less likely any of that ever gets fixed.
I’ve seen so many examples of that in the past that I lost track. All of them ran out of steam eventually, leaving the ecosystem in a state I consider worse than before: with more half-finished ideas scattered all over the place.
Upstream is the only thing that will survive all other efforts and still have the largest impact because that’s what new people start out with and take for granted. Yes, it’s operating under constraints that everyone I know rightfully considers very annoying at best, with many hard-to-predict barriers to contribution. But that’s the commons. I have no better answers than “make smaller changes, they’re easier to produce and to accept”. Let’s fix things step by step.
I fully agree on the first part, assuming we’re still talking about the Nixpkgs manual. I don’t think there is all that much effort being put into improvements (especially in comparison to other things being worked on), but the effort needed to get it into a shape I’d like would indeed be substantial.
But why not instead help to make it stop sucking? It’s mostly not fun, it’s not short-term rewarding, but this is where small changes will help immediately and add up over time. It’s a process that will quite predictably lead to good results, if ever so slowly.
Up to recently it wasn’t even easy to find what could be done concretely, but for a few months now we at least have a first stab at an overview.
I don’t know anyone who invests major time into the Nixpkgs manual. (Which doesn’t necessarily mean much, but I should know, right?) Doing things in parallel until they’re better than the original and can be used as a drop-in replacement is exactly the kind of far-reaching change that I don’t expect to come to fruition within a time span I’m comfortable planning for.
To get back to constructive feedback on what you’re trying to achieve: You’re tackling a real, far-reaching problem and I think you’re doing great. Expect the process to take a Long Time™ to come to conclusion. Showing a proof of concept is excellent. Don’t get discouraged if people don’t agree with today’s version of your vision. That’s too much to ask and you’ll need buy-in for the whole package to get it in. Again, making the package smaller will make it more likely.
Therefore I really recommend you to focus on getting incremental changes merged into Nixpkgs that don’t depend on the big outcomes such as the RFC to land or the Nix language evaluator to implement it. That will help you build up mindshare and increase the bus factor you’ll need to get the big chunks of upstream work that are still to come merged. Find your people, don’t rely on me or anyone already entrenched in something to approve or help in substantial ways. As @zimbatm implies, that would only slow down both sides.
I’d love to see you taking ownership of library documentation in Nixpkgs, and ideally share it with a few others. Where you really need approval for some reason: I definitely prefer changes the full extent and consequences of which I can gasp in a few minutes, and likely this is true for many others who are busy enough with their own area of responsibility.