It can be frustrating to try to determine which versions of things are available on a currently installed channel, despite the fact that frequently we do maintain multiple versions and record those versions in their derivation.
It’s also frustrating that so much of the power of Nix is locked up as very specialized knowledge. It’s amazing and incredible to go to IRC and get responses there about how to do a thing, but kind of a let down that it’s unclear how to figure those things out for one’s self.
I have vague thoughts about how to address these things, and multiple scales, but I wanted to bring the discussion here and see what other folks think.
I fully agree that there are many (seemingly) hidden features and workflows buried within Nix. People who are adapt at using Nix know of these are keep their knowledge up-to-date, but manuals and wikis are often somewhat lacking, outdated or do not represent the ‘right path’.
I’m feeling the number #1 issue is the command-line tooling. Pre-2.0 the tools were a bit inconsistent. Nix 2.0 improved a lot. The new commands are more consistent. However the new commands also seem to have less features and are largely undocumented. It is also often unclear when to use old commands or new ones (nix-build --show-trace is more useful than nix build. nix-env is still using the old standards)
In addition, tutorials and blogposts often recommend using a method that is (imo) not right for Nix. For instance import <nixpkgs> is often being referred to. I’m of the opinion that Nixpkgs pinning should be very much ingrained into the workflow of a Nix user. Otherwise reproducability will suffer. Whether that is reproduciblity of builds within a team or even the tutorial/blogpost itself(!). If the methods shown in a blogpost do not work after 4 years anymore it is not reproducible! Basically people need guidance on what the right workflow is with Nix. I really like using direnv with a default.nix in a repository that contains a pinned Nixpkgs, but I do see that often and also do not know whether this is actually the ‘right path’.
Both of these add to the pains of a starting Nix user I think. Just updating the manual on what commands there are isn’t enough. There should be some guidance on a shared workflow. Now you can get different ideas from different sections of the manual and different blogposts, but you still don’t know where to start exactly.
That is at least my idea. I still haven’t been able to implement Nix into the company I’m working, but I’ll keep trying to make things easier so that eventually people will weigh the benefits over the added complexity.
I think there’s a lot to be said about the Nix 2.0 toolchain - the stuff that’s there is great! The features that are missing are tragic. The fact that we need to fall back to 1.11 tools sometimes leaves me often confused about when I can use nix <something> versus an old tool. (Is there a 2.0 equivalent of nix-env for installs?)
So! I’d love for there to be a clear direction about where Nix-the-tool is going. I think I’ve understood @edolstra’s intention as nix eventually replacing nix-*, right? It seems like contributions there have a lot of value, but I’m honestly not sure how to get started.
One suggestion that came up in follow-on discussion with Dave was the idea of a nix shell-template command - essentially “I’m building a Python project and would like a python shell.nix.” Right now, this is well into the territory of implicit knowledge. I’d frequently go to IRC and ask, and someone has a git repo to show me and… But wouldn’t it be better if in nixpkgs there were a templates/ directory, and a nix tool to wrap around a nix build to produce a working shell.nix for … whatever dev environment you’re trying to set up. (They were trying to get a cabal build going and wound up blowing away their dev boxes in frustration.)
I think the direction another discussion here (Strong opinion: Library packaging - #7 by nyarly) is taking goes to this as well: unifying how Nixpkgs handles outside libraries makes the whole system easier for newcomers to understand, and makes it easier for intermediate users to build mental models of.
I also wonder if the Wiki and blogs ought to be considered antipatterns, to some degree. Clear centralized documentation would do an awful lot to clarify Nix workflows, and the manuals and man pages would seem to be the right places for that.
The inconsistent CLI is definitely a problem but I don’t think it’s the biggest issue.
The biggest issue is that it’s hard for people to read the nix code and understand what is going on. It took me a looong time to be able to do simple things. Now that I know how to code nix it’s great but it wasn’t easy at the start. Not as easy as writing a Dockerfile for example.
Another observation is that most of the time I am forced to open the nix code to figure out what it’s doing. In other languages I can look at the auto-generated library dock and just look at the interfaces, this isn’t the case with Nix because there is no auto-generated doc for Nix code.
The language itself also reports a lot of hard-to-understand exceptions. It took me a long time to figure out how to debug errors like error: value is an integer while a set was expected and it’s still not great. I really would prefer something like: variable X at position 3:4 is an integer while a set was expected with a nice squiggly underline of the code.
There is also a lot of magic happening in the default builder which makes things easy but hard to understand when it breaks. For example adding cmake to the build inputs creates some magical new steps.
In general there are quite a few concepts and paper cuts to understand. What is a channel, why can’t I list the channel when it’s clearly finding one. Why do I have to use nix-build to build my package but nixos-rebuild to rebuild my system configuration. What are profiles. What are GC roots. Why do I still have the entries in grub after garbage-collection. Why can’t I manage my user dependencies decoratively.
I think these issues are hard to solve because usually contributors are mainly interested in solving their specific problems, which is completely fine to be clear, but doesn’t help address those bigger issues.
Why does nix-env see my channels but nix-shell doesn’t?
I agree about the diagnosis there: contributors are solving their problems and not “the” problems. But I think increasingly there are Nix users for whom the problem is “how do I get other people to use Nix?” And it takes a larger view to see that solving the problems you enumerate are the way to do that.
So, how do we get to that larger view? I mean, I suppose the Nix conferences are the way to approach that, and I’ll admit that I’m chauvinistically ignoring their existence ;).
Talking is the right idea, that’s why I thought that the Discourse instance would be a good idea. Same for the RFC process really. I would also like us to start forming working groups to address specific areas as the domain is so large.
One big difficulty is that to find good design solution it requires the contributors to be heavily invested; it’s only after some time that you loaded enough context to have a clear view of the design space. And those people tend to then get distracted by PR reviews since they are the more knowledgeable. The best remedy to that is to simplify as much as possible so more people can get up to speed.
One big difficulty is that to find good design solution it requires the contributors to be heavily invested; it’s only after some time that you loaded enough context to have a clear view of the design space. And those people tend to then get distracted by PR reviews since they are the more knowledgeable. The best remedy to that is to simplify as much as possible so more people can get up to speed.
I think it’s also that many design trade-offs are package deals: there are two good options with different weaknesses, and a compromise is not always better than any of the options in any sense. This also makes design space exploration quite costly.
And also by the time you become an expert, you are now used to all the crazy things and they seem normal to you
For example why do we name our dependencies buildInputs, nativeBuildInput and propagatedBuildInputs when all the other package managers name those dependencies, buildDependencies and runtimeDependencies ? They don’t map exactly but it makes it harder for new-comers to package new things.
