Just came across an old talk by Elm’s creator, Evan Czaplicki, reasoning about UX, and mass adoption, and how to pitch things. First of all it is just a good talk, rather pleasing to listen to. But more importantly, it brings up points and examples that transfer well to the work that Nix community is doing and are generally good to keep in mind:
…you can actually get something going in very little time without getting hit with a ton of concepts
…
…he didn’t have to read a tutorial, he didn’t have to read a paper, he just looked at the pattern and added a thing in
“a javascript library for building user interfaces” - good, I know javascript, and I want to build a user interface
…
vs “an advanced purely-functional programming language” - do I want advanced, do I want pure-functional?
Cool, is this to provide tooling for working with nixpkgs repo? I figured to do nix develop github:blaggacao/nixpkgs-devshell but I’m not sure what follows. Oh and please do elaborate on templating if you would, I didn’t quite catch that
To the original point about doing something that otherwise can’t be done, I sometimes think the major payoff for Nix is taking the complexity past the point where in other situations you’d have to compromise or be unable to manage it. But this is not the best pitch for a newcomer.
As I evangelize, the features initial uses seem to love are things like setting up nix-shell/common dev environments for projects that JustWork. Or being able to reuse caches for that project.
The initial problems they run into are invariably: installing on OSX is finicky. How do I add some other random python package? Why can’t I pip install?
One thing that is a positive sign: I more often see random projects that have a “.nix” lying around. We can perhaps do a better job supporting and encouraging this, pointing it out, and providing assistance.
I personally think that a lot of nixos users get to the project
because they broke the OS (like ubuntu / arch) too often
→ they depend on a system that works (and cannot be broken too easily)
it is about control and transparency
pkgs
services
licenses
tooling like nix-tree
flexibility like overrides
…
What you describe are ideas of starting with nix on “day 1”:
on day 2:
it is about configuration of nixos options/services (more than installing nixos packages)
user could discover mach-nix for python pypi/conda-packages
and it’s about specific versions of packages/pinning and workflow automation
to not get stuck within the power nixos provides by creating pkg template for a lot of project packages [and “only” changing version and sha] - which is fine for “one main project” but not for adhoc / poc work
and build & deployment (ci/cd) …
I saw quite same people making a fast start in running nixos but crashed full-speed into the wall of limitation of " boilerplating"/recompiling very little change not in nixpkgs repo commit
working with py pkgs other then “requests” in their python env (like pytorch) → but even with “normal” pypi pkgs in pkgs.python38.withPackages is causing too much overhead/cost (time, cpu) - e.g. when having a (Data Science) environment and adding or removing any package without to have all other pinned (what would be nonsense when not in production)
In case of python ecosystem in nixos we need to change the not acceptable workaround of pkgs.python38.withPackages to a proper and (py-package and python interpreter independent) environment build
or the python community will never adopt nixos in mainstream
I recently engaged almost full time with great fun and pretty nice emotional rewards @nrdxp and @Pacman99 on improving the day 2 story (once a user wants to start using NixOS)
I’m going to confess here, that while I have pretty strong involvement, I’m still not using NixOS for my personal use (“I’m still at breakfeast on day 2”). I think that makes me a good fit and element in that trirumvirat to hold deer the interests of my fellow day 2 users.
We put great care in creating a strong contributing and sharing oriented model, which means contribution and sharing should be as easy as it could get, even for a day 2 person, with one (1) recommended and well thought through way of doing things (a concept we applied to all sorts of things from overlays, to exports, to backports, to patches, etc.).
Although bringing in more mainstream users seems like a great thing. The flipside is there’s more expectations of having refined user experiences. I think this is eventually achievable, but the nixpkgs community really does need to setup up some proper backing for maintaining the nixpkgs that we have currently. There should really be a fulltime individual (or better yet, team) that is able to at least address the current avalanche of issues and PRs that we currently have.
EDIT: One of the nice bi-products of having a purely volunteer organization has been the emphasis on quality. There’s a high premium on making process not painful for individuals investing their free time into a project. This has caused the odd duality of having more non-unique packages than Arch+AUR, but not having the manpower to deal with the upkeep of 80k+ packages.
@jonringer , do you think it’s possible that at least a portion of the gigantic number of PR’s could be addressed by whipping up a delegation strategy for BORS? I feel like empowering maintaners to merge their own work once it’s done would be a solid motivating factor. It’s kind of a bummer when you put some serious effort into a PR, only to watch it stagnate.
The good thing about BORS is that we could delegate the merge command to the PR author, but still require up to date review by community members to actually allow the merge. Of course, there is still the problem of finding trustworthy indivduals to perform the reviews, but at least those reviewers don’t necessarily need to have full commit access, making things slightly simpler.
I think this could work for simple packages. Nixpkgs gets a little more complicated because you often need to touch files like all-packages.nix which can greatly affect other packages. But I think something like ofborg + bors could be useful.
From a security perspective though, it would still be hard to delegate just because of the potential of malice, such as a user changing the upstream to malicious source. But maybe we could have a bors operation which verifies that the upstream source is similar. Unfortunately, I’m not familiar with another package manager that has a similar paradigm. The AUR has a voting system, and most of the “ownership of verification” relies on the user, not the maintainer to be aware of risks.
I disagree. We are 2021 and should be able to design systems that can ensure a equvialent or even superior outcome compared to our current non-scalable model comparable to the image of a public servant putting on the last stamp (if person and ink happen not to be in short supply).
If we regard our local maximum as the non-plus-ultra in system design / community workflows, in my opinion that’s a complete capitulation before the challenge that lies in front of us.
At least, please, let’s not stifle attempts or motions to tackle that challenge well in advance of fully acknowledging their benefits and at least have tested out things in a prototype and learnt from it.
again, bors wouldn’t allow this unless an up to date review is in place. This means that if the author changes or addes code after the review, the bors merge would fail.
That’s a concern indeed. More importantly, is merging PRs really a problem for nixpkgs? I’m not a maintainer and I don’t know
Just as a direction for thought though, perhaps automation wouldn’t sound as such a terrible scheme, if we had means of verifying:
If upstream’s signature keys haven’t changed since last derivation (the way, e.g., google play releases developer-signed APKs, I think)
perhaps that the PR has been approved by a maintainer responsible for the changed derivation
the change is local, e.g. limited to certain derivations, no spurious effects
For that we’d need to accompany every fetch* with the URL of the author’s signature for the respective file, and we’d need a way to reliably determine the scope of change that PR introduces.
If a PR meets all listed conditions, I don’t see why it wouldn’t be eligible for an unmanaged merge?
For example, if a PR just updates a revision of, say, colmap, which, being a “leaf node”, doesn’t trigger a re-build of any other derivation in the closure - that’s local, and we probably only need to make sure colmap still refers to https://github.com/colmap/colmap and is still signed with a key we’ve already used. If the PR, however, additionally introduces modifies some scripts or wrappers, etc - the bot could lock it until a human maintainer reviews it. Then, if there actually was a large number of “update rev” PRs queuing before merging, those could be “fast-forwarded”, and maintainers would only spend their time on actual novel code
I think right now there are no facilities in nixpkgs to verify GPG signatures of src.
The Nix maintainer list allows adding GPG public key fingerprints, here is mine
However, these information are unused right now:
On top of that, maybe 100 people out of over 1500 maintainers have their fingerprints listed here. If automerging is that desirable, maybe it could be enabled under the conditions:
Only version and src attributes of a derivation were changed and nothing else.
Commit is signed with a valid key of one of the listed maintainers.
PR has one approval of someone with write access.
However, that still has the approval bottleneck which an automerge probably wanted to avoid in the first place. Also, the signature would be checked as part of CI, requiring that the CI is not vulnerable to a forged signature attack. This reliance on CI is exactly what hit Homebrew in the guts.
That’s probably too strict to be a useful criterion.
A lot of people say: “Oh, those ML languages are hard to read”.
What they are saying in fact is: “Those people who write ML languages write code that’s hard to read”.
It’s not quite the same thing, it’s not an inherent fact of an ML language,
it is that there is an unprofessional style that’s very very common.
…
The goal here is, how do we have code that last for five or ten years, that when you go to a git blame on some change, you are actually going to be pointed to something real.
…
The point is, we are writing code for like, real use in the world, and so we need a professional style.
