Marketing Team: Can we present Nix/NixOS better?

  • I would like to see more emphasis on Nix/NixOS as a configuration management tool. Many of my coworkers in the last few years have grown to hate Ansible for its wacky non-deterministic behaviour, especially for large deployments. Some have gone back to Puppet. There is an opportunity to present Nix/NixOS as an Ansible/Puppet alternative, rather than as merely a Linux distribution or package manager.

This is planned, but is going to take some time before we get there.

  • All of the Kubernetes deployment tools I’ve used have kind of sucked. NixOS’s support for running k8s clusters still needs work, but I see a lot of potential there.

I can only agree with you. But this use case needs to be polished before we can promote it on the website.

  • Though nixpkgs doesn’t include OpenStack currently, it would be the perfect thing to deploy clouds with. The current standard way (from what I’ve seen, at least) of deploying OpenStack is kolla-ansible, which is a frightening mix of Ansible and Docker.

As with kubernetes, when somebody will polish the solution and document it I don’t see a reason why not to promote it.

  • Some have framed Nix as a competitor to Docker, but they aren’t really in the same space. Additionally, Kubernetes has become the standard way to deploy almost everything. That said, building Docker images with Nix guarantees a higher degree of repeatability than a Dockerfile with Alpine or what-have-you.

Building docker images with Nix is going to be features on the landing page. Very soon.

Thank you for the suggestions. I hope somebody takes on the Kubernetes and OpenStack use case.

2 Likes

As with kubernetes, when somebody will polish the solution and document it I don’t see a reason why not to promote it.

I was an OpenStack admin for a few years, so I can probably be of some help here, assuming I can find some free time.

1 Like

NixOps does a pretty good job in that regard, though it is a bit less general-purpose than Ansible since it assumes the configuration it will be managing is for NixOS machines.

1 Like

I think we need a very good default config for desktop NixOS. One that supports a lot of hardware, etc. It would be nice if users can use the default config as their primary OS as they still adapt to the Nix operating system. This idea is based on helping two people move to NixOS.

4 Likes

I think the opposite. We could make a repository like GitHub - NixOS/nixos-hardware: A collection of NixOS modules covering hardware quirks. but contains more features. For example:

  • Config specific to a laptop/machine hardware
  • Config for laptop, max battery life/balance/max performance
  • Config for gaming
  • C++ dev env, AI/machine learning dev env, Web dev env …
  • User can easily choose the features that they want by just import the config file to configuration.nix, or select in a GUI tool
  • User can contribute their config file to the repo by make a pull request.

For example:

  • A teacher could just tell his student to open nix-module-manager, select the module dev/c++ with option vscode and rebuild, then they could start writing C++ code with all environment set up by nix: compiler, editor, tool chain, lsp…
  • If I buy a new laptop, I could just install nixos, select the hardware/brand/my-laptop then install. It will solve most of problems: drivers, finger print not working, HiDPI for 4k laptop, power saving, brightness control, …

ArchLinux has great wiki but we can make it better, we can apply the fix in the wiki automatically. The idea is user only need to tell what they want, and the repo figure out what need to do.

7 Likes

https://github.com/NixOS/rfcs/pull/74

I’m committed to evolving this idea, waiting currently for some feedback from @garbas & @samueldr. Or anyone who’s got a great idea how this idea can be put to use.

Or …

… I might just ask for becoming part of the marketing team :stuck_out_tongue_winking_eye:

For my part, I am mainly interested in the process of developing a system config from scratch. My current config works and is pretty solid, but it’s built up on a bunch of ideas and structures that aren’t mine. I stole useful looking stuff from a bunch of other configs and sort of just jammed it all together until it did the stuff I wanted. This got me over part of the learning curve, but it leaves me in the awkward position of basically understanding the components, but not really being able to explain the “why” of all those choices.

For instance, I’d love to replace my Emacs overlay with emacs-overlay or rework my Haskell overlay to use haskell.nix. I think I know enough to do the former with some thought, but I have no idea if it’s sensible or even possible to use haskell.nix for a system-wide GHC or if it’s really just for projects, and I don’t know where to look to determine that.

I decided to just start reading the manuals and the pills end to end and taking notes to start tying ideas together, but the primary target of the Nix and nixpkgs manuals definitely appears to be people writing packages. This is fine and sensible, but it leaves me a bit adrift in answering the question “how do I, starting from nothing, write a quality system config that I can version control?”

The extra sticky bit is that I primarily use Darwin, so most of the NixOS documentation that might go toward answering this question is of varying usefulness. Most blog posts are about how to package such and such type of software or a recapitulation of the “how to package GNU hello” exercise, which tells me little about how to actually configure my machine the way I want it.

All of this is to say, I would be extremely interested in seeing (and even helping to write) a “Nix book” that isn’t just a manual.

Example: This is a really slick config, with a really straightforward installation and support for multiple machines and all that cool stuff. But! I have no idea what the assumptions made in this config are, what structural choices are good or just interesting, how I could write my own config like this other than copying it and tweaking. It even installs nix-darwin as part of its setup, which would remove one more manual step from applying my config on a new machine.

What is the gap in my knowledge that lets me read this config, basically understand what it’s doing, but not be at all certain where to start to make my own?

Parts of these could (and perhaps should) be addressed in the documentation of the projects themselves, but some kind of unified “Configure your machine with Nix” or even just a “Nix config cookbook” would be huge for me (and, again, I’d be happy to help). Figuring out and documenting these patterns centrally may also encourage their conversion to modules, so that users can say “yes, I want to interact with x in this way, so I just import the module and set some variables” or “no, I want to use this differently, but now I have the mental tools to write my own module to manage this”.

From a consumer perspective, it’s also a whole lot easier for me to pitch Nix to my team if I can say “just read this book and you’ll know a bunch of things you need to know” rather than “go read the manual to learn the vocabulary, now go read these 15 blog posts, but also ignore about a third of that content because Nix and nixpkgs have improved a bunch in the last year”.

To draw an analogy, there are a handful of books on how to use any given Linux build system, and a dozen “how to use Linux to do stuff” books. On the other hand, there a dozen books worth of content on the Nix build system, but only a handful of non-source-code resources for “how to use Nix to do stuff” (where “stuff” is “anything but building packages”).

4 Likes

You should take a look at GitHub - LnL7/nix-darwin: nix modules for darwin which is the macos equivalent.

I agree, it would be nice to have more of a narrative with nix/nixpkgs/nixos.

I started doing some videos https://www.youtube.com/channel/UC-cY3DcYladGdFQWIKL90SQ
I was going to do a more polished (slides, final voice over, graphics, maybe transitions) video series, after the 20.09 release. Maybe have them as the “unofficial” NixOS series.

Yea… unfortunately a lot of the conventions for nixpkgs all exist in the nixpkgs repo, which means they are subject to change over time. Nix allows for a pretty large breadth of expressiveness, and this means that there’s usually no idiomatic way to do something. Certain aspects are “too painful to change” (e.g. stdenv usage), but almost everything else could be changed.

3 Likes

I do use nix-darwin, though it’s one of those things that, as you say, could use more of a narrative element to its docs. The videos look cool, though! Happy to have another resource to pitch to curious coworkers!

1 Like

Something that just came to my mind: When I started using NixOS I was always a little surprised that the releases always seemed to be delayed. It’s the same for NixOS 20.09. By end of September it was still undergoing quite some big fixes and people start wondering “when will it finally be ready?”
Don’t get me wrong here, I see how much effort actually goes into the stabilization phase where a lot of bugs are being fixed.

I’m just wondering: Should we maybe consider not naming the release by the date it was actually branched from, but instead give it a name of when it is actually being officially “released”?

I think the current naming scheme just gives the impression that we’re late with our promises when in fact the release managers were very well aware of the fact that stabilization will very likely take a good month.

4 Likes

I think it is a bit worse than that. As an example, I remember when I tried to compile some Mate apps, but I never understood the makeWrapper mechanics. It just segfaulted and I could do nothing to fix or understand it. My frustration can be summed up as Hey, I can install it from sources on Archlinux or even Slackbuilds but I can’t do the same in Nixpkgs! (and the answer is “because they install everything in /usr and everyone sees anyone, it makes the things way easier!”)

Even if the Nix language doesn’t impose limits on the creativity of our package maintainers, I think there could be such a thing as a “cookbook” for such annoyances.

I think using the YY.03 and YY.09 is fine. It’s just our preparation for the branchoff is awkward. In unstable essentially we want everything to be the latest, however, that’s not how a lot of ecosystems work. Most notably for this release: plasma.

We should really stabilize “core” packages around the ecosystems which support them. The systemd 246 bump systemd: 245.6 -> 246 by flokli · Pull Request #94354 · NixOS/nixpkgs · GitHub caused two pretty serious regressions for the 20.09 release: NixOS Tests broken in master and staging · Issue #96197 · NixOS/nixpkgs · GitHub and KDE "Switch User" menu item and lock screen button no longer present after systemd 246 update · Issue #98141 · NixOS/nixpkgs · GitHub. We should select the major blocking ecosystems (plasma, gnome) and work backwards in selecting their dependencies for qt, systemd, and other more “critical” packages. Since we’re shipping plasma 5.18.5, and gnome 3.36, both which were released early this year; we should have kept the version of systemd to something that would have been developed against. Not having to deal with the staging cycles around fixing systemd regressions would have saved 2-3 weeks.

We could “delay” the release to align with plasma and gnome schedules (release something in october), but this would give the plasma and gnome maintainers essentially no time to package and polish those ecosystems during stabilization.

For me personally, I took off quite a bit of time from work in September to help do ZHF and stabilize the release. And now that the schedule slipped into october, and I was expecting the release to have been cut, I don’t have as much free time to dedicate to nixpkgs. Also, I got burned out from reviewing PRs and fixing builds.

Anyway, this is getting off-topic. I’ll probably have a constructive thread on how we can align schedules of staging-next PRs to avoid cramming 500+ staging commits a few days before branch off Staging next by FRidh · Pull Request #97146 · NixOS/nixpkgs · GitHub Staging next by FRidh · Pull Request #96437 · NixOS/nixpkgs · GitHub Staging next by FRidh · Pull Request #96280 · NixOS/nixpkgs · GitHub

This post wasn’t meant to be disparaging against anyone. Everyone in the nixos community is here by choice, and I don’t think I’ve seen another community with such high caliber of technical skill and passion. I’m coming from a place where I want to see NixOS succeed, and I think being accountable for timelines is necessary for larger adoption. However, we kind of set ourselves up for failure in doing stable releases, and the release date always suffers. Like I said above, I’ll have a separate thread on improving the timeline for doing a release, so that we have reasonable expectations.

13 Likes

sounds somewhat similar to the goal of https://nix.dev/

Would it make sense to align with Ubuntu and Fedora releases (or perhaps a little later) and perhaps also sync the versions of major components to theirs for a release? That way we could benefit from fixes and testing done in their ecosystems as well.

1 Like

It may, but remember that Canonical and Redhat both have teams dedicated to releases. We can’t have the same expectation of hours that they can put behind their release process. Most of these distros only “ship” with one Desktop manager, meanwhile we support 17 Desktop managers, and 34 window managers. Arch experiences similar pain points, but they also don’t do stable releases, so there’s no broken expectations. Our advantages are the declarative configuration management of nix, and passion of individuals.

Another option would to be more “relaxed” about backporting changes from the latest plasma and gnome ecosystems. So we could have a release at the end of march/september and then those ecosystems would just get a DE bump a month in. However, there would be significant divergence between master and the release, and there may be a lot of non-obvious brokeness (e.g. plasma+kdeApplications+qt)

EDIT:
I also mentioned the drawback of delaying the release:

We could “delay” the release to align with plasma and gnome schedules (release something in october), but this would give the plasma and gnome maintainers essentially no time to package and polish those ecosystems during stabilization.

If we did go this route, we should probably push it back to november

Pehaps we could try to keep up with their development branches between releases. If we had those polished up a month before release, the actual release branches shouldn’t be too big of a change.

I would go the other way: rather than delaying NixOS, just delay the Plasma/Gnome upgrades to the next release. It doesn’t seem fair to NixOS release managers to have to scramble at the last minute to package and polish new software for stable release, compared to just having users wait another 6 months for a smooth desktop env update.

4 Likes

I think this is normally the case, really it was systemd+plasma which caused the issues. Unfortunately systemd has a huge api surface area, and in this case, a non-obvious assumption was broken.

The original udev issue could have been caught with more testing around staging-next. But the printer dialog issue couldn’t have.

I have a feeling (but I am not going to use releases anytime soon, so it is just a slightly curious observer point of view) that the way to release NixOS half a month to a month earlier is to do branch-off half a month earlier, with backports having to justify they are low enough impact from that point on.

I think it was the wrong call from our side to bump systemd.

I think perhaps a good rule of thumb for next time is to bump a systemd major release only once every two cycles. Instead of bumping it each release. Such that we have a bit more stable interface to talk to that doesn’t change every release. Most security fixes get backported to systemd-stable branches anyway. This way we have a bit of a more stable interface.

OTOH I’d like nixpkgs-unstable to be on a newer branch ASAP to actually find these kind of issues early.

Now the thing is; that unstable is what is used as the basis of the next release branch; whilst sometimes certain bumps just need a bit more baking in the oven because their API surface is large; and it is used by many packages.

Other distro’s like The container optimized OS | The Fedora Project have a next branch and a stable branch; and the next branch contains experimental changes that don’t end up in the stable branches per se. E.g. CoreOS now has systemd 245 on stable; but next already has 246; but this doesn’t mean per se that stable will have 246 for the next release.