Nix installer workgroup

Development Nix
1

Hi,

As stated in our last update, we (Tweag) would like to do something to improve the situation with the Nix installation process.

This can’t be just Tweag’s doing – If only because I, for one, have only a very faint idea of all the devilish details that make the difference between a strong reliable process and an okay-ish one. So I’d like to gather a tiny community effort around that, starting with a first meeting next week to gather the requirements and get a first idea of what we want to build.

If you’re interested, please fill this when2meet so that we can settle on a date and time.

cc @Solene @fricklerhandwerk @mkaito @grahamc @edolstra @roberth @domenkozar @abathur

12 Likes
3

The installer has issues with Linux systems using SELinux.

Having an official standalone nix binary would also be awesome, I count this as “installing” because it’s a non-installation case.

2 Likes
4

General request: it’s good to be explicit (and honest) about the installer’s shortfalls (there are a number) and avoid broad negative generalizations. When threads with comments like this end up getting sanctioned by the search-gods, irrefutably vague quality claims end up getting repeated by people who don’t know better on HN, twitter, reddit, etc. where they feed FUD that causes people to avoid trying Nix, or to open threads/issues asking whether or not it works. (I saw these regularly for well over a year after we’d managed to fix the worst issues with the read-only root on Catalina, for example.)

(Edit: right on cue, an hour after I posted this: Introducing Riff: automatically provide external dependencies for Rust projects | Lobsters)

The installer works for the golden path on macOS and some (but not all) Linuxes. It isn’t idempotent (people need to follow the uninstall instructions before reinstalling). There are sharp corners when it comes to people who have an environment that doesn’t line up with the golden path.

9 Likes
5

While I can’t dedicate much time to it, here’s my wishlist:

  • document the difference between single/multi user install and the decision that the user needs

  • merge 4-6 scripts that create the installer so that the logic is centralized (right now each script has a different syle, functions, etc)

  • installer should be idempotent

  • there should be an uninstaller that works (this is mainly not to gain Nix haters)

  • create automation to test the installer on many platforms, especially for the new ones (macOS)

13 Likes
6

I’m really excited to see this post @thufschmitt. At DetSys we’ve been experimenting with an updated installer, and we may even be able to publish a beta version by the end of the week. We weren’t planning on sharing it so soon, but it sounds like we should :). Ana has been taking the lead on this. The design she’s been exploring focuses on being idempotent with rollback on failure. The beta we’re looking to publish isn’t going to be all things for all people, but is intended to be a start. Just filled out my times!

8 Likes
7

For the upcoming tools abstracting nix usage, namely:

a more “lightweight install” could be beneficial. The nix install can feel a bit “intrusive” for some users when the entire nix tool chain is installed. Also avoiding changes to the users environment would be beneficial. Not sure which part’s could possibly be left out.

Examples

Bob uses nix-build only. It’s parsing the output to resolve /nix/store paths and passes them to an internal shell. Something like nix-shell is not required in that case.

Devbox uses nix-shell directly.

2 Likes
8

Thanks for the answers :slight_smile:

Based on the when2meet feedback, I’ve scheduled a meeting on 2022-09-14T14:00:00Z → 2022-09-14T15:00:00Z.

Meet link

I’ve directly send a calendar invite to the people for which I have a mail address and I’ll add it to the commmunity calendar.

9

@thufschmitt could you please designate a space for notes (e.g. this thread or a dedicated GitHub issue) so those who can’t join synchronously can add input in advance?

10

Sure thing @fricklerhandwerk , here’s a link: https://hackmd.io/@thufschmitt/HktYmvheo/edit

@Ana , @grahamc that’s great news :slight_smile: I’ve put a presentation of your experimental installer on the agenda if that’s OK with you

@matnosner interesting idea… Afair @edolstra had been doing some related work to allow having a single drop-in statically linked executable that would just work™, maybe that’s something worth looking into for these use-cases

11

I don’t know where to report the issue but I once installed Nix on a corporate Amazon workspace and there were inconsistencies because the user that was assigned to me was part of some kind of ldap or active directory (I don’t remember really well sorry)… and it contained backslashes.

I also remember that Nix has some troubles to deal with usernames containing backslashes.

I still do have issues on those platforms, and I have to make some kind of workaround as you can see here: flake.nix · main · ECPHP / Devs profile · GitLab

Do you think dealing with these weird usernames could be part of this workgroup?

2 Likes
12

@drupol Please check for duplicate installer issues and otherwise open a new one. The workgroup should monitor that GitHub label to keep track of pain points.

2 Likes
13

Thanks to everyone involved in the meeting, it was really great to all sit around and discuss this. The notes are (still) here, or in the spoiler below in case the document disappears:

Notes from meeting #1

Agenda

  • Round table to get the different expectations from the installer
  • Presentation of DetSys’s experimental installer

Pre-meeting notes

  • Notes from @domen:
    • document the difference between single/multi user install and the decision that the user needs
    • merge 4-6 scripts that create the installer so that the logic is centralized (right now each script has a different syle, functions, etc)
    • installer should be idempotent
    • there should be an uninstaller that works (this is mainly not to gain Nix haters)
    • create automation to test the installer on many platforms, especially for the new ones (macOS)
  • Notes from @Solène
    • The installer has issues with Linux systems using SELinux.
    • Having an official standalone nix binary would also be awesome, I count this as “installing” because it’s a non-installation case.
  • Notes from @matnosner

    For the upcoming tools abstracting nix usage, namely:

    • devbox
    • riff
    • bob

    a more “lightweight install” could be beneficial. The nix install can feel a bit “intrusive” for some users when the entire nix tool chain is installed. Also avoiding changes to the users environment would be beneficial. Not sure which part’s could possibly be left out.

  • Notes from @fricklerhandwerk
  • Notes from @edolstra
    • Revived Nix-based installer test: Add an installer test by edolstra · Pull Request #7043 · NixOS/nix · GitHub
    • We should minimize the amount of code duplication / divergence between different installation paths, e.g. daemon vs non-daemon, bash vs. fish.
    • The best way to simplify the installer is to minimize the amount of work done by the installer, i.e. have more defaults in Nix or Nix auto-creating directories.
    • Even better is if you don’t need to install Nix at all, e.g. by using the auto-chroot statically linked nix binary.

Meeting notes

Round table

  • Ana
    • Determinate Systems (DetSys)
    • No strong opinion on what the installer should do, just wanna work on it
  • Chris
    • Tweag
    • Same thing mostly
    • Onboarded a couple friends, nearly everyone got bitten by some installer quirk
  • Travis
    • Suffers for people encountering install issues
    • Worked on some MacOS-specific issues on the installer.
    • Needs testing, testing , testing
  • Alexander
    • RedHat (unrelated to Nix)
    • Install on Fedora is… interesting
    • Heard about SELinux issues
  • Solène
    • Tweag, part of the Nix group
    • Learned about the installer issues only recently because exclusively uses NixOS
    • Interested to help with testing
    • A standalone binary could be great too
  • Graham
    • Determinate Systems
    • Talked about that for a few months at DetSys
      • Installer is bad
      • Uninstaller doesn’t exists and the instructions are scary
    • Started to reimplement the installer in the last couple weeks
  • Matthew
    • Flox
    • Not fetching a random script from the installer
    • Make it more “pure” because it’s a bad intro to the ecosystem right now
  • Eelco
    • Determinate Systems
    • Used to have an automated installer test in the Nix testsuite, but deprecated
      • Revived it with Graham’s matrix installer test
    • The installer should do less stuff (like not having to create all the build users)
    • Extreme form of that: Standalone statically linked binary
  • Théophane
    • Tweag
    • Installer is the entrypoint to the ecosystem for most people. So it needs to be perfect

DetSys installer

Just uses bash to bootstrap the actual installer which is a compiled binary.

Demo

Rust binary

  • Idempotency
    • The installer is doing a lot of things that should all be reversible
    • Should save the exact steps that have been done somewhere so that it can be rolled back (or replayed, or shared)
  • Architecture:
    • Fetch a (bash) script that does some feature checks then fetches the binary installer for the current platform (could adapt the Rustup wrapper script for that)
  • Currently very opinionated
    • Will switch flakes on by default
    • Only uses the daemon

Single-user install?

Not supported by DetSys installer. Because they feel that single-user mode is not a good experience and makes the ecosystem less uniform (so the userbase more complex).

How useful is the single user install?

  • Docker containers (because no nested sandboxing)
  • No-systemd systems
  • Security-constrained envs

@Solène offers to create a discourse poll to get a sense of how much it is used

WSL?

  • Can run systemd (even NixOS there), though in a hacky way
  • Technically uses its own init system, might be worth having a dedicated support for it

Installer language

Having a binary installer means running a binary blob, while a bash script is somewhat more introspectable.
But people will run a precompiled Nix anyways, so that doesn’t change too much.
And the benefit of having a proper language over bash outweights this by far

Distro packaging

  • Ubuntu has it (slightly outdated) in its repos, which is great
  • Rust gave up on that
  • nix-community has rpms

Having stuff on distro’s repos is a lot of pain.
Having Nix packaged as a deb/rpm is simpler, and makes people more comfortable, and we avoid the problem of having them outdated in the repos.

To be discussed later.

Testing

  • The installer should test itself at the end to make sure that stuff is available in the user’s shell
    • Manage the expectations. If people have a weird system that prevents the installer from working, it must be made explicit so that they know it

Future steps

We have a follow-up meeting planned on 2022-09-28T14:00:00Z → 2022-09-28T15:00:00Z, the details are on the NixOS.org calendar.

@mkaito also opened another thread to continue in an async fashion the conversations that we’ve started yesterday

2 Likes
14

We had a second meeting today, thank you to everyone who came. A lot of discussion was around testing: macOS, Linux, WSL, and how to replicate various scenarios users have had trouble with. Check out the notes for more: nix-working-group-meeting-2.md · GitHub (originally Nix installer workgroup meeting #2 - HackMD)

We are going to meet at the same time in two weeks: Nix installer workgroup meeting #3

We also opened a new Matrix room for the working group: #install:nixos.org. See you there!

4 Likes
15

We just finished our third meeting, thanks folks for attending! we talked some about the installer DetSys is working on, and what sort of acceptance criteria there would be for the greater community to want to use it. We decided to not get too far into details, as we can become more specific once we have an MVP out.

Here are the notes: nix-working-group-meeting-3.md · GitHub

We’ll meet again in two weeks: Nix installer workgroup meeting #4 See you then!

3 Likes
16

Another meeting today, TLDR is the installer is getting polished but it still require some work especially for MacOS (while WSL has been painless).

I would like to thank @grahamc for taking the following notes during the meeting:

2023-02-01

Agenda:

  1. Release
    • soft release with zero to nix to avoid saying it is ready-ready
      • learned a bunch of stuff about APFS volumes / persistence
    • postponed to February 24, but might move it up
    • rename it again
  2. Upcoming work
    • Deleting the nixbld users on macOS.
      • Not excited about releasing without deleting the created users.
      • Potentially want to take advantage of the auto-uid-allocation feature?
        • Needs testing & verification before putting it to users.
    • WSL
  3. What’s missing to have parity with the upstream installer
    • No single user install support
      • Looking at a root-only installation mode.
    • Architecture support
      • i686-linux support would be nice, and plausibly a blocker for upstream
      • Ana is making a ticket.
  4. Questions / Todo’s
    • Does the uid/gid allocation try to detect conflicts?
      • not today
      • → bug report
    • Let’s document that you can install NixOS on any aarch64 pi (todo: list)
      but only install Nix using this installer on pis running an aarch64 OS (todo: list)
2 Likes
17

Does the installer workgroup see itself as having a stake also in NixOS installation under WSL? I think it’s an important use case and it seems related to the big picture of Nix onboarding.

I use it at work, and it’s great. If anyone involved in this has the bandwidth to help nudge it along on its way to the Microsoft Store , that might be great for introducing many developers and sysadmins to NixOS in an undemanding way.

1 Like
18

Their is prior work done GitHub - nix-community/NixOS-WSL: NixOS on WSL(2) [maintainer=@nzbr] and mostly maintained by @nzbr , @K900 and me.

Neither nixpkgs or NixOS have good 32bit support and only dependencies of wine and the steam fhs are truly tested and fixed AFAIK. Before 32bit support for the installer is advertised the support state of 32bit should be clarified. I myself would not consider missing or removed 32bit support any kind of blocker in a PR for a lot of packages right now.

2 Likes
19

Here are the notes from the meeting today, 2023-03-1!

Notes
5 Likes
20
Here are the notes from 2023-03-29!
  • (upstream) GitHub Action
    • If nix-installer is, then the action will be
  • Single user install
  • Mac SSL issues
  • Increasingly exotic environments
  • mac zsh ssh/PATH-order issue?
  • Long-term plans
    • What does Theophane think is the most right thing to do, re: upstreaming the DNI?
      • nobody really understands the current installer / touches it except reluctantly
      • does Nix want to maintain their installer, or adopt the DNI?
      • Theophane is in favor of upstreaming, since it is technically superior, even though it doesn’t support some use cases.
      • unsupported cases:
        • single user mode, probably the biggest case
      • could fall back to the current installer in cases it doesn’t work
      • what comes from non-trivial bugs / regression reports?
      • Travis: generally in favor of this as well.
        • Are there major regressions or feature gaps? Are these feature gaps problems that need fixing?
      • Ana: any problem with the fact that DNI is in Rust?
        • Theophane: no
        • Travis: Bash is popular, but it isn’t helping us get quality contributions
        • Theophane: does make bootstrapping problems more complex, but this is not strictly an issue for the installer
        • Travis: If there is a downside, it isn’t end-user hackable.
          • For example: a user could download the installer and hack in a new UID or whatever… which may mean that user is less likely to contribute an upstream fix
      • Ana: would the Nix project want to upstream the backend that takes the backend which redirects based on branches and PRs?
        • Theophane: unclear, DNI is useful with just S3 or whatever
      • Travis: should the installer work with older versions of Nix?
        • Ana: We don’t now, but potentially we could?
        • Theophane: currently its tied to a specific version
        • Graham: would rather not take on the technical complexity needed to support many versions of Nix
      • Graham: Who gets to choose? What’s the process?
        • Theophane: trusts your judgement to decide…
          • Who is going to make the effort to do it?
        • Travis:
          • intermediate step makes the current URLs continue to work as-is
          • start pulling the Rust version in as a non-primary version
          • encourage people to start testing the Rust version … in the event they’re having trouble.
      • Théophane: What would upstreaming look like from the Determinate Systems side?
        • DetSys would probably keep its own version as a way to freely experiment and keep it opinionated
        • Upstream would be a fork of that installer
        • The Nix org could also use the public rust library and create a modified version of the DNI
        • A peculiarity of the DNI is that it sends automated reports on installation failures. That probably wouldn’t work for upstream, and cause the DetSys version to be aware of some problems earlier
      • Travis:
        • Automation could automatically send patches from the DNI repo to a Nix installer repo
      • Matthew:
        • Long term, who is maintaining the Nix version of the repo?
        • Theophane: Installer WG would be a good fit
      • Next Steps:
        1. Theophane to check with the Nix team to:
          • officially say they want to experiment with it.
          • Create a fork under the NixOS org and produce binaries, made available at a convenient “official” URL which is not the main installer URL
          • Recommend using the Rust installer experimentally, or as a fallback if the bash version doesn’t work.
        2. ???
        3. Install nix

TLDR next steps for the DetSys installer are:

  1. Theophane to check with the Nix team to:
  • officially say they want to experiment with it.
  • Create a fork under the NixOS org and produce binaries, made available at a convenient “official” URL which is not the main installer URL
  • Recommend using the Rust installer experimentally, or as a fallback if the bash version doesn’t work.
5 Likes
21
Notes 2023-10-11
4 Likes