Nix installer workgroup

Development Nix
22

As a nix-installer user, I have been hacking with the DeterminateSystems nix-installer lately. Is there anyway I can join these meetings live? Where I can get the schedule of next meeting?

1 Like
23

They’re every other Wednesday at 10am EDT/EST. I can invite if you dm me an email address.

1 Like
24

Sure, thanks. That would be great!

1 Like
25

Please thank Hoverbear (and everyone else involved at DS) for all the hard work regarding the installer. nix-darwin does not play nice with it but their work is much appreciated!

3 Likes
26
notes for 2023-10-25
2 Likes
27
notes for 2023-11-08
1 Like
28

Regarding that and the recent success of the detsys installer, is there a tracking issue for replacing the current installer with a fork of the detsys one? I’ve seen talks about this being done and also heard that there are some changes that have to be incorporated before it can happen, but didn’t find anything on our issue tracker.

29

There is no tracking issue. The changes can generally be characterized as disabling telemetry and undoing the ~opinionated settings/defaults.

1 Like
30

Since we haven’t been posting updates here, I’ll summarize a bit.

Work to implement changes requested by the Nix team has been done for a while now and we were working towards an initial release in May and early June, but the macOS Sequoia update breaking existing Nix installs took priority until several weeks ago.

AFAIK outstanding work at the moment looks like:

  • Finish WIP sync-up with upstream (covering roughly versions 0.19 to 0.27). I have a WIP conflict resolution up on a branch, but it poses fresh decisions that Matthew and I still need to work through.
  • Work out basic initial release process. Infra team expressed preference for us to just release via GH rather than push to S3.
  • Poke infra team to point some official obviously-experimental URL under nixos.org at the released installer.
  • Start some as-of-yet undefined process of giving project stakeholders ownership over the path forward, here.
notes for 2024-11-06
  • Travis found time since last meeting to pull together WIP merge to sync up with upstream as of post 0.27. Conflict resolution posed fresh decisions we need to sort out.
  • Matthew hasn’t had time to look at WIP merge or release process yet.
  • Tom requested meeting notes and a general update.
  • Cole reported on Matrix that upstream is getting confusing issue reports against their repo from people trying to install the “prerelease” of the experimental installer
    • Domen set this up for the project early this year and included in the devenv installation instructions. It uses a workflow to commit installer binaries on the prerelease branch.
    • This depends on our Hydra jobset, which we inadvertently disabled during conflict resolution in the May sync-up, though enabling the jobset alone didn’t fix the issue.
    • We debated whether to put effort into debugging this (potentially distracting us from merge and normal release process), ask Domen to do so, or just defer worrying about this.
    • Cole likely saved us from this decision by spotting the probable bug in the Python script responsible for the release.
    • After meeting: Travis committed/pushed this fix. It looks like this worked, but the broader workflow now fails when it tries to check out the prerelease branch. (Whenever the workflow was first run, the prerelease branch didn’t have the binary files. Now that it does, there’s a conflict with the freshly-generated files when the git-auto-commit action tries to switch to the branch.) Punting to Domen on this.

(These are from memory; may have omissions.)

6 Likes
31

Discourse isn’t letting me reply more than 3x in a row, so I’ll just edit summaries in here for now:

notes for 2024-11-20

There wasn’t anyone on with perms to let us in to the usual meeting room, so if you tried to join today you might have failed.

After a few minutes Matthew started up a separate call.

  • We mostly used the time for a working session on the issues mentioned previously with the WIP merge. I think we’ve got the obvious ones sorted out. I need to find time to go back and do some history rewriting on the merge to clean up before I open a PR. (Hopefully this week if not tonight.)
  • Matthew did have time to skim the merge commit and confirm he didn’t see any glaring misses/issues with my conflict resolution beyond the new questions we’ve been sorting out.
  • Matthew has started work on a release workflow that should give the infra team what they need to get this wired up on nixos.org. No PR yet, but he expects to be able to open one soon and that it should be fairly straightforward to review.
notes for 2024-12-04

Matthew and Travis both managed to open PRs for the merge (https://github.com/NixOS/experimental-nix-installer/pull/25) and release (Rework release-script to use GitHub releases by mkenigs · Pull Request #23 · NixOS/experimental-nix-installer · GitHub) work mentioned in the previous note, we talked through both:

  • Most discussion around the merge PR is focused on finding diff-minimizing ways to snip out determinate-nix related functionality added upstream. General agreement on next round of tweaks here. Travis will try to get it updated this week.
  • Talked through how release process in PR differs from existing prerelease script. General agreement apart from needing to hardcode version string in outer install script. Cole pointed out how upstream uses sed to rewrite this in their equivalent release script, so adapting that to our case is the next action item here.

Also discussed what’s in recent upstream releases and looked at a recent WSL report against the official installer (but no new insight).

notes for 2024-12-18

Merge to update to post 0.27.0 finally landed last week, leaving two items on our TODO list before re-contacting the infra team about getting usable experimental installers hosted under nixos.org:

  1. Finish release workflow. Currently draft PR; Matthew can hopefully finish this week.
  2. Update nixpkgs pin. It’s fairly old at the moment. Travis tried a few times along with the merge PR but all revs tried introduced build failures. Matthew or Travis will hopefully try this week.

Apart from that pending work, we also discussed:

  • Two reports against nixos/nix mentioning different daemon connection errors. No clue on cause, but collected some fresh information-gathering recommendations for Travis to pass along soon.
  • An issue observed at Flox with a macOS user who’d recently enabled FileVault. Sounds like the user just needs to reboot after enabling FileVault.
  • In anticipation of having official-experimental installers hosted at nixos.org, discussed what our next steps may be with Tom. Nothing concrete here yet, but this is mostly about:
    • Announcing the experiment to get people to try it out and drum up interest in longer-term improvement/maintenance.
    • Nixos org (maybe just the Nix team, or maybe including SC) evaluating the experimental installer to make decisions about adoption, how installer and Nix releases fit together, etc.
  • Brief discussion of some success Flox has had with homebrew formulae that wrap the installer and perform macOS uninstall.
  • We’ll skip next meeting because it would fall on New Years Day.
notes for 2025-1-15

As mentioned in the previous note, we skipped the meeting that would have fallen on New Years Day.

Within a few days of the December 18 meeting, Matthew and I were able to check off the remaining items on our TODO list and re-contact the infra team. The infra team pointed us to the marketing team / homepage repo, where I opened the following issue:

We didn’t have much “news” to discuss in this session since there hasn’t been any activity on the issue yet.

@idabzo from the marketing team joined us for the first time today, so we spent most of the session running through the history, goals, and current status of the installer workgroup.

Beyond that:

  • Matthew will give Tom another poke regarding the issue above (helpful since he’s been in some of our workgroup meetings and is on the marketing team, Nix team, and SC).
  • Travis is opening a PR today for a small issue with the macOS Sequoia UID migration script.
3 Likes
32

@abathur this is awesome news, thanks a lot for sticking with it for all this time! Looking forward to the release of the experimental installer.

1 Like
33
notes for 2025-1-29

There has been activity on our open issue (endpoint for experimental installer · Issue #1614 · NixOS/nixos-homepage · GitHub) since the last session, but we’re still roughly in a holding pattern for that to move forward.

We mostly discussed recent developments upstream and the possibility that Matthew and I may review our current diff against upstream to figure out if we have any ideas for either upstreaming tweaks or reworking changes to minimize the maintained diff. (We aren’t sure we’ll have time for this, but perhaps.)

If the above goes well, we may also take a swing at one more sync with upstream.

notes for 2025-2-12

Short call today since we’re still in a holding pattern for the redirect from nixos.org.

We mostly discussed recent developments upstream. Matthew and I haven’t had time to bite off any diff minimization or look seriously at another sync. I suspect we’ll want to wait for some pending refactoring of recent new features upstream before we try again, though.

notes for 2025-2-26

Short call today since we’re still in a holding pattern for the redirect from nixos.org.

notes for 2025-3-12

Skipped this call since Cole was out of town.

notes for 2025-3-26

Short call today since we’re still in a holding pattern for the redirect from nixos.org. Matthew’s out of town, so Cole and I discussed a bug report and called it early.

notes for 2025-4-9

Still mostly in a holding pattern for the redirect for nixos.org.

  • Matthew opened a PR last night removing the “nix-community” feature we’d added early on. The intent was to conditionally compile the changes we needed (and leave some good signposts in the code around what those are), but over time we’ve come to feel like it also exacerbates the conflict-resolution burden of syncing with upstream. I’ll try to find time to review it this week or weekend.

  • We also discussed whether to do another sync. General agreement that it’s worth doing another in the near future, but I’m not sure Matthew and I will have the bandwidth to do it. I’m going to check in with Emily about whether there’s already work underway here. If we can’t get to it, the sync up will be a good task to start familiarizing others in the org with the codebase.

  • Discussed me pinging Jörg in the infra issue we opened, and I’ve already done this.

  • Discussed Valentin querying about status and having some interest in helping push a bit. I will update him once I hear something on ping above.

notes for 2025-5-21 (and two previous)

We did meet twice since I last posted notes, but the calls have been short and we also skipped two for vacations.

Still mostly in a holding pattern for the redirect for nixos.org.

  • Reviewed/merged Matthew’s PR removing the nix-community feature (to reduce diff a little).

  • Discussed recent issues and PRs against both installers. Mostly about edge-case system conditions (around shell profiles).

notes for 2025-6-4

Tom dropped in for this meeting with a few questions. This note is a little more reflective than a strict summary of discussion. It mostly centered on three things:

Where things stand

Still in a holding pattern (but Tom said he’d bring it up elsewhere, and the infra wheels have started turning since we met).

Single-user install support

The upstream detsys installer doesn’t implement this, and AFAIK there’s also no released prior art from the Lix installer folks (unless it’s in a PR–I didn’t go hunting). Cole did mention that the detsys installer has a flag for disabling init setup (but the shell/profile setup also differs in single-user installs).

I personally see this as one of several open questions that will need to get solved as the org builds up opinons/muscles about and around how to put the installer to work (i.e., one beyond my own commitment to slowly push on this until it is ~ripe for the broader community/project)

I assume we’ll want to have it, but I think there are multiple paths there:

  • Keep maintaining the current single-user shell installer and wrap the compiled installer for multi-user.
  • Try to persuade detsys to implement (or implement and persuade them to carry). I think at one point they said they were not interested, but it’s possible there is ~demand from their customers.
  • Build out and maintain the functionality in our fork.
  • Emilazy expressed some interest in extracting the ~common base of functionality of both the NixOS org’s experimental installer fork and the Lix fork to deduplicate effort. I recall Lix’s installer having an open issue about single-user support, so if they’re still interested we might be able to implement in that base.

Detecting existing Nix installs from other system package managers

Didn’t have a ton to say about this one. It’s a good idea (since these installs can be ~weird and may not be safe to take normal curing steps on), but I don’t recall it coming up as a ~goal in previous WG calls.

Apart from these, we also discussed the trickle of people with missing macOS volume mount issues. I haven’t tried to intentionally collect all known causes, but the main one I brought up on the call is people disabling the poorly-identified sh background items that show up in their Login Items & Extensions.

Cole mentioned nix-daemon: use PathState to wait for nix store by booxter · Pull Request #1385 · nix-darwin/nix-darwin · GitHub as maybe lighting the path towards improving on that situation.

Since the meeting, I feel like I remember at least one case where mixing installers caused this as well, since the detsys installer does or at least did name keychain credentials differently than the official shell installer. (IIRC this was to avoid having to depend on a UUID that doesn’t exist until diskutil creates the volume.)

4 Likes
34

Notes for 2025-12-03

Tentative plan for experimental-nix-installer next steps

- Rename to nix-installer and detach the fork

- Update Hydra jobset

- Post on Discourse

- Mention in upcoming release notes

- Test for one Nix release cycle

- Start recommending as official installer in one cycle for supported platforms unless there are issues

- Keep recommending shell installer for platforms not supported by the Rust installer, and there are likely lots of dependencies on it in the wild

8 Likes