Brainstorm for RFC: Assimilate home-manager into Nixpkgs monorepo

Development RFCs
1

Hello!

As I have said so many times before, I am proposing to assimilate home-manager into nixpkgs monorepo.

Below is a draft of RFC I want to propose. Feel free to roast it!

RFC Draft I
  • feature: All Your Home Manager Are Belong to Us
  • start-date: (fill me in with today’s date, YYYY-MM-DD)-
  • author: Anderson Torres
  • co-authors: (find a buddy later to help out with the RFC)
  • shepherd-team: (names, to be nominated and accepted by RFC steering committee)
  • shepherd-leader: (name to be appointed by RFC steering committee)
  • related-issues: (will contain links to implementation PRs)

Summary

Assimilate home-manager project into Nixpkgs monorepo.

Terminology

Henceforth,

  • Home Manager will be called HM;
  • The typical unprivileged user of a system will be called basic user;
  • The typical privileged user of a system will be called superuser;

Motivation

Nix the language has at least three famous consumers, namely:

  • Nixpkgs, the biggest packageset in
    existence;
  • NixOS, the reference work on declarative configuration
    and deployment of a Linux distribution; and
  • Home Manager, another
    reference work on declarative user-specific configuration management and
    deployment.

Since at least 2014, NixOS was assimilated into Nixpkgs monorepo, now living
inside nixos directory.

This RFC proposes a similar assimilation of HM into Nixpkgs.

Benefits

In principle, Nix already leverages Nixpkgs for basic users. However, the raw
usage of Nixpkgs is not too ergonomic, especially when compared to the more
structured model of NixOS.

HM provides a similar centralized, modular, declarative package management
experience to basic users, without the need of granting them superuser
privileges.

Given that NixOS - a system that leverages Nixpkgs for superusers - is already
bundled inside Nixpkgs tree, a fortiori HM - a system that leverages Nixpkgs for
basic users - should be included as the default, Nixpkgs-blessed system for
basic users.

Further, this assimilation will benefit two interesting publics:

  1. Users of Nixpkgs and/or NixOS that are reluctant in using HM, since it is
    neither official nor well-integrated into Nixpkgs workflow.

  2. Users of NixOS that already use HM in tandem, as a convenient way of
    separating basic users’ business from system administration tasks.

Another great advantage of assimilating HM to Nixpkgs is a tighter integration
between both projects:

  1. Merging HM commit history tree into Nixpkgs eliminates a barrier between
    both projects, therefore conveying more flexibility in code sharing,
    deduplication and refactoring in general.

  2. Refactors, reformulations, deprecations and removals of packages and other
    related functionalities are commonplace around Nixpkgs. Since HM is
    dependent on Nixpkgs, it needs to monitor and synchronize such activities.

    The eventual assimilation proposed here drops those issues dramatically.

  3. The merge of communities brings a new point of view about package maintenace
    in general.

    NixOS is the main point of structured conglomeration of packages; because of
    this, Nixpkgs is more inclined to favor a superuser view of system
    administration.

    Bringing HM to Nixpkgs provides a new point of view, more akin to the basic
    user.

  4. Making HM an official, blessed tool conveys more credibility to both
    Nixpkgs and HM.

Detailed design

  1. Prepare Home-Manager to migration
    • Treewide formatting and refactoring according to most recent written and
      unweritten rules of Nixpkgs code style
  2. Merge Home-Manager repository, putting all its relevant files inside
    home-manager directory;
  3. ? ? ?
  4. Profit!

In terms of Git hardwork, the assimilation of HM is expected to be not too
difficult; a judicious appplication of
git-filter-repo plus some
reposurgeon can solve this without
much hassle.

Problems are expected when dealing with the interface between Nixpkgs and HM,
albeit they can be reduced if we treat HM as an inner overlay.

Examples and Interactions

Ideally, after the assimilation, a basic user will experience few to no changes
in their workflow. The channels of distribution of packages will be merged,
promoting a cleanup on setups that otherwise would need synchronization.

On the other hand, from this assimilation forward the typical package maintainer
will interact with potentially two sets of too similar module systems.

Drawbacks

The main drawback of this assimilation is the resulting complexity.

Standardization

There are some discrepancies between the practices of both communities.

Synchronize the communities

There are almost 660 contributors in HM to this date. How they should be
allocated?

Ideally they should keep the same roles.

CI

Currently HM uses a different setup for continuous integration. Ideally the
Nixpkgs setup should be updated to include HM’s specific needs.

Alternatives

The alternatives are

  • The trivial “do nothing”

    It just exacerbates the status quo, bringing nothing in the long term.

  • Bless another home management tool

    HM is battle-tested and well renowned. There is no other tool remotely
    comparable to it.

Prior art

As an example of prior art, there is our Scheme-based cousin, Guix Software
Distribution. Since at least 2022 AD they bring a similar tool, conveniently
called Guix Home.

The nicest thing about this tool is that it is tightly integrated with Guix, to
the point of home being a mere subcommand of guix.

Unresolved questions

How the future package inclusions should be carried out?

Future work

  • Update an extend the CI
  • Set expectations on portability among present and future platforms Nixpkgs
    supports
    • Especially outside NixOS
  • Factor HM and NixOS’s shared code into some service abstraction structure

References

34 Likes
2

I love this idea, but how does it work for those using home-manager outside of nixos? Would it be as simple as changing the channel URL, or do they have to make large changes to ensure things still work?

3 Likes
3

Please let’s talk with Home Manager maintainers here? @rycee and many others may have an opinion on this.

14 Likes
4

Not sure if this is actually likely to happen, but I really like the idea. It feels like there are a lot of things that could benefit from greater integration if home-manager could be integrated into nixpkgs, not to mention the possibility of coordinating updates of nixpkgs packages to home-manager modules that deal with said packages.

edit: Also, one thing that has always bothered me is the dramatic split between home-manager modules and NixOS modules for the same programs. There’s often some machinery that is done for the global configuration that isn’t done for the corresponding home-manager configuration or vice-versa. In some cases it seems like the code could literally be shared, allowing for parity between home-manager and global configuration in some cases.

I would guess that the idea is to not have any impact to existing home manager users, e.g. you could just import more or less the same home manager modules from Nixpkgs instead and use them e.g. in your vanilla Nix or nix-darwin installation. (Of course, what would be even better is if Nixpkgs could also merge in nix-darwin and make everything first class, but that’s probably a pipe dream. As challenging as Nix on other platforms can be, I really want Nix everywhere, all the time.)

1 Like
5

Worth noting an issue I recently found in home-manager, when I wanted to add two flutter packages to home.packages:

I don’t mean to highlight the issue itself, but it reflects an important issue: some components should have been reused in home-manager and maintained consistent behavior with NixOS modules. But actually we didn’t do it.

We also have many modules, mostly under programs, including some window managers, which would be more natural to configure them as per user, rather than system-wide. If we can merge home-manager, we can deduplicate many of these modules and make our configuration more flexible.

Not to mention, home-manager is like GNOME’s dash-to-dock at this point in the NixOS community (and users benefiting from Nix on other distros). Officially it is not considered necessary, but in fact it may be more than 80% of users (I don’t have statistics, btw). With so many users, keeping it as an external component, whose usability doesn’t match the user base, may no longer be a wise choice. Regarding the usability issue of external module systems, we can refer to the decision to merge the NixOS repo into Nixpkgs. I believe Anderson has explained this well.

4 Likes
6

Would we be pulling in all of the current modules or would we put them through the ringer to ensure a certain code quality. I know catppuccin/nix had some issues recently because of some of HMs quirks. You can read a bit more here nix/modules/home-manager/swaylock.nix at 9eb0610d48dd0e1fecf772bbdacf9050d7b82d7c · catppuccin/nix · GitHub

7
  • Add in batches, with reviews
  • Add altogether, with planned cleaning up

Both looks good to me.

1 Like
8

Factor HM and NixOS’s shared code into some service abstraction structure

what did you mean by this? my knowledge of HM is a few years out-of-date, but they do everything with nixos activation scripts, right? reading “service abstraction” as “setup the home directory using systemd services” is probably not what you meant, or is it?

it’s one area i have a concern with though: there’s at least 3 ways i’ve considered doing home management in NixOS: activation scripts, integrate with the service manager, or build /home as a nix derivation with symlinks to the stateful bits which can live externally (i.e. “impermanence”). i used to do the first (via HM). i’m presently doing the second (via my own modules). i’d like to investigate the third at some point. there’s obvious value in declarative home management. i’m less certain that the specific solution HM uses for “how to get those files into the right places at the right time” is the “right” solution. i guess i’d hope that if we integrate that into nixpkgs, we do it in a way that’s friendly to experimentation there. clear separation between the user-facing config statements, and the behind-the-scenes bits.

an example where we’ve not done that is exactly those nixos/modules/services directories. there’s plenty of nixos modules which need to declare a daemon. many of them don’t really care about the actual service manager which starts that daemon or keeps it running, quite a few don’t even need to specify any dependencies/ordering. but they’re all wed to a specific implementation none-the-less (systemd.services.FOO = ...), and the actual data those modules are providing lacks a certain potential for reuse as a result.

not saying i want first-class support for 3 different service managers, nor for 3 different home managers. just as (if) we add a home manager, it’d be nice to identify some natural interfaces we can embrace to keep the door open for experimentation externally.

1 Like
9

I’d like to manage my home with NixOS. But I thought it’s better to do that from scratch in NixOS instead of pulling in the large third-party mess. As long as we can manage packages, services and files in home, module maintainers can support user config optionally and gradually and HM can stay as a cross platform tool.

10

How is the code quality of Home Manager?

11

Given that this is not the status quo for neither Nixpkgs or NixOS, I do not see this as a requirement for merging or even relevant to the discussion.

I don’t see why you’d need any fancy tools here or why you’d even want to rewrite history; why not simply keep all the history in-tact?

All we need is one merge commit with the most recent Nixpkgs revision as its first parent and the most recent hm revision as its second parent. In the commit’s tree, all the hm stuff would be moved into the subdir. Diff visualisation tooling should correctly pick that up as renames if given a high enough rename detection limit.

I think this could be done using git subtree but you could always do it the dirty way:

  1. Prepare desired tree ahead of time
  2. Do a dirty merge
  3. Reset working tree to the prepared tree; resolving all conflicts
  4. Commit

I think that’s fine to keep for now. As long as it’s limited to the home-manager/ subdir, it doesn’t affect anyone else outside of super basic things like commit hygiene but I think hm is up to snuff there.

NixOS necessarily also still has “discrepancies” w.r.t. nixpkgs and that’s fine too.

Again, I don’t see why that needs to happen in step 1. Why couldn’t hm keep its CI setup as is (modulo code location)?

WDYM by “future package inclusions” and what does it have to do with home-manager?

What I think is missing is the community side of things.

How do we deal with existing issues? Will we simply transfer them over and attach a home-manager label?
What will happen to open hm PRs?

How do we deal with labels? How do we deal with issues/PR automation? (i.e. ofBorg, not sure if hm has an equivalent)

Should home-manager committers need nixpkgs commit access?
And isn’t the set of home-manager committers nearly a subset of the nixpkgs committers anyways? (Ignoring the current turmoil which I assume to be temporary.)

How should Nixpkgs’ release channels work with home-manager? Should HM CI block channel advancement?

Even important however, I do not see any representation of the existing hm community in the RFC; how do they want the future of home-manager to look like?

I’d require at least one core hm maintainer to co-author this RFC in order to take it seriously; the home-manager community must also want this, not just us Nixpkgs people.

One final note: I think it’d be very poor timing to turn this into an actual RFC right now, given the current uncertainty about who the Nix community actually is. I don’t see that being resolved any time soon and wouldn’t expect it to be resolved to a point where this sort of thing could be done in less than a year.

I could trivially see the hm community not wanting to merge with Nixpkgs purely due to the uncertainty around the NixOS organisation.

I’d prefer we waited until either the current community manages to rid itself of its toxic parts to be reasonably healthy or until a separate community overtakes it as the primary “Nix” community.
I could see the RFC process itself requiring an overhaul in either case.

5 Likes
12

There is at least one RFC that requires formatting. In a big enough amount of time it will happen.
I am merely anticipating the scenario.

Read this small block of text as “we have tools for doing this assimilation”, somewhat like a mathematician that writes “there is at least one solution”.

If you are correct, less one problem to think about!
As I said before, I am anticipating objections.

There are many packages that have HM modules as well as NixOS modules.
How such packages should be dealt in the future? Maybe a shared module that can be specialized later for each corresponding framework…

I am not aware of GitHub assimilation tools, but ideally they should be transferred to NixOS monorepo as well.

This is precisely what I was talking about the HM CI side.

Yes.

Most certainly.

This is why I opened this brainstorm topic.

13

Actually, it’s almost there: Enforce nixfmt for new/changed files, treewide reformat pass 1 [skip treewide] by infinisil · Pull Request #322537 · NixOS/nixpkgs · GitHub :smiley:

6 Likes
14

We do not need “HM”. We need user-level configuration management - with decent quality, contributed by people who actually read some of the systemd docs before writing systemd services.

To that end, we might benefit from making our module architecture more “system-agnostic” - i.e. one layer for generating configuration, and another layer for deciding where that configuration goes (in /etc, in /home/whomever/.config, …) The first agnostic layer can be consumed both within and outside nixpkgs, enabling the simplification of projects like hm and nix-darwin, or even used outside of nixos in the context of system-manager.

I believe [RFC 0163] Portable Service Layer by svanderburg · Pull Request #163 · NixOS/rfcs · GitHub proposed something similar but slightly different.

6 Likes
15

Well, why to reinvent the wheel? HM is battle-tested after all.

Yes, and I am watching it. When this RFC succeeds, it will make HM integration way easier.

16

I’m not suggesting to reinvent anything. I’m suggesting to leverage the existing configuration generation that we have to prevent having to duplicate code.

HM has basically no quality gate, I’d rather not import that entire codebase into nixpkgs just like that. And if anything, nixos is more battle-tested, so we should use the config generation that we already have.

3 Likes
17

HM’s quality review should actually be similar to Nixpkgs’s module part, really. Although they do differ in the way they handle service.

2 Likes
18

One of the most common questions I get on Discord is how to manage HM and NixOS together - how to weave them together. So, this proposal would meet a need where people want a unified system. I cannalso see advantages wrt services having a shared definition between system and user.

There are some positives, even though my first inclination is to adopt HM into the org, this proposal is more bold.

7 Likes
19

If you’re only talking about formatting, you should have said so.

To me it reads more like “refactor all the code to my personal preferences”, especially the “unwritten rules” part.

IMHO, if a rule is important enough to be decided upon in an RFC, it should at least be written down and it should probably have community consensus too.

It reads like a recommendation to use these specific tools to me.

If you know of likely objections, those should be addressed/discussed in the RFC text.

Have you talked with anyone about this who might object?

I see two options here: Either as you either keep things duplicated (nothing inherently wrong with that) or find a technical solution. Neither of these require broad community consensus.

I don’t see deciding that to be in scope for this RFC, that is if it requires an RFC at all.

Hm, before a public discussion such as this one, I would have at least expected you to have at least cleared this with the people it’d affect the most in private.

IMHO, consent-wise, it’s enough for us Nixpkgs people to be indifferent to the merge but the hm folks must explicitly want this to happen. Otherwise, I can’t help but feel that this would be seen as an unwanted takeover.

1 Like
20

I wouldn’t call it quality gate, but rather significance check. Many modules consist of an enable, package and extraConfig option which in itself is not very useful to have if there is a generic way to create a config file and add packages to the user config.
In NixOS we only miss the way to create the user based config files in a structured way.
If there are no config checks or pre done settings or integrations into other modules, then the module lacks significance or “value” and would be likely declined in review.

Also there are good examples of what things can be managed with home-manager like gitconfig and also rather terrible ones like the config files for the Plasma Desktop.

7 Likes