Brainstorm for RFC: Assimilate home-manager into Nixpkgs monorepo

Ideally both channels would be merged.

The modules are very readable, similar to those from NixOS.
The harder part is the Bash scripts, as expected.

This is basically what HM does - it uses the existing module system in order to generate configuration.
The duplication happened mostly because they are not in the same repo.

I beg to differ.

I announced my intent in their Matrix channel, a not so long time ago. For the sake of announcement I will do another right now.

There’s no real testing or even code review, stuff just gets merged if it “looks good”. For concrete examples, the sway and waybar modules were so awful I reworked my config just to get rid of HM. Even their systemd “abstraction” is not really an abstraction, just a straight toINI that doesn’t account for upstream service files.

Again I want to reiterate: the need is not for HM, the need is for user-level config management. The only reason HM keeps coming up is that it’s currently the only well-known option in that space. We can do much better.

4 Likes

I was not so lucky in my personal experience.

How much longer we should wait for those “we” that “can do much better”?

3 Likes

I didn’t say it with folding HM into nixpkgs in mind, but I have previously wondered aloud if moving module-related utility logic towards package passthrus might build leverage by enabling more reuse across module systems and dev shells.

I think the last time I mentioned it was in:

6 Likes

Yes we should merge it into the Nixpkgs/NixOS repo, so we can start deduplicating with NixOS, as others have said.

5 Likes

Kde configuration requires to alter their files in place as it too often uses the same file for both configuration and state

1 Like

I think we should create several middle grounds between the package and the NixOS modules (which must be used as a whole for now), including service abstraction, wrapper abstraction and dotfiles abstraction. We may need several RFCs to advance this.

4 Likes

The RFC process is all out of sorts (as someone far removed from it, admittedly) but perhaps folks can point me to stuff for nix RFCs and such that sorta kinda mirror what’s in here (process, roles, etc) https://www.rfc-editor.org/rfc/rfc6635.txt - we’re not that important but seems like a good document for sharper eyes to peruse and gather takeaways from (if they have yet to do so). Anyway, I sorta like this idea and the need for “user-level configuration management” grows by the day

I think we need not more plans, but more experimentation.

It’s hard to reduplicate when things are in different repos, and we just don’t know what sort of abstraction a 3rd repo that NixOS and home-manager should both use should look like. Conversely, with both things in the same repo, we can start deduplicating individual modules and iterative discover what the body of common modules should look it.

Same reason I would want to through NixBSD in the someday and then iteratively deduplicate.

13 Likes

I guess after merging the HM we should then also merge nix-darwin and possibly system-manager?

4 Likes

Yes, good point! This is the first I am learning of system-manager too; that is a nice idea that should be able to be gotten almost “for free” as midpoint between all the others

2 Likes

Why should everything be in a monorepo though? In the 3.5 years I’ve been using nix the nixpkgs repo has grown in size from roughly 19MB to 42MB. But the software I use has not doubled. So effectively I’m wasting more MBs per usage of nix-packaged software than I did a couple of years ago. And it’s not just me, this is the case for every single user of nixpkgs. If we add Home Manager, nix-darwin, system-manager, maybe other stuff, that percentage of wasted bandwidth and storage will only grow. I have really noticed the substantial increase in size in download times which are becoming increasingly more annoying to put up with.

I get the point that it is easier to experiment in a monorepo, but on the other hand it’s nowhere near impossible to experiment across many repos. It might be worth experimenting with a more modular setup of the repositories to make code sharing possible without just putting everything in nixpkgs and exploding its size.

4 Likes

While you’re at it, you might as well merge nixos-hardware into nixpkgs too because why not ;p

4 Likes

As far as I remember, NixOS the operating system was a separate repo, but it lagged behind Nixpkgs the package collection.
Before the creation of overlays and flakes, it was a hard thing to sync them.
Now that they live in the same monorepo, it is way easier to update both a Nix package and its corresponding NixOS module.
Also it is way easier to make integration tests for each of them.

A long time ago I was talking with Okada about splitting NixOS into its own repo, but we never tried to do it.
Nonetheless I still advocate for a monorepo. Filling three PRs for a single package is annoying enough.
Further, NixOS is less than 8% of Nixpkgs monorepo, and the whole HM (plus dotgit) occupies just two NixOSes.

This particular problem will not disappear, since we do not have titanic barriers to include software.
Also, I have vented my complaint with multirepos a short time ago:

Hey, are you stealing my idea? Nops, I steal that idea before

3 Likes

This is somewhat of a tangent, but how code is organised and how its artefacts are deployed are two independent questions. Maintaining code in one file system tree and with one version history simplifies its manipulation a great deal. It doesn’t mean that the result has to be shipped in the same shape. One can automatically split out components into separate units so they can be deployed independently. This requires maintaining clean internal boundaries, and while that’s a bit of effort, it’s a lot less effort than working around stronger but inflexible boundaries, such as separate repositories.

Many years ago I thought that repositories are natural component boundaries and their rigidity helps with maintaining interface discipline. If you dig around you’ll even find me posting about this. But with experience I realised that this completely neglects the value of being able to do sweeping atomic changes, the high cost of learning techniques or building tools to work across separate repositories, and the low cost of automatically producing simpler units of deployment from a unified code base.

In the end it of course doesn’t matter where the code lives. What matters is how much effort it is to work with it for today’s and tomorrow’s maintainers, those people who can really make a difference. The most valuable thing we have is people’s attention and time, and we should optimize for that.

11 Likes

Wow thanks for the numbers! I’ve never thought about the relative sizes before.

Here is a fun thought experiment: even if we decided to split back out NixOS from Nixpkgs, I would still want NixOS, Home-Manager, Nix-Darwin, System-Manager, etc. in the same repo. The “module-using things for system configuration” should be much more tightly integrated with each other than with the bare package set.

8 Likes

While small and insignificant, NixNG (or whatever I rename it too when I get a good idea) would benefit too, many modules are partial copy-pastes from NixOS, the only bits that differ are the actual config attrs. I have thought about pulling in options from NixOS but it would be hard to keep in sync.

1 Like

i love the idea of a common options file and then each implementation writing their own separate file containing config

some argue that’s duplication but i believe without that you run into too many problems

It looks like the RFC 163 is way fundamental, paving the way to all those projects:

  • home-manager
  • nix-darwin
  • nixos-hardware
  • system-manager
  • NixNG

and why not,

  • wrapper-manager
3 Likes

any idea how that is progressing? i believe there hasn’t been too much activity lately