I suggest more concrete steps: how about maintaining the abstractions we already have in Nixpkgs and improve them?
For example, for both stage-1 and networking setup we have two independent implementations obeying the same set of options: one is using shell scripts, the other using systemd. Let’s start by not talking about removing the script variants and instead focus on making them 100% functionally equivalent (they’re not) and writing more tests.
Then we could think about providing (and maintaining) abstractions for minor (but widely used) features like systemd.network.links (udev rules as alternative) and systemd.tmpfiles.rules (pretty easy to do with shell scripts) and so on.
Finally one could think about doing systemd.services.
So you are suggesting to reimplement parts of systemd feature-to-feature and bug-to-bug in stringly typed shell code and then maintain it ourselves or am I misunderstanding something here?
If so please excuse my bluntness but I fail to see a value here except that it is no longer C code that Poettering started. But we would have a lot more development and maintenance burden and loose the work that went into systemd and could no longer count on the systemd maintainers and other distros to find bugs and implement features.
Summed up: not invented here
I agree that networking would be a really great place to start. Both the script based and networkd based implementations are good, but can confirm they definitely aren’t interchangeable currently, especially if you are statically setting up ips and routes
No, you completely missed my point. It’s not about getting rid of systemd or reinventing the wheel: it’s about making the backend independent from the NixOS module system.
The init system being systemd, SysVinit, launchd or whatever should be abstracted away and the best way to do this is:
not making NixOS any more dependent on one particular backend
maintain and improve the abstractions we already have
provide more similar abstractions
But we would have a lot more development and maintenance burden and loose the work that went into systemd
It’s inevitable if you want to bring NixOS to platforms not supported by systemd, which is basically just non-embedded Linux.
While I agree this is the best use case, I’m not sure just removing the systemd dependency is sufficient, considering how hard it is to deploy NixOS to such systems anyway. I’ve yet to see a NixOS that didn’t need some 20GB of disk space available, which is usually far too much.
I suppose it would be a first step to enabling people who want embedded systems, and with some iteration the other problems could slowly be ironed out. But I feel like systemd isn’t the main problem when it comes to disk space.
If the goal is to enable embedded uses cases, perhaps a new embedded profile and such should see some looking at first, and if this gets NixOS into a reasonable range then perhaps someone with an actual embedded use case could chime in on what other blockers there are? Or maybe it is discovered that systemd makes up a huge portion of the remaining size, at which point there is a clear motivation for this.
We should do that regardless and develop more abstraction with some direction, not just plan-less start to abstract systemd out and later notice that this wasn’t the biggest problem in the first place.
That is also a general problem right now regardless of this discussion. We need a smarter way to describe dependencies while keeping the amount of duplications in the nix store low. I imagine a system where all required features are collected and then the library is build with all of them which are then consumed by the packages. But this would trigger a lot of configuration specific rebuilds and likely requires CA derivations to be viable at all.
And for embedded devices we would need to drastically reduce the minimal footprint of many many packages and likely replace some of them. There is a reason why OpenWrt has its own package repository and is from the ground up build for embedded device which have very limited resources and not just a flavor of another Distro. Many of the decisions done do not make sense on a a normal sized computer where eg. 30 MB of extra disk usage do not matter at all. Keeping embedded systems in mind with every dependency change will put a lot of extra work on reviewers and maintainers alike and the situation would likely be similar to pkgsMusl/pkgsStatic which are regularly broken by updates.
In my opinion it would make way more sense to design NixOS for embedded devices from the ground up with them in mind while trying to reuse as much of NixOS as possible instead of adapting and slimming down NixOS from its bloated point. This would also ensure NixOS can develop without keeping embedded devices in mind with every change.
Then the ground up design can freely choose its init system, change the module system to be more adaptable friendly which would also aid home-manager a lot but also would be free to do its own service collection which fit its very special and different requirements.
I would love to see this. My experience with embedded integration systems (well, mainly yocto/bitkbake) hasn’t been convincing so far. A nix-based distro would be awesome for this. I’m not sure I see industries moving to it anytime soon with the certification those kinds of systems often need, though.
I also think the whole nix ecosystem is a little too nixpkgs centric for this to happen easily. Lots of third party flakes, deployment tools, and even just the nixpkgs lib are such a shame to lose, while not actually being particularly distro-specific. As nice as nixpkgs is, it shouldn’t be this hard to go without it.
It’d be very nice if some kind of “core nixpkgs” api could be abstracted instead - it would make experiments like “what if I just removed systemd?” much easier too
But yeah, probably not the right place for that discussion. I agree the systemd abstraction is not really worth it without a real use case. Though I also understand why it is so tempting, from the eye of a software engineer (especially if you dislike Poeterring, which I can’t really relate with knowing nothing about him beyond systemd/pulse/some cool blog posts on signed boot, so maybe I’m biased).
The idea of a “from the ground up” novel distribution using Nixpkgs certainly interests me. I was thinking on something like that, a “linux from scratch” but injecting Nix (and Nixpkgs) as soon as possible.
I agree that systemd isn’t really ideal for the embedded case. What would really be awesome in these space constrained situations is a file system that is aware of network focused dedupping algorithms like casync. Just found this issue, for example.
Would go long way to save space in constained environments, but it would also open the door for us to use this on the remote cache end to make the network traffic more efficient in general.
The current implementation of the module system in NixOS is also probably too heavy for these cases, but something like RFC 22 could help that in the future.
I know of at least one case where something similar is in production. A friend of mine is working in a new-space startup that uses Alpine + nix on their satellite. Basically, NixOS was too heavy for them and they had trouble with some proprietary binary drivers. As far as I understand, they now build a base image on Alpine with the proprietary stuff, add a single-user install of nix on top of that and then use that to manage dependencies and updates to the majority of their firmware. They can roll back all the critical parts of the firmware as desired, and the base image doesn’t need to be updated in space, so they just leave that alone, making all firmware updates as risk-free as they would be on full-blown NixOS.
I think until something like a minimal embedded-friendly version of NixOS exists, this will be the way to go for embedded.
For work I’ve made a custom embedded OS using nix which is probably not too different than Not-OS. We now need better networking capabilities so I am experimenting with switching from runit to systemd (for networkd in particular).
Also, prior to this need I have continuously struggled with packages bringing in systemd as a dependency. They usually have switches to disable the dependency, but hunting all that down has been a pain. I’ve had cases where a package did not depend on systemd, but after an update suddenly started depending on systemd and then I would have to go figure it out. I am looking forward to not having to do that anymore after this systemd integration.
Nice. I know this has been controversial due to the systemd, anti-systemd debate, but a service layer can really help us generalize the ecosystem outside of Linux (BSD, MacOs, etc), not to mention help out where systemd isn’t needed or wanted.
Jup, that’s the one. The company is called OroraTech, they do wildfire detection with satellites. The longer talk from Nixcon 2022 might also be interesting: https://www.youtube.com/watch?v=RL2xuhU9Nhk
