Hi, I’m relatively new but managed to build netboot output as follows. At the top level I’m running nix-build ./build.nix, which roughly looks like this:
let
bootSystem = import <nixpkgs/nixos> {
configuration = import ./configuration.nix { };
};
pkgs = import <nixpkgs> {};
in
pkgs.symlinkJoin {
name = "netboot";
paths = with bootSystem.config.system.build; [
kernel
netbootRamdisk
];
preferLocalBuild = true;
}
This all works fine, but even with the minimal profile, systemd is pretty large and contains stuff like the systemd-importd unit that I don’t need and that pulls in other packages. I’d like to use pkgs.systemdMinimal instead, or even better, configure the individual arguments to pkgs.systemd (such as withImportd) myself.
My problem is that I can’t seem to work out how to do this… I’ve tried the following:
systemd.package = pkgs.systemdMinimal;
… but this raises an error that leads me to believe I need to apply this change at a higher level?
This is because NixOS enables systemd-oomd by default. You can disable it with systemd.oomd.enable = false;
Note that NixOS enables a number of systemd units by default that systemdMinimal may not have, and I’m not sure if all of them have convenient switches like that. The systemd.suppressedSystemUnits option allows you to force it not to include units you don’t want though (whatever the consequence may be). EDIT: Looks like they properly gate those default units on their features being enabled, so it should be good, though I haven’t tested.
Thanks for the idea. It so happens I do want oomd but for sake of the argument let’s assume that I don’t – how can I avoid having to include it? Sorry if I didn’t make this clearer.
By the way, I think the size of pkgs.systemd is not large.
True, but the services (e.g. importd) pull in other deps (e.g. gnupg at 10 MB disk size.) I’m trying to get this image as small as possible. If you look at nix-store --query --tree, systemd pulls in lots and lots of stuff.
You can disable it with systemd.oomd.enable = false;
Thanks for the suggestion but I’ve tried that, the unit won’t get loaded but it will still get linked in – for example, with systemd.importd.enable = false I still end up with gnupg in the store.
Do you know why the error message mentions example? It sounds like some example code complains, but why is it trying to link an example into my build?
$ ls -d /nix/store/*importd*
/nix/store/63i3cyjf1janbincpzg03ql01ajnilfg-unit-systemd-importd.service-disabled
$ ls -d /nix/store/*gnupg*
/nix/store/1nwdvd4bkac2g3a3sd84qbfi4mf07l3y-gnupg-2.3.7
$ nix-store -q --referrers /nix/store/1nwdvd4bkac2g3a3sd84qbfi4mf07l3y-gnupg-2.3.7
/nix/store/1nwdvd4bkac2g3a3sd84qbfi4mf07l3y-gnupg-2.3.7
/nix/store/hlf86vmyh14scxwann44fy3azvc6njaj-systemd-251.7
$
In pkgs/os-specific/linux/systemd/default.nix I can see only one mention of gnugp and it’s guarded behind ++ lib.optionals withImportd. I think this should mean that systemd pulls it in only if withImportd is true… so it must be including the systemd package somewhere else with different flags. How could I find out where that’s coming from?
I hope you’re doing a GC or something between these experiments, because these paths don’t just get deleted when you change to different ones.
Anyway, the other problem is that there are lots of packages that depend on pkgs.systemd, rather than the systemd you have configured for your system. Changing that would require recompiling massive swaths of nixpkgs. So although you’re probably successfully configuring NixOS to use a minimal systemd by setting systemd.package = pkgs.systemdMinimal;, other packages are probably pulling in pkgs.systemd because otherwise you’d have to recompile a lot of nixpkgs.
And just to be clear, this:
systemd.services.systemd-importd.enable = false;
is not necessary with systemdMinimal or any systemd with withImportd = false, because the upstream unit is simply not included if that setting is false.
Ah, thanks, that’s extremely helpful. I had a feeling my mental model is a bit off.
I hope you’re doing a GC or something between these experiments, because these paths don’t just get deleted when you change to different ones.
I’m not doing GC at the build level, but the store I’m checking is inside the netboot image, which is built from scratch every time… I’m pretty sure that would be clean without gc, right? Because that’s a store rebuilt from scratch for every build?
I’ll try rebuilding after a GC now just to make sure.
Changing that would require recompiling massive swaths of nixpkgs
If you mean reconfiguring manually, that would be unfortunate. But if it’s only about recompiling, I don’t mind that at all. It doesn’t take that long to build the whole image from scratch, an hour maybe including the time it takes to build the custom kernel and modules.
Do you know of a relatively simple way to globally replace the systemd package?
And of course there’s also the potential for the other packages in the OS to depend on things you’re not happy about indirectly. Making NixOS properly minimal is not an easy task.
Amazing, I think that’s exactly what I was after. Going to give this a try now (and will report back on how long it took to build stuff… bracing myself.)
Making NixOS properly minimal is not an easy task.
Yeah I realize that, and I don’t need it to be minimal in the mathematical sense… just trying to chop off a few low hanging fruit. I’m hoping that with a more minimal systemd I can reduce the image size from ~400M to something like 2-300? But not at any price, curious now how long this will take!
Anyway, really appreciate your help and patience here, thanks!
Ah wait, actually I have one more question. When it comes down to it, I don’t actually care that much about the systemd unit itself, it’s more the indirect (unnecessary) dependencies that blow up the image size. So if this is going to turn into a half-day build, do I have any other options? For example, is there a way to tell Nix to ignore the systemd->gnupg dependency?
I decided to see how much a minimal nixos config would depend on systemd if you don’t do the overlay. To make it happy to use systemd.package = systemdMinimal;, I had to do this:
A lot of it seems to ultimately come down to util-linux, btrfs-progs, and mdadm having direct dependencies on systemd, but I didn’t look all that closely.
Not really? Those dependencies aren’t really classified as “unnecessary”. If a store path refers to another store path, then it has to exist. Otherwise usage of the first one could be completely broken.
I do understand that this is one of Nix’s tenets and it’s what I love about it… I was just wondering if there was an escape hatch for situations like this one.
Oh right that makes sense… systemdMinimal is defined in terms of systemd, so you can’t redefine systemd in terms of systemdMinimal. I think you’d just have to do systemd = prev.systemd.override { ... and copy the stuff from the systemdMinimal expression in nixpkgs unfortunately.
Ah right, my mistake, prev.systemd.override works, I had pkgs.systemd.override.
That’s totally fine, I want to customize the flags individually anyway. Build is off!
Build took ~35 minutes, about half of which was for kernel + ZFS modules and the other half added by the change. Not too bad really. And… great success, gnupg is gone!!! Very happy now. I’ll start chopping off more bits (so far only added withImportd = false).
I’ll see that I can report back here in a few days with total MB saved and re: removal of which features might cause other packages to break.
Thanks again @ElvishJerricco for helping me understand this all better (thanks @mlyxshi as well for chiming in)!
Most VPS have more than 512MB RAM. Therefore, I think it is unnecessary to reduce the initrd size. I’m curious about the size of your netboot initrd and total MB saved