I understand that the post author starts with a “minimal” nixOS image of around 900MB of disk space, and ends up, after significant reduction efforts, with a system size reduced to approximately 447MB. The author’s conclusion is very pessimistic (“But doing it on top of NixOS currently feels like a bad path to take.”).
As someone who need to create small nixOS images, this post rings a bell (I would not remove Python myself, though, as this is an essential part of the systems I am building).
Has this post been commented here before?
Are there additional insights that the post author, or the HN commentors, might have missed?
Just wondering what this re-occurring obsession is with “minimizing” the size of OS images is all about. I get what they are doing, but rarely it’s written about why this seems to be such a deal-breaker to a point where they are calling it a bad server OS.
Especially running microVMs like the author seems to be doing, I can just share the host Nix Store between them. So if they all run the same base-config, N microVMs will take the same amount of space for the base-system as one will.
So, could anyone enlighten me on why this is such a big deal?
Yes, for some use cases, sharing the nix store between “isolated” environment may be an option. Not for all (e.g. real VMs, or when accessing host directories is not possible or impractical from the guest environment, or in Edge / IoT scenarii).
That was a surprisingly enjoyable article despite the title
I didn’t dive into what they actually want to run inside of those microVM’s, but usually when people are trying to create really small images it might be a better fit to use Nix to build a non-NixOS image - like with dockerTools for docker images. I wonder if something similar exists for microVM’s, or perhaps they’re trivial enough that they don’t even need such build infrastructure?
The answer is generally that NixOS is not optimized for minimal images, and that the maintenance effort to offer that is more than the community can handle today. If you want that, roll your own distro with nix, perhaps reusing the bootstrap from nixpkgs, but building all other packages above that by hand. That’s what most of the embedded space does already anyway, just typically on top of yocto.
If that’s too much effort, as the other comments say, it’s worth inspecting whether you actually need minimization of the actual images, and why. Unless you’re in the embedded space, there’s almost always a better solution than to try and hack off half the functionality of a distro to save a few thousand bucks on an enterprise deployment. That, or your use case is bespoke and important enough that you probably should invest the time to carve out your own use case, this is all FOSS after all.
Nice blog post - thanks to the author! (not sure if they’re on here?)
Re minimizing NixOS, perhaps the alternative direction is build something NixOS-like that is minimal, and then add back compatibility for some subset of options and services? As alpine is to debian/ubuntu, kind of. If so, maybe something like @telent’s Liminix would work as a base?
One reason it matters is because it saves time. In isolation it may not seem like a big deal but pushing and pulling millions of unneccessary bytes in pipelines again and again slows the CI/CD feedback loop that ideally should be as short as possible.
In large organisations also small differences add up and may make optimization on this level worthwhile. If you develop applications to be deployed as containers this may matter.
Likewise you can have security scanning of built images, and the larger they are, the more to scan, and also adds time.
Additionally, if Python 3.X has a security problem and the scanning software flags this, you have a problem you rather would have avoided, irrespective of it being a “real problem” or not. (Using Python here as an example as the article mentioned it as a mostly useless dependency for the author’s case)
Nix is hungry for disk space, memory, and network bandwidth, and little if anything has been done to improve this situation. The structure of NixOS modules and many of the default options for packages often exacerbates this. I can appreciate that the projects are volunteer-driven and there’s little to no interest into improving this, and that’s why it is this way.
Actually I just recalled one project that I’ve seen repeatedly over the last couple years:
I don’t know how usable it is, I never tried it. But it claims to be a lot more “minimal”.
If you plan to use it, I’d suggest setting up a central builder and cache for the other machines to take advantage of, otherwise you’d end up building everything each time you deploy on a new possibly underpowered machine.
