๐Ÿ”— distri: a Linux distribution to research fast package management

https://michael.stapelberg.ch/posts/2019-08-17-introducing-distri/

Another distro that is trying to re-think packaging. I like that itโ€™s using a squashfs image per store path. I suspect that unpacking NAR files onto the filesystem actually takes quite a long time for Nix.

1 Like

Itโ€™s about trade-offs, I think. I actually like that nix doesnโ€™t restrict you on filesystem choice and doesnโ€™t require โ€œmangling the mount namespaceโ€ when running packages (by which I mean containers, virtualization, etc.)

It seems like distri is using FUSE to create an overlay on the /ro prefix (their /nix/store equivalent) and avoid mounting 1000s of filesystems. The trade-off is probably that installation is very fast but reading from the store is slower due to the FUSE<->kernel context-switching overhead.

Still, we could have both modes in Nix. Just like sandboxing has to be disabled inside of docker containers.

2 Likes

Some more interesting discussion over here: https://lobste.rs/s/yiumwr/distri_linux_distribution_research_fast

It sounds like they could (should?) use the in-kernel OverlayFS for this, which gives native performance after files are opened.

It sounds like they could (should?) use the in-kernel OverlayFS for this, which gives native performance after files are opened.

I would guess that a FUSE union filesystem is easier to reconfigure on the fly without closing the files untouched by the change.

There is also this promising ExtFuse project. The idea is that some parts of the FUSE driver can be loaded in the kernel as eBPF extensions. Depending on what the FUSE driver does this can speed up quite considerably.

I looked into it a bit, turns out they did use OverlayFS originally but it has (had?) really long mount times when overlaying many directories.