Genodepkgs - Extending Nixpkgs/NixOS to Genode

Announcements
1

During the 20.03 to 20.09 release period two new platforms were added to Nixpkg, Genode followed by Redox. Both are modern, microkernel-based operating systems designed for security and reliability.

I pleased to announce that NLnet will be funding the developing of support for Genode via the Genodepkgs project. This effort will focus on packaging and configuring Genode native software, building existing packages against the Genode POSIX library, and using Genode as a hypervisor for hosting NixOS guest VMs. The project is planned to run for the remainder of this year.

The goal is basically to build systems using the same methodology as NixOS, but to reduce the attack surface and technical debit of the base system by orders of magnitude.

This project will be limited initially to the realm of building server systems and will not extend to the desktop. If you are interesting in a secure NixOS desktop, then SpectrumOS is the project to contribute to (also funded by NGI0).

For the next month or so I will be focused on the configuration abstractions, so there will not be much visible activity until that is resolved.

For more information on Genode I recommend reading the foundations book, or watching some of the FOSDEM talks.

The NGI Zero Privacy & Trust Enhancing Technologies fund: NLnet; Privacy & Trust Enhancing Technologies
The project repository: ~ehmry/sigil - Aborted Genode/Nix distribution - sourcehut git

If you have questions, want to talk about Redox, or need help porting nixpkgs to other exotic OSes, the IRC channel is irc://freenode.net/#nixos-exotic

16 Likes
2

Update

A summary of changes in the past couple of months:

Dhall interpreter

The native Dhall interpreter for Genode is close to compliance with the language spec, but due to performance problems with the parser, this sub-project is temporarily stalled. The library can be used to construct and normalize expressions programatically, but parsing code is currently too slow to comfortably generate configuration. De-emphasizing dynamically generated configuration means more focus building on static systems defined using the traditional NixOS method.

Init

The Genode init system has been patched to accept a slightly different configuration schema than the upstream init. This change removes wildcard rules for routing RPC services between components. These rules were in place to make init configurations more ergonomic to manually describe, but with a sufficiently powerful DSL these are no longer needed. This change somewhat reduces the complexity of the init schema system and makes configurations easier to manually audit. At some point it may be practical to replace the init component for something that accepts a more Nix-friendly configuration schema, but not within a foreseeable future. Note that this init cannot appropriately be compared to systemd - systemd is +300KLOC and the Genode init is around 10KLOC, including libraries.

Linking

The LLD linker has been patched to link libraries by absolute path, that means that rather than use references like libcrypto.so, ELF artifacts will refer to full paths such as /nix/store/…-openssl-1.1.1g-x86_64-unknown-genode/lib/libcrypto.so. This explicitly establishes what the runtime dependencies of a binary are and optimizes loading times. Note Genode loads libraries externally, the loader parses the ELF header locally then makes an RPC requests by library name for a memory region containing the library. The libraries are then loaded from the file-system by a dedicated component that manages a cache of binaries and libraries. Linking libraries by absolute path as a general policy within Nixpkgs has been proposed, but has not been implemented. Genode makes this much easier by bypassing the libc during loading.

Link to libraries through absolute paths?

NixOS modules

The biggest update is the transition from a custom test framework to simply adopting the NixOS module system for building test systems. There are now NixOS modules that define which kernel to boot from and a hardware module that configures drivers and IP stacks to coincide with standard NixOS parameters.

Systemd

I am currently working on translating systemd services to Genode init children. This is achievable in simple cases by extracting config.systemd.services.….service.serviceConfig.ExecStart and requiring a few extra parameters. In a simple example this currently looks like this:

{ pkgs, ... }: {
  imports = [ ../nixos-modules/systemd.nix ../nixos-modules/hardware.nix ];
  networking.interfaces.eth1.genode.stack = "lwip";
  services.foo = { };
  systemd.services.foo.genode = {
    enable = true;
    interface = "eth1";
  };
}

Systemd translation is still a work in progress.

Nix store

You might wonder what form the Nix store takes in such a system. Currently a store closure is collected from the system configuration and packaged as a tarball. This tarball is bundled with the initial binaries that are used to bootstrap the system and is passed to the Genode core component from the bootloader via the kernel. Sub-systems may access this store tarball by a read-only file-system RPC session or using the ROM loading RPC. The entire Nix store is available to each client component, but hosting multiple store instances will soon become practical.

The current systems I’m able to build are similar to a Linux system that hasn’t booted beyond an initramfs. When I’m able to successfully translate systemd services I will add block-device drivers and file-systems, as well as a framebuffer for log output.

Project infrastructure

Hydra / CI

My Hydra is gone I need to arrange for a new one. If anyone can donate some Hydra time, please let me know. Being without a Hydra unfortunately means that the Sotest hardware tests have also stopped.

Cachix

There is now a Cachix cache that contains the custom toolchains and many of the packages.

Summary

I will keep working on translating existing NixOS configurations until at least the next update. Without proper documentation it would be difficult for additional contributors to get accustomed to working within the repository, however, if anyone would like to get involved and has a specific system they would like to build, I’d be happy to assist, and it would help me streamline the development processes.

4 Likes
3

This looks pretty neat. I’ve been learning about Genode recently and really like the concepts, and was thinking it would be great to be able to build it with nix.

I read some of the issues you posted and other material. It feels like you have a pretty strong vision of how this should all fit together. Can you articulate it a little more somewhere?

I’d personally like to have a much simpler computer system than I have now. Nixos helps with that some, but I think Genode could improve on that.

4

Sorry for the late reply. My plan shifts I as come across new inconsistencies between the two systems, but I am overdue on a project update and describing how I envision a complete system.

I’ve been avoiding building a high level demonstration because I want to be sure that the base level of the system is sound. At the moment I’m still working on how drivers are instantiated from nixos-modules and how to store and retrieve packages without relying on a system-global file-system. I’m working alone and things aren’t moving quickly because I’m trying to make the system as formal as reasonable, but I hope this will keep the result sustainable for a small maintenance crew.

2 Likes
5

I’ve stopped working on this project - post-mortem report

3 Likes
6

Getting NixOS to run on a different kernel/microkernel architecture is still something we as a community should aspire to. Obviously a difficult project for one person to undertake.

1 Like
7

Hi ehmry,
I’ve just come across this thread because someone asked about how to add support for Genode. After reading your post-mortem, I’d like to share some info that may help anyone who’s interested in such a project. While I don’t have an ambition to implement alternate kernel or init support myself I’ve been involved with changes since your last post that make another attempt or continuation a bit more feasible.

  • It’s feasible and in at least some cases faster to generate images instead of using 9p filesystem passthrough. nixpkgs#181746
  • The NixOS test framework was recently ported to the module system. This allows it to be refactored to accommodate arbitrary OSes instead of NixOS. You may be able to change out NixOS using the internal node.type option, although the options that form the interface between the test framework and the NixOS-like system is unspecified.
  • NixOS can be refactored so that modules only depend on what they need. This speeds up evaluation and reduces memory requirements. (I suspect that Dhall may have worsened your evaluation requirements as well, but I haven’t looked into the details to confirm this; apply a ~grain~ ton of salt).
  • The NixOS module system can accommodate a lot of flexibility, but the required techniques are underdocumented. (And at least one technique was unavailable; to retroactively add a RFC42 style freeform type to existing option trees. I don’t know if you needed this, but it was only supported since half a year. nixpkgs#156533. Infinisil also contributes fixes and improvements every now and then.)
8

Thanks for pinging me, since you’ve mentioned it I’ll try and update to a recent Nixpkgs.

Yes, I would prefer to use EROFS during system bootstrap, but I need a native driver for it. I think its worth writing one but its a low priority.

I will look into refactoring how I use the modules, but only after I can be sure I have the toolchains up to date, which can be tedious. It would make things easier for me if I could push Clang patches to nixpkgs but I’m not sure its justified.

9

EROFS

EROFS isn’t the only fs that comes with a tool to construct images in user space. For instance mkfs.ext{2,3,4} take a -d option for this purpose. I just didn’t pick ext* because -d would require an extra copy. Patching the tool to take multiple -d seems easier than adding EROFS to another OS. SquashFS may also be a good choice if that happens to be supported. It’s quick with compression/dedup disabled.
Also, turns out @lheckemann is working to upstream a significant performance fix for 9p in qemu, so that’s still worth considering.

look into refactoring how I use the modules,

I feel like I might have made it sound too easy, the refactoring NixOS part. Modifying a large code base never is easy, but if you have the patience to do it, it is worthwhile.

push Clang patch to nixpkgs but I’m not sure its justified

You can always ask. They’ll probably want to know much patching is needed. If it’s not too much effort you can make the PR. If it is too much effort, well maybe it doesn’t make much sense to do it until the project becomes more popular.