NixOS as OpenVZ 7 guest

Hi, so I’m trying to run NixOS inside an OpenVZ 7 container. The host is running a 4.15.0 kernel, and before I go further, this is my situation:

  • I cannot get a shell on the hypervisor itself (this is a VPS provider)
  • I cannot reproduce the failure locally. NixOS boots just fine on my own OpenVZ 7 host after replacing /sbin/init in an Ubuntu 18.04 container.

So I made my rootfs with nixos-generate -f lxc, with the following configuration change:

# ramfs is not available on the host
boot.specialFileSystems."/run/keys".fsType = lib.mkForce "tmpfs";

(This setup works fine for a couple of LXC providers)

Shortly after systemd was started from stage 2 (I do know systemd was started), it crashed. As it’s not possible to get a rescue shell (wrapper of vzctl enter, which starts /bin/bash hardcoded) when PID 1 is not running, I created a hacky /sbin/init that starts the stage 2 script on every second boot:

#!/nix/store/a3fc4zqaiak11jks9zd579mz5v0li8bg-bash-4.4-p23/bin/bash

systemConfig=/nix/store/ia9qb52bk8whv7v20sg4l85rcc1s2zj2-nixos-system-nixos-20.09pre-git
export PATH=/nix/store/z1qvlavy35wanw5k54fvvfffws5bvigj-coreutils-8.31/bin

date >> /boot-log

val=$(cat /boot-mark)
mod2=$((val % 2))
echo $((val + 1)) > /boot-mark

if [ $mod2 == "0" ]; then
	echo "booted to rescue" >> /boot-log
	$systemConfig/activate
	while :; do sleep 1; done
else
	echo "booted to system" >> /boot-log
	exec $systemConfig/init 2>&1 >> /boot-log
fi

I’m positive that systemd was started as I was able to get into the rescue shell in the brief window before it crashed and saw systemd running as PID 1.

However, I’m unable to obtain useful logs from systemd. I have no access to /dev/console or /dev/tty*, /var/log/journal is empty, and I don’t see a way to coax it to emit anything to stdout/stderr (I did remove the lines that reset the stdout/stderr descriptors from the stage 2 init).

Any ideas of how to proceed? In case you would like to waste some time on this yourself, I’m using Gullo’s Hosting and they are running Christmas deals. The 128MB bundles are pretty cheap, so you may want to give it a go if you also feel like running NixOS in some cursed environment :slight_smile: