How difficult would it be to port NixOS to PPC32? Any documentation on how to port NixOS to another architecture?
Background: I have a perfectly serviceable PPC Mac Mini that I would like to a run current Linux. Unfortunately, almost everyone has abandoned the PPC32 platform. I’ve built Linux from Scratch before but if I have to bootstrap Linux on the system, why not put my effort into NixOS…
As far as I’m aware there isn’t any documentation on this, but I might be able to help out. I have a colorful iMac G3, it would be fun to get NixOS running on it.
The boot sequence for PowerPC is IEEE1275 which is unlike any currently-supported NixOS platforms. That might take some experimentation to get working, so I’d recommend first installing e.g. Ubuntu or Debian as the base OS, and trying to get multi-user Nix on non-NixOS working.
For Nix on non-NixOS, we’d first want to cross-compile Nix and bootstrap tools for PPC32. This requires adding the platform definition for powerpc-unknown-linux-gnu to lib/systems/examples.nix.
That sounds like a good way to go. Although 16.04 is the last supported PPC version of Ubuntu, Debian PPC is still available (tier-2) so I don’t think we need to cross compile for architectures, only for platforms.
Thanks for the bootstrap tools link. I’ll take a look at it. (I already figured out that there needs to be a new entry in lib/systems/examples.nix.)
Time for a status update…
My G4 Mac Mini already has Debian 8 LTS installed but we will need a newer Debian, as building Nix requires newer compilers and libraries. Initially I considered installing Debian 10 but there appear to be enough problems with installing the boot loader that rather than reinstall I choose to create a chroot environment based on the 2021-02-02 ISO. Note: although the ISO says it is Debian 10, it is really Sid/Bullseye.
Make the packages from the ISO available and install using debootstrap:
$ wget https://cdimage.debian.org/cdimage/ports/snapshots/2021-02-02/debian-10.0.0-powerpc-NETINST-1.iso
$ wget https://cdimage.debian.org/cdimage/ports/snapshots/2021-02-02/SHA256SUMS
$ grep debian-10.0.0-powerpc-NETINST-1.iso SHA256SUMS | tee debian-10.0.0-powerpc-NETINST-1.iso.SHA256 | sha256sum -c # OK
$ rm SHA256SUMS
$ sudo mount -o ro,loop debian-10.0.0-powerpc-NETINST-1.iso /mnt
$ sudo apt-get install debootstrap
$ sudo debootstrap --no-check-gpg sid sid file:///mnt # not supported so no signatures
Configure apt to access the work-in-progress Debian powerpc port:
$ cat <<EOF | sudo tee sid/etc/apt/apt.conf.d/99no-recommends
APT::Install-Recommends "0";
APT::Install-Suggests "0";
EOF
$ sudo mv sid/etc/apt/sources.list sid/etc/apt/sources.list.orig
$ cat <<EOF | sudo tee sid/etc/apt/sources.list.d/deb.debian.org.list
# Have to trust; powerpc has no signatures
deb [trusted=yes] http://deb.debian.org/debian-ports sid main
EOF
Set up a chroot environment:
$ sudo mount -o bind /dev sid/dev
$ sudo mount -o bind /dev/pts sid/dev/pts
$ sudo mount -o bind /proc sid/proc
$ sudo mount -o bind /sys sid/sys
$ sudo chroot sid /bin/bash
# apt-get update --allow-unauthenticated # ignore signature warning
# /usr/lib/dpkg/methods/apt/update /var/lib/dpkg
# apt-get install locales
# dpkg-reconfigure # en_US.UTF-8
I expected to build Nix from source but because the ISO is really for Sid/Bullseye, Nix is already available:
$ sudo apt-get install nix-bin
$ nix --version
nix (Nix) 2.3.10
To summarize, I now have a G4 Mini with a Debian 11 Sid/Bullseye environment with Nix installed. That is enough for today. I need to think of what steps come next…
I am stuck making the cross compile bootstrap tools. I added entries for powerpc:
$ git diff
diff --git a/lib/systems/examples.nix b/lib/systems/examples.nix
index 389c4eebcc8..2bf62e8c310 100644
--- a/lib/systems/examples.nix
+++ b/lib/systems/examples.nix
@@ -34,6 +34,14 @@ rec {
gcc = { abi = "elfv2"; }; # for gcc configuration
};
+ ppc32-gnu = {
+ config = "powerpc-unknown-linux-gnu";
+ };
+ ppc32-musl = {
+ config = "powerpc-unknown-linux-musl";
+ };
+ ppc32 = ppc32-gnu;
+
sheevaplug = {
config = "armv5tel-unknown-linux-gnueabi";
} // platforms.sheevaplug;
diff --git a/pkgs/stdenv/linux/make-bootstrap-tools-cross.nix b/pkgs/stdenv/linux/make-bootstrap-tools-cross.nix
index d8ab96952b7..c0dc2d3b9dc 100644
--- a/pkgs/stdenv/linux/make-bootstrap-tools-cross.nix
+++ b/pkgs/stdenv/linux/make-bootstrap-tools-cross.nix
@@ -17,6 +17,8 @@ in lib.mapAttrs (n: make) (with lib.systems.examples; {
armv6l-musl = muslpi;
aarch64-musl = aarch64-multiplatform-musl;
riscv64 = riscv64;
+ powerpc = ppc32;
+ powerpc-musl = ppc32-musl;
powerpc64 = ppc64;
powerpc64-musl = ppc64-musl;
powerpc64le = powernv;
which took me to the next error message:
$ nix-build ./pkgs/stdenv/linux/make-bootstrap-tools-cross.nix -A powerpc --show-trace
error: while evaluating 'wrapBintoolsWith' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11495:5, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:12293:14:
while evaluating 'callPackageWith' at /home/mkg/src/nixpkgs/lib/customisation.nix:117:35, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11499:7:
while evaluating 'makeOverridable' at /home/mkg/src/nixpkgs/lib/customisation.nix:67:24, called from /home/mkg/src/nixpkgs/lib/customisation.nix:121:8:
while evaluating anonymous function at /home/mkg/src/nixpkgs/pkgs/build-support/bintools-wrapper/default.nix:8:1, called from /home/mkg/src/nixpkgs/lib/customisation.nix:69:16:
while evaluating the attribute 'cc.nativeTools' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/default.nix:166:14:
while evaluating the attribute 'gcc' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:10139:3:
while evaluating 'addMetaAttrs' at /home/mkg/src/nixpkgs/lib/meta.nix:15:28, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:10330:11:
while evaluating 'wrapCC' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11490:12, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:10330:20:
while evaluating 'wrapCCWith' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11464:5, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11490:16:
while evaluating 'callPackageWith' at /home/mkg/src/nixpkgs/lib/customisation.nix:117:35, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11478:7:
while evaluating 'makeOverridable' at /home/mkg/src/nixpkgs/lib/customisation.nix:67:24, called from /home/mkg/src/nixpkgs/lib/customisation.nix:121:8:
while evaluating anonymous function at /home/mkg/src/nixpkgs/pkgs/build-support/cc-wrapper/default.nix:8:1, called from /home/mkg/src/nixpkgs/lib/customisation.nix:69:16:
while evaluating the attribute 'buildInputs' of the derivation 'glibc-2.32-40-powerpc-unknown-linux-gnu' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:203:11:
while evaluating the attribute 'depsBuildBuild' of the derivation 'linux-headers-5.11' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:197:11:
while evaluating 'getOutput' at /home/mkg/src/nixpkgs/lib/attrsets.nix:482:23, called from undefined position:
while evaluating anonymous function at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:138:17, called from undefined position:
while evaluating the attribute 'stdenv.cc' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/default.nix:166:14:
while evaluating the attribute 'cc' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/default.nix:166:14:
while evaluating the attribute 'cc' at /home/mkg/src/nixpkgs/pkgs/stdenv/native/default.nix:122:5:
anonymous function at /home/mkg/src/nixpkgs/pkgs/build-support/cc-wrapper/default.nix:8:1 called without required argument 'lib', at /home/mkg/src/nixpkgs/pkgs/stdenv/native/default.nix:128:5
Adding what appears to be a missing lib attribute in the import of cc-wrapper:
$ git diff pkgs/stdenv/native/default.nix
diff --git a/pkgs/stdenv/native/default.nix b/pkgs/stdenv/native/default.nix
index b79b81253ad..2a8daaaaf82 100644
--- a/pkgs/stdenv/native/default.nix
+++ b/pkgs/stdenv/native/default.nix
@@ -129,10 +129,10 @@ in
name = "cc-native";
nativeTools = true;
nativeLibc = true;
- inherit nativePrefix;
+ inherit nativePrefix lib;
bintools = import ../../build-support/bintools-wrapper {
name = "bintools";
- inherit stdenvNoCC nativePrefix;
+ inherit stdenvNoCC nativePrefix lib;
nativeTools = true;
nativeLibc = true;
};
solved that problem. But now I’m stuck:
$ nix-build ./pkgs/stdenv/linux/make-bootstrap-tools-cross.nix -A powerpc --show-trace
error: while evaluating 'wrapBintoolsWith' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11495:5, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:12293:14:
while evaluating 'callPackageWith' at /home/mkg/src/nixpkgs/lib/customisation.nix:117:35, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11499:7:
while evaluating 'makeOverridable' at /home/mkg/src/nixpkgs/lib/customisation.nix:67:24, called from /home/mkg/src/nixpkgs/lib/customisation.nix:121:8:
while evaluating anonymous function at /home/mkg/src/nixpkgs/pkgs/build-support/bintools-wrapper/default.nix:8:1, called from /home/mkg/src/nixpkgs/lib/customisation.nix:69:16:
while evaluating the attribute 'cc.nativeTools' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/default.nix:166:14:
while evaluating the attribute 'gcc' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:10139:3:
while evaluating 'addMetaAttrs' at /home/mkg/src/nixpkgs/lib/meta.nix:15:28, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:10330:11:
while evaluating 'wrapCC' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11490:12, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:10330:20:
while evaluating 'wrapCCWith' at /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11464:5, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11490:16:
while evaluating 'callPackageWith' at /home/mkg/src/nixpkgs/lib/customisation.nix:117:35, called from /home/mkg/src/nixpkgs/pkgs/top-level/all-packages.nix:11478:7:
while evaluating 'makeOverridable' at /home/mkg/src/nixpkgs/lib/customisation.nix:67:24, called from /home/mkg/src/nixpkgs/lib/customisation.nix:121:8:
while evaluating anonymous function at /home/mkg/src/nixpkgs/pkgs/build-support/cc-wrapper/default.nix:8:1, called from /home/mkg/src/nixpkgs/lib/customisation.nix:69:16:
while evaluating the attribute 'buildInputs' of the derivation 'glibc-2.32-40-powerpc-unknown-linux-gnu' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:203:11:
while evaluating the attribute 'depsBuildBuild' of the derivation 'linux-headers-5.11' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:197:11:
while evaluating the derivation attribute 'name' at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:197:11:
cannot coerce null to a string, at /home/mkg/src/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:197:19
Apparently, either attrs.pname or attrs.version are null in line 197 of make-derivation.nix but I have no idea why and can’t find where they would be set anyway.
From your traceback it looks like stdenv is trying to use stagesNative for bootstrapping, but you want stagesLinux if you’re building bootstrap tools. Try adding powerpc-linux = stagesLinux to stdenv/default.nix here: https://github.com/NixOS/nixpkgs/blob/a8f6369dda06b3e47299f7760b47c6806c03fb95/pkgs/stdenv/default.nix#L42-L63
(AFAIR the stagesNative issue has been occurring since the stdenv.lib attr was removed, but you don’t need to fix that for your purposes)
I tried a different tack and made some progress. Following the cross compiling wiki article:
$ mkdir ppc32
$ cd ppc32
$ git clone https://github.com/NixOS/nixpkgs/
$ git diff
index c6d90ba9850..b0b5fe5e9a0 100644
--- a/lib/systems/doubles.nix
+++ b/lib/systems/doubles.nix
@@ -26,8 +26,9 @@ let
# Linux
"aarch64-linux" "armv5tel-linux" "armv6l-linux" "armv7a-linux"
- "armv7l-linux" "i686-linux" "mipsel-linux" "powerpc64-linux"
- "powerpc64le-linux" "riscv32-linux" "riscv64-linux" "x86_64-linux"
+ "armv7l-linux" "i686-linux" "mipsel-linux" "powerpc-linux"
+ "powerpc64-linux" "powerpc64le-linux" "riscv32-linux"
+ "riscv64-linux" "x86_64-linux"
# MMIXware
"mmix-mmixware"
diff --git a/lib/systems/examples.nix b/lib/systems/examples.nix
index 6a8f4e091aa..2dbecd3d7de 100644
--- a/lib/systems/examples.nix
+++ b/lib/systems/examples.nix
@@ -21,6 +21,13 @@ rec {
config = "powerpc64le-unknown-linux-musl";
};
+ powerpc = {
+ config = "powerpc-unknown-linux-gnu";
+ };
+ powerpc-musl = {
+ config = "powerpc-unknown-linux-musl";
+ };
+
ppc64 = {
config = "powerpc64-unknown-linux-gnu";
gcc = { abi = "elfv2"; }; # for gcc configuration
Now set up an empty cross compilation environment:
$ cat <<EOF > crossShell.nix
with import <nixpkgs> {
crossSystem = {
config = "powerpc-unknown-linux-gnu";
};
};
mkShell {
buildInputs = [
];
}
EOF
$ export NIX_PATH=nixpkgs=$(pwd)/nixpkgs
$ time nix-shell crossShell.nix # 44:28.17 (on i5-2500K)
$ powerpc-unknown-linux-gnu-cc --version # powerpc-unknown-linux-gnu-gcc (GCC) 10.3.0
$ nix build -f ./nixpkgs --arg crossSystem '(import <nixpkgs>{}).lib.systems.examples.powerpc' hello
$ file result/bin/hello
result/bin/hello: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked, interpreter /nix/store/i5sqay18wybbghf8ncp6gqr0b3xlf7lv-glibc-powerpc-unknown-linux-gnu-2.32-46/lib/ld.so.1, for GNU/Linux 2.6.32, not stripped
Definitely progress. I need to do some thinking of what to do next. Suggestions are welcome, of course.
I’m interested in having a NixOS for PowerPC 32bit in the long run for work. I’m right now upstreaming some changes needed to have a fully working PPC64 NixOS image, but PPC32 is also on my radar.
If hello works, I’d try to build a minimalist NixOS image, and fix things as they come.
An example config:
{ lib, modulesPath, ... }:
{
imports = [
(modulesPath + "/profiles/minimal.nix")
];
boot.initrd.includeDefaultModules = false;
systemd.shutdownRamfs.enable = false;
xdg.mime.enable = false;
services.udisks2.enable = false;
nixpkgs.crossSystem = lib.systems.examples.powerpc;
}
Another possibility would be to try and compile the kernel. You can look at lib/systems/platforms.nix to take inspiration if your kernel needs a special defconfig, or other tweaking.
Feel free to hit me up on matrix @Minijackson:matrix.org
Have there been any more work on the ppc32. I would interested in this. First I only need Cross compile, but in the long run kernel would be nice.
I have a full NixOS image that I’m using for work, which is cross-compiled from x86_64-linux to ppc32.
I try to upstream every fix that I can, but there might still be some missing.
I’m not using the mainline kernel in my case, I’m using the NXP one, but it was kind of a mess to integrate.
I hope that someday we will redo the Nix infrastructure for building the Linux kernel.