Can't build bcachefs kernel

Help
1

I’m trying to build a bcachefs enabled ISO, since I have a spare laptop with 500GB HDD and 16GB SSD.

The definition file is from the Wiki and is simple:

[excalibur@nixos:~/nixpkgs]$ cat ~/Downloads/iso.nix 
# This module defines a small NixOS installation CD.  It does not
# contain any graphical stuff.
{ config, pkgs, ... }:
{
  imports = [
    <nixpkgs/nixos/modules/installer/cd-dvd/installation-cd-minimal.nix>
  ];
  boot.supportedFilesystems = [ "bcachefs" ];
  isoImage.squashfsCompression = "gzip -Xcompression-level 1";
}

I’m building with the following command (NIXPKGS_ALLOW_BROKEN is needed for building against tag 22.11 since Package ‘zfs-kernel-2.1.6-6.0.10-bcachefs-unstable-2022-10-31’ in /home/excalibur/nixpkgs/pkgs/os-specific/linux/zfs/default.nix:197 is marked as broken):

NIXPKGS_ALLOW_BROKEN=1 NIX_PATH=nixpkgs=/home/excalibur/nixpkgs nix-build '<nixpkgs/nixos>' -A config.system.build.isoImage -I nixos-config=iso.nix

However. when building against nispkgs HEAD, I’m keep getting the following error:

...
/build/source/include/os/linux/spl/sys/kmem_cache.h:204: warning: "kmem_cache_alloc" redefined
  204 | #define kmem_cache_alloc(skc, flags)    spl_kmem_cache_alloc(skc, flags)
      | 
In file included from /nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/source/include/linux/fs.h:45,
                 from /nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/source/include/linux/huge_mm.h:8,
                 from /nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/source/include/linux/mm.h:737,
                 from /build/source/include/os/linux/kernel/linux/simd_x86.h:160,
                 from /build/source/include/os/linux/kernel/linux/simd.h:29,
                 from /build/source/include/os/linux/spl/sys/simd.h:28,
                 from /build/source/module/zfs/vdev_raidz_math_avx512f.c:31:
/nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/source/include/linux/slab.h:468: note: this is the location of the previous definition
  468 | #define kmem_cache_alloc(_s, _flags)    \
      | 
make[3]: *** [/nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/source/Makefile:1992: /build/source/module] Error 2
make[3]: Leaving directory '/nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/build'
make[2]: *** [Makefile:55: modules-Linux] Error 2
make[2]: Leaving directory '/build/source/module'
make[1]: *** [Makefile:934: all-recursive] Error 1
make[1]: Leaving directory '/build/source'
make: *** [Makefile:795: all] Error 2
error: builder for '/nix/store/v1sss8piiky2skp6l5457aa78gb8nvsl-zfs-kernel-2.1.9-6.1.3-bcachefs-unstable-2023-02-01.drv' failed with exit code 2;
       last 10 log lines:
       > /nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/source/include/linux/slab.h:468: note: this is the location of the previous definition
       >   468 | #define kmem_cache_alloc(_s, _flags)    \
       >       |
       > make[3]: *** [/nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/source/Makefile:1992: /build/source/module] Error 2
       > make[3]: Leaving directory '/nix/store/b1dh4w40qx38m10frrn4cr7rjz0x9lpn-linux-6.1.3-bcachefs-unstable-2023-02-01-dev/lib/modules/6.1.3/build'
       > make[2]: *** [Makefile:55: modules-Linux] Error 2
       > make[2]: Leaving directory '/build/source/module'
       > make[1]: *** [Makefile:934: all-recursive] Error 1
       > make[1]: Leaving directory '/build/source'
       > make: *** [Makefile:795: all] Error 2
       For full logs, run 'nix log /nix/store/v1sss8piiky2skp6l5457aa78gb8nvsl-zfs-kernel-2.1.9-6.1.3-bcachefs-unstable-2023-02-01.drv'.
error: 1 dependencies of derivation '/nix/store/770d5gckw082swp8ngbwm1gg3q635hqj-kernel-modules.drv' failed to build
error: 1 dependencies of derivation '/nix/store/62m4hmkf4znhalhb3rbdffss4cp2ywyv-linux-6.1.3-bcachefs-unstable-2023-02-01-modules.drv' failed to build
error: 1 dependencies of derivation '/nix/store/4jbi94vb84vi2p6qpb4kas5a9nnn2chi-linux-6.1.3-bcachefs-unstable-2023-02-01-modules-shrunk.drv' failed to build
error: 1 dependencies of derivation '/nix/store/vcr6zyy2r221chlrzlr4cwybv8c1fn5a-nixos-system-nixos-23.05pre-git.drv' failed to build
error: 1 dependencies of derivation '/nix/store/hn27m605ih2gxp19b2wgpiqz0kpw94kl-closure-info.drv' failed to build
error: 1 dependencies of derivation '/nix/store/kvcbnahanrp6lq3bx6nybmcjhf5gpb0h-efi-directory.drv' failed to build
error: 1 dependencies of derivation '/nix/store/ip403ixl5dyvhfag8kr3h6ldnymg3k9s-isolinux.cfg-in.drv' failed to build
error: 1 dependencies of derivation '/nix/store/bxf2cccahd4zjzzj7rhflg1swag8xcl8-stage-1-init.sh.drv' failed to build
error: 1 dependencies of derivation '/nix/store/hi1m2fx97jdi9rvg8q3ka9jg4arxdvpi-nixos-23.05pre-git-x86_64-linux.iso.drv' failed to build

which I think should be buildable according to #214265.

Switching to tag 22.11 to build linux-6.0.10-bcachefs-unstable-2022-10-31 is successful:

...
xorriso : UPDATE :  4.09% done
xorriso : UPDATE :  29.46% done
xorriso : UPDATE :  47.45% done
xorriso : UPDATE :  70.71% done, estimate finish Wed Apr 12 05:23:37 2023
xorriso : UPDATE :  88.22% done
ISO image produced: 2299752 sectors
Written to medium : 2299752 sectors at LBA 0
Writing to 'stdio:/nix/store/1klfpd1b6i8pdrkby1pldyd65w923ggj-nixos-22.11pre-git-x86_64-linux.iso/iso/nixos-22.11pre-git-x86_64-linux.iso' completed successfully.

/nix/store/1klfpd1b6i8pdrkby1pldyd65w923ggj-nixos-22.11pre-git-x86_64-linux.iso

My system is running 22.11 with a few unstable GUI application packages and kernel from unstable. Those should not affect the build environment.

@PedroHLC can you verify this?

2

Well, it’s marked as broken for a reason. That’s the derivation that’s failing to build. ZFS doesn’t build with the bcachefs kernel currently. The kernel is building fine. You can build the ISO without ZFS by using installation-cd-minimal-new-kernel-no-zfs.nix instead.

1 Like
3

I have a PR open as draft: linux_testing_bcachefs: 6.1.3-unstable-2023-02-01 -> 6.2.2-unstable-2023-03-06 by PedroHLC · Pull Request #219446 · NixOS/nixpkgs · GitHub which works out-of-box.

Kent ported some of the bcachefs-tools to Rust. And last I checked, you now need to re-enter password in the mount utility, which does not support piping stdin and breaks our tests (and may break final users’ usage flow).

Also Note: ZFS needs zfsUnstable with 6.1+

1 Like
4

I don’t use ZFS so I only pass that option to generate an ISO to install a bcachefs system (on-going right now). But it’s good to know that there is a proper way to handle it.

Thanks. I only searched the forum for reports, and I was not aware of that PR. I’m glad there is some progress to fix this issue.

5

To be clear, the error message you posted is because of ZFS, not bcachefs. The ISO enables ZFS by default and it isn’t compatible with some kernels. There may be other issues with bcachefs, but your error was ZFS.

1 Like