Segmentation faults when building with flake on Raspberry Pi 3B+

Help
1

A couple weeks ago, I successfully installed NixOS on a Raspberry Pi 3B+. I installed it using a simple configuration.nix.

Last week I then tried to integrate its config into my flake config as the first NixOS host not based on x86_64-linux and ran into a problem while rebuilding, that I simply do not understand.

I’m getting the following error:

error: builder for '/nix/store/mbhydpqm2gass1f6dwyfkfhf9ckkrxpb-options-db.xml.drv' failed with exit code 139;
       last 1 log lines:
       > /build/.attr-0l2nkwhif96f51f4amnlf414lhl4rv9vh8iffyp431v6s28gsr90: line 19:     8 Segmentation fault      (core dumped) xsltproc --stringparam elementId 'nixos-options' --stringparam optionIdPrefix 'nixos-opt' --nonet -o $out/nmd-result/nixos-options.xml /nix/store/41vh9fr1znwk5vjjprk046s6g4f3jc2m-options-to-docbook.xsl options.xml
       For full logs, run 'nix log /nix/store/mbhydpqm2gass1f6dwyfkfhf9ckkrxpb-options-db.xml.drv'.
error: 1 dependencies of derivation '/nix/store/7p6zrsgch5f70qf0hmf3kcgjgghq8cbc-nmd-documents.drv' failed to build
error: 1 dependencies of derivation '/nix/store/2fks7zkvjlgc6wikhgqs0fwf4fmhj42h-manual-combined.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wc0v40fh0ycmhnn1jq6yh80r5f1jj8z5-man-pages.drv' failed to build
error: 1 dependencies of derivation '/nix/store/bczp3skc6xsrf2fai2i240cjz1khwg44-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/n743cfrs944s31lpza1pmvjsh99rn97b-home-manager-generation.drv' failed to build
error: 1 dependencies of derivation '/nix/store/hbsbw3afw3gmv7z40s0gn87q9jz2wz1c-user-environment.drv' failed to build
error: 1 dependencies of derivation '/nix/store/wjc0b73crhhry7xh2bs81jv3y85fpxz5-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/k43pbjy1w6pik6xpnf1zv3pxymrnjfhk-nixos-system-raspberry-23.05.20230330.e3652e0.drv' failed to build

Any pointers as to what that could be caused by? Could this be related to the documentation.nixos.options.allowDocBook option? It looks like it could be due home-manager, but I use it in the basic configuration.nix as well and that builds fine.

https://search.nixos.org/options?channel=unstable&show=documentation.nixos.options.allowDocBook&from=0&size=50&sort=relevance&type=options&query=docbook

I’m trying to build with sudo nixos-rebuild build --flake .#raspberry --impure --show-trace but I do not get a more verbose output than this.

2

Not sure exactly how you try to integrate the Raspberry’s configuration.nix into the flake config, but perhaps you can just disable the generation of the docBook, because it’s likely you don’t need it on the Raspberry anyway.

1 Like
3

Thanks for your response! I’ve disabled the NixOS documentation on this host entirely anyways, since I wanted to save some space on the SD card.
But even though nixos.documentation.enable is set to false, the segfault happens.

Do you know whether there is some home-manager documentation with DocBook dependency that can be disabled, because I haven’t found any such option?

4

Have you set boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; on the build host?

See binfmt.

Or are you building the flake on the Pi?

5

Well…seems like I’ve found the culprit…
According to this issue, disabling manual generation with manual.manpages.enable = false; seems to do the trick.

I’ll give it a try soon and report back.

6

I’m trying to build the flake on the Pi and am not using any emulation.
I know there are some issues, but I hope to be able to build my simple configuration directly. I might have to look into this topic later on, once I add more ARM-based hosts.

7

What is the output of nix log /nix/store/mbhydpqm2gass1f6dwyfkfhf9ckkrxpb-options-db.xml.drv? Is the xsltproc that is being used there working?

8

Not much insight gained from the log:

/build/.attr-0l2nkwhif96f51f4amnlf414lhl4rv9vh8iffyp431v6s28gsr90: line 19:     8 Segmentation fault      (core dumped) xsltproc --stringparam elementId 'nix-darwin-options' --stringparam optionIdPrefix 'nix-darwin-opt' --nonet -o $out/nmd-result/nix-darwin-options.xml /nix/store/41vh9fr1znwk5vjjprk046s6g4f3jc2m-options-to-docbook.xsl options.xml

How can I find out which version of xsltproc is being used in the build process?

If I install libxslt 1.1.37 from unstable, xsltproc seems to work fine.

Output of xsltproc --version:

xsltproc --version
Using libxml 21003, libxslt 10137 and libexslt 820
xsltproc was compiled against libxml 21003, libxslt 10137 and libexslt 820
libxslt 10137 was compiled against libxml 21003
libexslt 820 was compiled against libxml 21003
9

With manual.manpages.enable = false; I’m still running into similar issues.

error: builder for '/nix/store/22hvnl5npdrv5hsc12aj9sfz7p0vprn6-dbus-1.drv' failed with exit code 139;
       last 1 log lines:
       > /build/.attr-0l2nkwhif96f51f4amnlf414lhl4rv9vh8iffyp431v6s28gsr90: line 8:     9 Segmentation fault      (core dumped) xsltproc --nonet --stringparam serviceDirectories "$serviceDirectories" --stringparam suidHelper "$suidHelper" --stringparam apparmor "$apparmor" /nix/store/n536iaha2b8kzm7dcjiy8b4h8aijbbw6-make-system-conf.xsl /nix/store/wjzj0pyabb4nqmh72xlzdxin9p9w7h0x-dbus-1.14.4/share/dbus-1/system.conf > $out/system.conf
       For full logs, run 'nix log /nix/store/22hvnl5npdrv5hsc12aj9sfz7p0vprn6-dbus-1.drv'.
error: 1 dependencies of derivation '/nix/store/mhqrjkq09a66ycwlhfdl3m085vrkxp7p-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/y9n738gjrz3z1bylln3jyrkvk1sn6zxb-nixos-system-raspberry-23.05.20230411.3fb8eed.drv' failed to build

Again nix log simply shows this one line.

@Sandro Any ideas?

10

You could open a nix repl on the same version as your nixos flake input/channel and then run :l . and libxslt.bin and should get you the path of the drv of that libxslt. You can build it with :b libxslt.bin. If you start nix repl with -L you get also a log. Or maybe you can pot it in the logs of downloaded files. Also a nix-tree on the failed to build drv should reveal its dependency. Or you could grab it out of the drv file directly that failed to build.

1 Like
11

Thanks, I tried your suggestions, but the derivation this happened with seems to be sort of random.
I managed to find this issue and have now reopened it since a bunch of people are running into this segfault issue with libxslt on different Pis.

12

@totoroot, could you try with your OS running from USB-based storage? Or at least put swap on USB and maybe the nix store. sdCards are slow. (I’ve seen similar misbehavior on a Core i7 running from an sdCard.)

I imagine a segfault could occur if the kernel gets an error when paging-in from storage or if the page read takes too long. Shared libraries are treated as virtual memory (read-only swap).

13

@ericgundrum I’m pretty sure that would not make a difference for the Raspberry Pi 3B and 3B+.
Both just have 4xUSB 2.0 ports, so you are limited with devices attached via USB as well. I’m using pretty good SD cards from Samsung with relatively good random reads/writes.

To make sure, I did some testing. Here are the results:

SD card

NixOS booted

sudo dd if=/dev/random of=test.bin bs=10M count=100
1048576000 bytes (1.0 GB, 1000 MiB) copied, 44.4081 s, 23.6 MB/s

sudo dd if=/dev/mmcblk0 of=/dev/null bs=10M count=100 iflag=direct
1048576000 bytes (1.0 GB, 1000 MiB) copied, 44.832 s, 23.4 MB/s

sudo hdparm -tT --direct /dev/mmcblk0
/dev/mmcblk0:
Timing O_DIRECT cached reads: 44 MB in 2.02 seconds = 21.78 MB/sec
Timing O_DIRECT disk reads: 66 MB in 3.05 seconds = 21.65 MB/sec

USB 3.1 drive on USB 2.0 port

Formatted as ext4 and mounted manually

sudo dd if=/dev/random of=test.bin bs=10M count=100
1048576000 bytes (1.0 GB, 1000 MiB) copied, 47.3308 s, 22.2 MB/s

sudo dd if=/dev/sda of=/dev/null bs=10M count=100 iflag=direct
1048576000 bytes (1.0 GB, 1000 MiB) copied, 41.7776 s, 25.1 MB/s

sudo hdparm -tT --direct /dev/sda
/dev/sda:
Timing O_DIRECT cached reads: 30 MB in 2.00 seconds = 14.98 MB/sec
Timing O_DIRECT disk reads: 68 MB in 3.05 seconds = 22.28 MB/sec

HDD with USB 3.0 connector on USB 2.0 port

Formatted as ext4 and mounted manually

sudo dd if=/dev/random of=test.bin bs=10M count=100
1048576000 bytes (1.0 GB, 1000 MiB) copied, 32.7049 s, 32.1 MB/s

sudo dd if=/dev/sdb of=/dev/null bs=10M count=100 iflag=direct
1048576000 bytes (1.0 GB, 1000 MiB) copied, 37.174 s, 28.2 MB/s

sudo hdparm -tT --direct /dev/sdb
/dev/sdb:
Timing O_DIRECT cached reads: 56 MB in 2.04 seconds = 27.51 MB/sec
Timing O_DIRECT disk reads: 76 MB in 3.14 seconds = 24.19 MB/sec

I don’t want to install NixOS to USB storage on the Pi, but I can test putting swap on these two devices.

Update: I enabled swap on each of the USB devices and the result was as expected negative. The builds both still segfaulted with the described error.

14

I reinstalled NixOS on the Raspberry Pi 3B+ using the 22.11 aarch64 release with the latest kernel and this time I was able to build my flake-based configuration without issues.

Another user mentioned on the corresponding GitHub issue that his issue resolved itself with a recent nixpkgs update as well, although they too couldn’t pinpoint the commit that lead to or resolved the issue:

https://github.com/NixOS/nixpkgs/issues/167181#issuecomment-1513803694