Compile Postgresql for 16k page size

Help
1

I’m trying to migrate my homeserver setup to a Raspberry Pi 5. This has only experimental community support. So far, I’m following the recommended community project. It has two versions to build the system. One is basic, only makes it work, the other attempts to make more modifications (nixosSystemFull) by applying a lot of overlays.

The first version works for me. The system builds and operates. But Postgres crashes complaining about the page size. The issue seems to be that it is using jemalloc, which must be compiled for the appropriate page size. nixpkgs sets that to 4k on x86 and 64k on aarch64, but the Pi5 is aarch64 with a page size of 16k.

I suppose this is one of the things that nixosSystemFull would solve, but I’m unable to make that work, as it compiles a lot of stuff locally and quite a few things break in the test phase. I do not have the capacity to debug that.

So, instead, I have attempted (successfully, I believe) to create an overlay for just jemalloc

  nixpkgs.overlays = [
    (self: super: {
      jemalloc = super.jemalloc.overrideAttrs (old: {
        configureFlags = (lib.filter (flag: flag != "--with-lg-page=16") old.configureFlags) ++ [ "--with-lg-page=14" ];
      });
    })
  ];

Checking the store for derivations shows that one with the correct flags has indeed been built (in addition to the leftovers from experimentation). This is not the case for postgresql. Only one derivation there. This is not what I expected. Since it uses jemalloc I’d expect that a rebuild of that should also rebuild postgresql, no?! I’ve briefly looked at the postgresql.nix in nixpkgs but found no mention of “jemalloc” or “alloc” for that matter. So I wonder how that thing is built. Is it compiled from source? Or downloaded somewhere and just patched?

How do I manage to rebuild that with a fitting version of jemalloc for the 16k page size pi 5 kernel?

2

I tried to use a flake, and added an extra module to my system:

...

        mypc = nixos-raspberrypi.lib.nixosSystem {
          specialArgs = inputs;
          modules = [
            ./mypc/override-jemalloc.nix
            ./mypc/raspberry-pi-5.nix
            ./mypc/configuration.nix
            ./mypc/hardware-configuration.nix
          ];
        };

...

then in mypc/override-jemalloc.nix I put:

{ config, pkgs, lib, ... }:
{
  nixpkgs.overlays = [
    (final: prev:
      {
        jemalloc = prev.jemalloc.overrideAttrs (old: {
          # --with-lg-path=(log2 page_size)
          # since our page size is 16384 (2**14), we need 14
          configureFlags = (lib.filter (flag: flag != "--with-lg-page=16") old.configureFlags) ++ [ "--with-lg-page=14" ];
        });
      }
    )
  ];
}

But postgres is still dying with unsupported system page size. jemalloc did rebuild, so I think it must be something vendoring jemalloc. :confused:

3

Ok so my issue is with immich, which preloads vector.so nixpkgs/nixos/modules/services/web-apps/immich.nix at 2421b36eb4cb5324e63bc9772310a0993adae88e · NixOS/nixpkgs · GitHub

which is a rust crate that depends on another rust crate: pgvecto.rs/Cargo.toml at 2b290b34e8ba69104ea2f800fa53328c6ed6c236 · tensorchord/pgvecto.rs · GitHub

that vendors jemalloc and look it has a “fix” for ios.

Not sure how to untangle that in nix though.

The rust package itself accepts the necessary environment variable: JEMALLOC_SYS_WITH_LG_PAGE

( crates.io: Rust Package Registry)

so I just need to override this vector-rs plugin that is deep in postgres’s build machinery.

4

I ended up copy-pastaing postgres build, I’m sure there is a way to do it better but I copied nixpkgs entire pkgs/servers/sql/postgresql/ directory then making my overlay:

{
  config,
  pkgs,
  lib,
  ...
}:
{
  nixpkgs.overlays = [
    (final: prev: {
      inherit (lib) recurseIntoAttrs;

      jemalloc = prev.jemalloc.overrideAttrs (old: {
        configureFlags = (lib.filter (flag: flag != "--with-lg-page=16") old.configureFlags) ++ [
          "--with-lg-page=14"
        ];
      });

      inherit (import ../pkgs/postgresql prev)
        postgresqlVersions
        postgresqlJitVersions
        libpq
        ;

      inherit (final.postgresqlVersions)
        postgresql_13
        postgresql_14
        postgresql_15
        postgresql_16
        postgresql_17
        ;

      inherit (final.postgresqlJitVersions)
        postgresql_13_jit
        postgresql_14_jit
        postgresql_15_jit
        postgresql_16_jit
        postgresql_17_jit
        ;
      postgresql = final.postgresql_17;
      postgresql_jit = final.postgresql_17_jit;
      postgresqlPackages = final.recurseIntoAttrs final.postgresql.pkgs;
      postgresql13Packages = final.recurseIntoAttrs final.postgresql_13.pkgs;
      postgresql14Packages = final.recurseIntoAttrs final.postgresql_14.pkgs;
      postgresql15Packages = final.recurseIntoAttrs final.postgresql_15.pkgs;
      postgresql16Packages = final.recurseIntoAttrs final.postgresql_16.pkgs;
      postgresql17Packages = final.recurseIntoAttrs final.postgresql_17.pkgs;
    })
  ];
}

the only thing I changed in the directory is:

diff --git a/pkgs/postgresql/ext/pgvecto-rs/package.nix b/pkgs/postgresql/ext/pgvecto-rs/package.nix
index b78d948..ca55431 100644
--- a/pkgs/postgresql/ext/pgvecto-rs/package.nix
+++ b/pkgs/postgresql/ext/pgvecto-rs/package.nix
@@ -73,6 +73,8 @@ in

       # Bypass rust nightly features not being available on rust stable
       RUSTC_BOOTSTRAP = 1;
+
+      JEMALLOC_SYS_WITH_LG_PAGE = "14";
     };

     # This crate does not have the "pg_test" feature

probably need to submit a patch to upstream to allow this to be overriden at the top level in postgres package