Rebuild nixos with musl

Was talking about it in this issue Musl as default instead of glibc · Issue #90147 · NixOS/nixpkgs · GitHub

wanted to move the discussion here, as not to clutter the issue with unrelated stuff

1 Like
error: a 'x86_64-unknown-linux-musl' with features {} is required to build '/nix/store/xpqpdy7npqr55rq4k187avph6l3970wm-bootstrap-tools.drv', but I am a 'x86_64-linux' with features {}

that’s the issue i’m facing now… why and how can i solve this?

i couldn’t do it though

i followed Yoshis intructions and arrived here

was thinking of combining it with this architecture for my cpu, but one step at a time, still can’t use musl to compile my stuff…

FWIW I also tried my own suggestion, and I didn’t get such an error message. It evaluated pretty quickly and is still building 3 hours later.

welcome back to discourse!

really? i’ll look into it tomorrow morning, thank you very much!

okay, can’t sleep, would love to build this while I do so… something different now:

❯ sudo nixos-rebuild switch --upgrade-all --flake ./#$(hostname) --option eval-cache false
warning: Git tree '/home/thiago/.nixos' is dirty
error:
       … while calling the 'seq' builtin

         at /nix/store/h5hhc58qp2rcv0f3d9di6569pxrpsi1j-source/lib/modules.nix:322:18:

          321|         options = checked options;
          322|         config = checked (removeAttrs config [ "_module" ]);
             |                  ^
          323|         _module = checked (config._module);

       … while evaluating a branch condition

         at /nix/store/h5hhc58qp2rcv0f3d9di6569pxrpsi1j-source/lib/modules.nix:261:9:

          260|       checkUnmatched =
          261|         if config._module.check && config._module.freeformType == null && merged.unmatchedDefns != [] then
             |         ^
          262|           let

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: Package ‘musl-1.2.3’ in /nix/store/h5hhc58qp2rcv0f3d9di6569pxrpsi1j-source/pkgs/os-specific/linux/musl/default.nix:148 is not available on the requested hostPlatform:
         hostPlatform.config = "x86_64-unknown-linux-musl"
         package.meta.platforms = [
           "aarch64-linux"
           "armv5tel-linux"
           "armv6l-linux"
           "armv7a-linux"
           "armv7l-linux"
           "i686-linux"
           "x86_64-linux"
           "m68k-linux"
           "microblaze-linux"
           "microblazeel-linux"
           "mips-linux"
           "mips64-linux"
           "mipsel-linux"
           "mips64el-linux"
           "powerpc64-linux"
           "powerpc64le-linux"
           "riscv64-linux"
           "s390x-linux"
         ]
         package.meta.badPlatforms = [ ]
       , refusing to evaluate.

       a) To temporarily allow packages that are unsupported for this system, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1

          Note: When using `nix shell`, `nix build`, `nix develop`, etc with a flake,
                then pass `--impure` in order to allow use of environment variables.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowUnsupportedSystem = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowUnsupportedSystem = true; }
       to ~/.config/nixpkgs/config.nix.

If you just want to build nixos with musl, the correct thing to use would be nixpkgs.hostPlatform = lib.systems.examples.musl64; in your nixos config. However, for a whole nixos system, you’ll find this doesn’t even evaluate right now. This really isn’t (and has never been) a supported use case.

meaning there is no way for me to do so?

Meaning you’re going to have to dig into nixpkgs and fix all the evaluation and build errors that pop up when you try.

that sounds like a hassle allright…

I wouldn’t mind building packages because of the absence of cached binaries, but to fix evaluations and build errors as i go…

i feel frustrated right now…

My build finally finished! By failing. My error reads “unsupported platform for the pure Linux stdenv”.

edit: If you do want to press on and fix packages one at a time, perhaps it would be easier to have an overlay final: prev: { inherit (prev.pkgsMusl) /*...*/; }. That’s where my experience with musl ends anyway. Good luck to you.

That’d do cross-compilation which is another can of worms. It ought to somehow be possible to effectively set pkgs = pkgsMusl because that package set mostly evals and even often compiles and works just fine.

IIRC @qyliss ran (or even daily-drove?) NixOS built with Musl at some point, perhaps she can weigh in.

No it wouldn’t. That only happens if you also set nixpkgs.buildPlatform. If you only set hostPlatform, you get a native package set. Try it.

nix-build "<nixpkgs/nixos>" \
  --arg configuration \
  "{ nixpkgs.hostPlatform = (import <nixpkgs/lib>).systems.examples.musl64; }" \
  -A pkgs.hello

/nix/store/fz3xrpgwvrdrra55csl3zp013p2r1qyf-hello-2.12.1

nix-build "<nixpkgs/nixos>" \
  --arg configuration \
  "{ nixpkgs = with (import <nixpkgs/lib>).systems.examples; { buildPlatform = gnu64; hostPlatform = musl64; }; }" \
  -A pkgs.hello

/nix/store/yx5hcc63x2zy62qp3rlhkk7ggn90g2nn-hello-x86_64-unknown-linux-musl-2.12.1

Notice only the one specifying buildPlatform has the x86_64-unknown-linux-musl cross compilation suffix.

Some of it does. Not enough for a full NixOS. pkgsMusl is the same package set you get from setting nixpkgs.hostPlatform

nix-build "<nixpkgs/nixos>" \
  --arg configuration \
  "{ nixpkgs.hostPlatform = (import <nixpkgs/lib>).systems.examples.musl64; }" \
  -A pkgs.hello

/nix/store/fz3xrpgwvrdrra55csl3zp013p2r1qyf-hello-2.12.1

nix-build "<nixpkgs>" -A pkgsMusl.hello

/nix/store/fz3xrpgwvrdrra55csl3zp013p2r1qyf-hello-2.12.1

Too many packages are not tested with musl and either fail to evaluate or build

2 Likes

I’ll wait for alyssa to respond and see her take on this, and then see what i’d do from that

Well if one’s content with importing a bunch of nixpkgs instances, can’t you literally do that?

nixosSystem { modules = [ ... ({ config, ... }: { _module.args.pkgs = (import <nixpkgs> { /* inherit (config.nixpkgs) config overlays;*/ }).pkgsMusl; }) ]; }

which is, again, literally the same as just setting nixpkgs.hostPlatform, except it precludes you from setting nixpkgs.config values in modules.

I have one laptop running nixos with musl-libc, using the following config:

  nixpkgs.hostPlatform = {
    system = "x86_64-linux";
    config = "x86_64-unknown-linux-musl";
  };

And there is a nixos module with a bunch of fixes I needed to apply at CyberChaosCreatures / musl-nixos · GitLab

However, I haven’t updated it in a while, and the last time I tried building the native (as in, using a toolchain that is also running with musl, not cross-compiled from glibc) system it failed building the kernel.

4 Likes

yureka suggestion worked!
had a build fail on clang after an hour and fifty, but retried and i’m past clang

didn’t need no fixes yet, but will keep an eye out for it.

will just wait the build, give it a reboot and mark the solution

1 Like

Okay, problem building go… Doesn’t seem to relate to musl to me…

@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
Running phase: updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "configurePhase" }
Running phase: configurePhase
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
Running phase: buildPhase
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) 0
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 30
file size                   (blocks, -f) unlimited
pending signals                     (-i) 125764
max locked memory           (kbytes, -l) 8192
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1024
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 125764
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited
/build/go/src /build/go
Building Go cmd/dist using /nix/store/zgn3hsnp6zb8yl425b6dpw60amqp67b2-gccgo-wrapper-12.3.0. (go1.18 gccgo (GCC) 12.3.0 linux/amd64)
found packages main (build.go) and building_Go_requires_Go_1_20_6_or_later (notgo120.go) in /build/go/src/cmd/dist

i’ll check the nix cache and search for problems

i’ll check the nix cache

what do you mean?


Ah yes, the go bootstrap is different on musl because go.dev does not provide binaries for musl systems, so we bootstrap via gccgo, which gets outdated from time to time (and apparently that just happened)

1 Like