How to actually compile a git repository with make using flakes?

I need to compile GitHub - Redecorating/apple_set_os-loader: Tiny EFI program for unlocking the Intel IGD on the MacBook for Linux and Windows and replace the boot efi, I want to do it via flakes. I already installed pkgs.gnu-efi pkgs.gcc pkgs.gnumake as environment.systemPackages using nixos-rebuild switch --no-flake, so now I would make a derivation for the repo and run nixos-rebuild switch to get it going, but I get the error:

error: builder for '/nix/store/p4nisd0qb95nq4rcx3r2zgvxnkwsmmpc-apple_set_os-loader.drv' failed with exit code 2;
       last 10 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/092rna0fsssk6r56a5icnlzvqi0p2v08-source
       > source root is source
       > patching sources
       > configuring
       > no configure script, doing nothing
       > building
       > cc -I/usr/include/efi -I/usr/include/efi/x86_64 -DGNU_EFI_USE_MS_ABI -fPIC -fshort-wchar -ffreestanding -fno-stack-protector -maccumulate-outgoing-args -Wall -Dx86_64 -Werror -m64 -mno-red-zone   -c -o lib/int_event.o lib/int_event.c
       > make: cc: No such file or directory
       > make: *** [<builtin>: lib/int_event.o] Error 127
       For full logs, run 'nix log /nix/store/p4nisd0qb95nq4rcx3r2zgvxnkwsmmpc-apple_set_os-loader.drv'.
error: 1 dependencies of derivation '/nix/store/aqg62synzfxgydndp69ixrgp128zqy0c-system-path.drv' failed to build
error (ignored): error: cannot unlink '/tmp/nix-build-linux-config-6.4.2.drv-2/linux-6.4.2': Directory not empty
error: 1 dependencies of derivation '/nix/store/1hkqs6g726b09vp3bhwiph2fcz8dkss7-nixos-system-mbp2019-23.05.20230715.af8279f.drv' failed to build

To clarify: cc, gcc, make all those commands work already.

My module looks like this:

{ config, pkgs, lib, ... }@specialArgs: let 
apple_set_os-loader = pkgs.stdenvNoCC.mkDerivation {
    name = "apple_set_os-loader";
    src = pkgs.fetchFromGitHub {
      owner = "Redecorating";
      repo = "apple_set_os-loader";
      rev = "r33.9856dc4";
      sha256 = "hvwqfoF989PfDRrwU0BMi69nFjPeOmSaD6vR6jIRK2Y=";
    };
    builder = ./apple_set_os-loader.builder.sh;
  };
in {
  environment.etc."modprobe.d/apple-gmux.conf".text = ''
  # Enable the iGPU by default if present
  options apple-gmux force_igd=y
  '';
  environment.systemPackages = [
    apple_set_os-loader
  ];
}

The apple_set_os-loader.builder.sh contains:

source $stdenv/setup

buildPhase() {
	make -f Makefile
}

installPhase() {
	dir="$out/boot/EFI/boot"
	if [ ! -f "$dir/bootx64_original.efi" ]; then
		cp "$dir/bootx64.efi" "$dir/bootx64_original.efi"
	fi
	cp bootx64_silent.efi "$dir"
}

genericBuild

For test purposes I tried to compile it right in the store:

[mbx@mbp2019:/nix/store/092rna0fsssk6r56a5icnlzvqi0p2v08-source]$ make
cc -I/usr/include/efi -I/usr/include/efi/x86_64 -DGNU_EFI_USE_MS_ABI -fPIC -fshort-wchar -ffreestanding -fno-stack-protector -maccumulate-outgoing-args -Wall -Dx86_64 -Werror -m64 -mno-red-zone   -c -o lib/int_event.o lib/int_event.c
In file included from lib/int_event.c:1:
lib/../include/int_event.h:3:10: fatal error: efi.h: No such file or directory
    3 | #include <efi.h>
      |          ^~~~~~~
compilation terminated.
make: *** [<builtin>: lib/int_event.o] Error 1

Yes sure, it can’t work, because those dependencies are not linked to the system, since cd: /usr/include/efi: No such file or directory. I think now I am experiencing “the bad” about NixOS I was reading about: The inability to simply compile something as I wish, right? I have read a lot “good” as well, just saying :wink:

My question is how to fix this issue, so my derivation will work, so that the package will be compiled and installed properly.

Thank you.

For starters stdenvNoCC does not include a C compiler, hence the error: make: cc: No such file or directory. Just use stdenv instead. Also you can most likely get rid of the custom builder.sh as well, mkDerivation has it’s own defaults which should invoke make automatically.

The missing header is indicative of a missing dependency, you may need to add a list of package dependencies via the nativeBuildInputs, or buildInputs list attributes.

2 Likes

Thank oh, oh wow, of course “NoCC” means … “no cc” haha. Yes I’m a starter and I had trouble to find information about “stdenv vs stdenvNoCC”. Looking into buildInputs and it helps me to understand what I’m doing wrong.

I will feed my buildInputs into my builder and resolve the paths in there, so I came up with:

{ config, pkgs, lib, ... }@specialArgs:
{
  environment.etc."modprobe.d/apple-gmux.conf".text = ''
  # Enable the iGPU by default if present
  options apple-gmux force_igd=y
  '';
  environment.systemPackages = [
    (pkgs.stdenv.mkDerivation { 
	    name = "apple_set_os-loader";
	    src = pkgs.fetchFromGitHub {
	      owner = "Redecorating";
	      repo = "apple_set_os-loader";
	      rev = "r33.9856dc4";
	      sha256 = "hvwqfoF989PfDRrwU0BMi69nFjPeOmSaD6vR6jIRK2Y=";
	    };
            buildInputs = with pkgs; [ gcc gnu-efi gnumake ]; # I also tried baseInputs and nativeBuildInputs
	    builder = ./apple_set_os-loader.builder.sh;
    })
  ];
}
set -e
source $stdenv/setup # will load default functions and map input packages to PATH

function buildPhase() {
  make
}

function installPhase() {
  bootEfi="/boot/EFI/boot"
  if [ ! -f "$bootEfi/bootx64_original.efi" ]; then
    cp "$bootEfi/bootx64.efi" "$bootEfi/bootx64_original.efi"
  fi
  cp bootx64_silent.efi "$bootEfi" # no link across partitions!
}

genericBuild

I understand buildPhase is redundant here, but I need to understand this. I also learned $out points to the store directory (pwd of the builder), so I hopefully fixed my installer phase now. I set my buildInputs, also tried baseInputs and nativeBuildInputs. It still can’t resolve efi.h. I found examples for these three inputs, but no clear documentation about their difference.

My current understanding is that includes/inputs are not linked outside of the store, but rather linked to PATH by the builder, in the sourced setup, but I still have trouble to understand how dynamically linked binaries work in NixOS, but I will get there. My builder still fails as it cannot resolve efi.h, because gcc is called with -I/usr/include/efi ... so that cannot work (I think).

So long story short: Using stdenv I get the missing efi.h error as excepted from my previous test, but I still cannot resolve efi.h.

Questions:

  • What it the difference between baseInputs, buildInputs and nativeBuildinputs
  • How to approach this problem?
  • What the hell am I doing (anyways)?
  • How can I make it work or are there any Starter samples that do what I need to do?

Any help is appreciated! :slight_smile:

Update:

On my system, I can find efi.h at

  • /nix/store/im2carjkyagn638xyfaz1x3f4kyxj91s-gnu-efi-3.0.15/include/efi/efi.h and
  • /nix/store/nkzvcvi0498mmnyq48xj2yb977pvqq3j-gnu-efi-3.0.15/include/efi/efi.h

In a traditional setup it should reside in /usr/include/efi. /run/current-system/sw/usr isn’t a thing in Nix. I’d like to know how libraries get symlinked, the setup.sh didn’t give me enough answers.

How can I conviniently print my final PATH of the builder to the console when my derivation is in the process? I just put echo $PATH in my builder.sh, but nothing prints. However, looking at the produced drv file reveals some useful information:

Derive([("out","/nix/store/xsrs8km5ks82mj34iylz4kpcvaaprv8x-apple_set_os-loader","","")],
[("/nix/store/4k1rsyxkyf3gz4y9mnci2gvbd8yzfxq2-gnu-efi-3.0.15.drv",["out"]),
("/nix/store/50rgnrdv4amkwba762gv88y7lsnbag3b-gnumake-4.4.1.drv",["out"]),
("/nix/store/an46diyg3nsq59n9pj1wfpzbymazg6pp-gcc-wrapper-12.2.0.drv",["out"]),
("/nix/store/mcjbr0j20q0sj5a31d3acmxgjabb5xs6-bash-5.2-p15.drv",["out"]),
("/nix/store/ngr35ljn2k1lakcnl0q6bshjj3cjh7vy-source.drv",["out"]),
("/nix/store/yg9zwby88lkh3idv76j2kac9d5imp7mg-stdenv-linux.drv",["out"])],
["/nix/store/dwqbfkf07h3jx9y261484sz9cd5qr7gz-apple_set_os-loader.builder.sh"],
"x86_64-linux","/nix/store/8fv91097mbh5049i9rglc73dx6kjg3qk-bash-5.2-p15/bin/bash",
["-e","/nix/store/dwqbfkf07h3jx9y261484sz9cd5qr7gz-apple_set_os-loader.builder.sh"],
[("__structuredAttrs",""),
("buildInputs","/nix/store/lcf37pgp3rgww67v9x2990hbfwx96c1w-gcc-wrapper-12.2.0 /nix/store/im2carjkyagn638xyfaz1x3f4kyxj91s-gnu-efi-3.0.15 /nix/store/54fjibb2hl2dvmvaa7y6258ylp0yy9vc-gnumake-4.4.1"),
("builder","/nix/store/8fv91097mbh5049i9rglc73dx6kjg3qk-bash-5.2-p15/bin/bash"),("cmakeFlags",""),
("configureFlags",""),("depsBuildBuild",""),("depsBuildBuildPropagated",""),("depsBuildTarget",""),
("depsBuildTargetPropagated",""),("depsHostHost",""),("depsHostHostPropagated",""),
("depsTargetTarget",""),("depsTargetTargetPropagated",""),("doCheck",""),("doInstallCheck",""),
("mesonFlags",""),("name","apple_set_os-loader"),("nativeBuildInputs",""),
("out","/nix/store/xsrs8km5ks82mj34iylz4kpcvaaprv8x-apple_set_os-loader"),("outputs","out"),
("patches",""),("propagatedBuildInputs",""),("propagatedNativeBuildInputs",""),
("src","/nix/store/092rna0fsssk6r56a5icnlzvqi0p2v08-source"),
("stdenv","/nix/store/91hwyc6c4r474zs56n9idbqj17dlcnna-stdenv-linux"),("strictDeps",""),
("system","x86_64-linux")])

/nix/store/4k1rsyxkyf3gz4y9mnci2gvbd8yzfxq2-gnu-efi-3.0.15.drv contents point $out to /nix/store/im2carjkyagn638xyfaz1x3f4kyxj91s-gnu-efi-3.0.15, this looks right.

I think efi.h would need to be in /nix/store/im2carjkyagn638xyfaz1x3f4kyxj91s-gnu-efi-3.0.15/usr/include/efi/efi.h, but it’s in /nix/store/im2carjkyagn638xyfaz1x3f4kyxj91s-gnu-efi-3.0.15/include/efi/efi.h, so how should the problem be approached now? Does gnu-efi’s package need to be fixed, so it also can also be found in /usr/include/efi/ or how can I fix this?

Update: I tried to change the make call to make all CFLAGS=-I/include/efi, but that didn’t help either. The issue remains.

Update: I understand that dependencies are not copied to my temporary build, but rather linked into the store. So my previous attempt to link it in the temporary directory was BS. I just run nix-shell -p gcc gnu-efi gnumake --pure and then env to see what environment variables the builder will have.

NIX_CFLAGS_COMPILE= -frandom-seed=xi6gsi0vkj -isystem /nix/store/nkzvcvi0498mmnyq48xj2yb977pvqq3j-gnu-efi-3.0.15/include -isystem /nix/store/g4y5fhlrgif3pbb0m27rcmmhi189iwzk-gnumake-4.4.1/include -isystem /nix/store/nkzvcvi0498mmnyq48xj2yb977pvqq3j-gnu-efi-3.0.15/include -isystem /nix/store/g4y5fhlrgif3pbb0m27rcmmhi189iwzk-gnumake-4.4.1/include
NIX_LDFLAGS=-rpath /nix/store/xi6gsi0vkjrh8hjig0wf44g3iqi1svyq-shell/lib64 -rpath /nix/store/xi6gsi0vkjrh8hjig0wf44g3iqi1svyq-shell/lib  -L/nix/store/nkzvcvi0498mmnyq48xj2yb977pvqq3j-gnu-efi-3.0.15/lib -L/nix/store/nkzvcvi0498mmnyq48xj2yb977pvqq3j-gnu-efi-3.0.15/lib

I see how it tries to come up with the right paths, but the Makefile actually wants the flags to point to the efi subdirectory, the path should be /nix/store/nkzvcvi0498mmnyq48xj2yb977pvqq3j-gnu-efi-3.0.15/include/efi.

Can I tell mkDerivation to link with a suffix on gnu-efi?

It’s probably just that it’s looking for include/efi.h while it’s actually in include/efi/efi.h. Just substituteInPlace (source files here) --replace efi.h efi/efi.h postPatch.

Do yourself a favour and don’t override the stdenv builder. You should almost never do that. If you need to override phases, just set the phase in mkDerivation.

Great. I just wanted to report that I came up with this to circumvent this build issue within my nix file:

makeFlagsArray=(CFLAGS=-I${pkgs.gnu-efi}/include/efi\ -I${pkgs.gnu-efi}/include/efi/x86_64\ -DGNU_EFI_USE_MS_ABI\ -fPIC\ -fshort-wchar\ -ffreestanding\ -fno-stack-protector\ -maccumulate-outgoing-args\ -Wall\ -Dx86_64\ -Werror\ -m64\ -mno-red-zone)

and I hated it, I hated everything of it for obvious reasons. I’d rather want to have those paths available in the environment variables of the builder script and do my thing there at least. Now I want to follow your advice, I don’t want to override the builders, since they do more, so I will use hooks instead.

Applying your solution now, replacing the above with a postPatch that tackles the includes:

postPatch() {
  substituteInPlace include/*.h include/*.c *.c  --replace "<efi" "<efi/efi"
}

Yes, this works, but it also feels botched together. I hate it as well, not as much, but still a little. Isn’t there a more elegant solution to such a problem (when dependencies don’t side at the right location for the includes)? This looks very error prone when trying to migrate verstions.

Lastly, the errors continue:

       > ld -T /usr/lib/elf_x86_64_efi.lds -Bsymbolic -shared -nostdlib -znocombreloc /usr/lib/crt0-efi-x86_64.o -o bootx64.so lib/int_event.o lib/int_graphics.o lib/int_mem.o lib/int_dpath.o lib/int_print.o lib/pci_db.o bootx64.o /nix/store/b7hvml0m3qmqraz1022fwvyyg6fc1vdy-gcc-12.2.0/lib/gcc/x86_64-unknown-linux-gnu/12.2.0/libgcc.a /usr/lib/libgnuefi.a
       > impure path `/usr/lib/elf_x86_64_efi.lds' used in link
       > make: *** [Makefile:19: bootx64.so] Error 1

And I get it. The Makefile is full of this impure garbage: https://github.com/Redecorating/apple_set_os-loader/blob/9856dc4e6d6105759ef788a473577bcaebb90a89/Makefile#L5-L14

And to make this work for Nix, I have to find all those dependencies, provide them as buildInputs, and hope they will be fixed for me, otherwise I have to continue botching? Like botching even with the Makefile?

This is one of my first mkDerivations I need to come up with and sadly it’s one of those that need to be built by hand, as this hybrid graphics efi module is not available in nixpgks yet.

I love the concept of Nix and NixOS in general, but this makes it taste quite bitter, I wish linkage errors could be solved more intuitively. Maybe there are better ways to solve those kind of issues?

please use mkDerivation’s NIX_CFLAGS_COMPILE instead. Trying to achieve all of this at the builder level is just going to be pain when everything is set up to not have to do that.

The “better” solution would be to write a patch (and perhaps submit it upstream). If the source has the wrong path to the header, it needs to change, not the dependency providing the header.

Again, bad sources need patching. That path should not be hard-coded and the hard-coded output path feels like a crime. Patch that shit.

Looking at the makefile, they seem to include a mechanism for you to declare where to find the efi.h: makeFlags = [ "INC=${lib.getDev gnu-efi}/include/efi" ]; obviates the header patching at least. (Seems like they had this issue before. Why they didn’t just correct the include paths? IDK.)

The hard-coded LDFLAGs need to be patched to point at the efi linker script from ${lib.getLib gnu-efi}/lib/elf_x86_64_efi.lds and put the result into $out/lib.

Ideally, ideally you should make the Makefile’s linker script prefix a variable such that you can set it via makeFlags, the out PREFIX configurable and submit the patch upstream.

I’m not quite sure which dependencies you’re looking for? You’ve already found the package’s sole dependency: gnu-efi.
You can cross-check that against their Dockerfile.

“Botching with” the Makefile is the only botching required here it seems. It’s a bad Makefile, so of course it needs to be patched to package it properly.

No. None of what you did to troubleshoot or had to actually do required you do get your hands as dirty as you made them. Now, you’ve probably learned quite a few concepts in packaging by doing that, so I won’t put that approach off as wrong but be informed that you have made this a lot harder than it would otherwise be.

What you actually need here AFAICT is a patch to the bad Makefile and add a makeFlag like I mentioned earlier. That should be it.

Well, you tried to package something that’s broken upstream (wrong include paths + bad and inflexible Makefile). That’s not trivial for your first packaging job. Most packages aren’t like this.
Many packages are and you need to be prepared but this sort of thing comes with experience.

Me too but that’s a general C/C++ tooling problem.

In this case, the solution was quite intuitive. The linker tried to access some FHS path and helpfully errors out with a message because that’s obviously wrong.
Such a thing can only really occur when someone hard-coded said FHS path somewhere which must have happened in the build system definition and indeed the Makefile was the culprit.

3 Likes

Thank you for that great explanation, Atemu.

NIX_CFLAGS_COMPILE is an env variable. I could patch that in a hook, but I can’t access my buildInput paths in my builder.sh reliably, but that is a different story as the solution seem to patch the Makefile as you said and I fully agree that the Makefile is just bad, since it works with absolute paths.

Putting this specific issue aside it shows a fundamental issue for me: The packaging within NixOS requires modifying source codes (compile configurations such as Makefile) of non-nix-related projects just to make them compatible with NixOS, when those projects would compile just fine on traditional Linux systems. The order in chain is not what I’d like it to be. In an ideal world I would see that NixOS can handle those “mistakes”, that seem only to be mistakes in unintended scenarios. The authors probably didn’t have NixOS in mind, and it compiles just fine for them.

My initial line of thought or how I imaged how mkDerivation actually would work (before reading the manual): I know this is irrelevant, but hear me out. I was thinking programs are built in a temporary environment that gets all files linked to the store. So instead of telling make the right paths to the dependencies in the store, I would just tell the derivation where to put links to those dependencies into the temporary build path, so it could resolve those without modifying any files in the package itself. In terms of absolute paths, chroot could maybe could be leveraged, but what do I know, I know nothing, but that’s what I thought Nix would actually somehow do.

Instead, Nix has a generic builder script that is huge, because it tries to cover anything and everything as best as possible. It further abstracts the configuration API of make and thus it is limited to its own capabilities while introducing another level to the compiler toolchain. And worst about that is: As soon as we hit a package that builds, but doesn’t build “the right way”, we are in trouble. I’m not sure if I’m a fan of this approach, but I have to trust that it’s the right decision, because I know nothing and Nix has a very long history with many experts behind it, so what do I know.

Because I know nothing, but I think this is just a great experience to learn. I never really did C++ and only did a little bit of C. I’m familiar with a lot of higher level languages, but I should not be afraid to cross that line, because I have to in the future anyways, so I won’t give up on NixOS, just because I need to get my hands dirty. It’s just not a pleasant way to start and my time is very limited at the moment, which is nobody’s fault.

I was hoping to quickly get the key packages compiled, so I can run NixOS properly on this hardware and I don’t have to constantly boot into macOS for my convenience. I need to focus on other stuff for now, but I will return to this issue in some time.

Good then, so what is the recommend way to tackle this? I would probably checkout the git repo locally, change fetchFromGitHub to a local path and start to modify the Makefile until the derivation passes, right?

Please stop doing this at the builder level. It’s not how that’s done and it’s just going to be painful.

There is a reason we have the mkDerivation abstraction.

Note that patching the build system is extremely common in packaging in general and not specific to Nixpkgs.

Nixpkgs absolutely has additional constraints which build systems must be able to work with but so do other packaging systems. We’re not the only ones disallowing internet access during the build or installing files into hard-coded paths.

The only reason this compiles on other systems is because big assumptions were made that usually hold true on a subset of FHS distros.

It does handle a lot of “mistakes” like that but it can’t handle all of them. If the Makefile is broken, it can’t fix that automatically.

That’s theoretically possible using fhsenvs and I think people have tried before but sanitising the build system is usually less effort and, IMHO, cleaner. Also leaves the option to upstream the patches such that other distros can clean up their packaging too.

Yes and no. If something doesn’t build “the right way”, the package is probably broken in the first place and that’s a good indication that you should look at its build system definition in more detail.

mkDerivation does offer escape hatches such as NIX_CFLAGS_COMPILE and you can generally tweak quite a few things about it.
Even if that all isn’t enough, you can always override a phase to do something entirely different. If the stdenv’s handling of make conflicts with the package somehow, you can always just override the buildPhase with a custom way to call make.

See for example Anki. Upstream has a …troubled history with build systems. Our current derivation is quite complex and we need to override a few phases but the stdenv still supports us where it can:

I share your concerns and they are shared among others in the Nix community too but so far, the stdenv has held up very well.

Sounds like a good plan!

1 Like

Maybe this is too subjective, but I wouldn’t call a package broken if it compiles on the systems it was designed for. I think the goal was never to distribute this package in a way like I try to do. There is definitely room for improvement, though.

fhsenvs is really what I’d want to use, but I acknowledge that making the Makefile portable would be the better move, as other build systems could benefit from it. It’s still nice to know there is that option in case of more complex scenarios, thanks for linking that up.

Well, that’s true, but it’s not so trivial. For instance, how could I access the path to gnu-efi in the builder script? I won’t go down that route, because it’s unnecessary in this case as we concluded, but this is what I tried to do, initially. Initially, I just wanted to override the buildPhase and I came to the conclusion that I only have $buildInputs which contains all paths separated with a space. Not even an array of which I could simple pick the 2nd item from to get the nix store path to gnu-efi. This is somewhat limiting. Again, I agree there are better ways, but I think this option would give more security moving forward.

When you toString a derivation, you get its output path. From then on it’s just a simple case of string interpolation:

buildPhase = ''
  do something with ${gnu-efi}/include/efi
'';

buildInputs is only for the stdenv builder to discover dependencies automatically.

I noticed that, but want the phases to be defined in a builder script (separate sh file), not in the nix file. I prefer this separation to get proper syntax highlighting. I think I can just use a shell hook to export the path from the nix file into my script, so consider this solved. Thanks for all the great input. I will revisit this when I have more time! :slight_smile:

1 Like

“It works” does not mean it’s correct or good. Hacks often work, and then fail the moment someone does something subtly different with it (like try to run it on a non-fhs distro).

These things usually happen because developers focusing primarily on developing the actual software need a build system but just don’t have time/experience to write a good makefile (or the ecosystem awareness to use a better build system/programming language with fewer footguns). This is fine in the short term, we refer to that as “technical debt”, and should be cleaned up in the future by someone who actually knows their build systems.

Upstream would likely be quite happy if you sent them a PR that fixes all these hardcoded paths, that’s likely what they did to get things working, not their desired solution.

Even if they aren’t happy with you sending a patch, that makefile is a mess. Build scripts can be bad software too, the idea that just hardcoding stuff in makefiles is fine drives so much pointless churn in our industry.

E.g. Nix’ design (also bazel, pants, buildstream, …) needs build systems to not do horrible things, so that they can provide all their cool features, so you’re running into this churn right now. It sucks, doesn’t it?

2 Likes

That’s the thing I’d recommend against. Most of the times you need to change something about the phases, you aren’t writing some complex script that requires highlighting, just a few straight-forward commands as far as syntax is concerned.

If you really wanted to have a phase in a separate file for syntax highlighting, you could just put the snippet into i.e. buildPhase.sh and then buildPhase = import ./buildPhase.sh. It’d be a bit of a weird pattern and likely still wouldn’t be accepted into Nixpkgs but at least it wouldn’t break mkDerivation.

2 Likes