I have a package with cross compilation, that was not set up by me, and it relies heavily on using binary cache with necessary inputs. The package is very problematic, and often fails to build so I thought that maybe it would be better to have the package in cache as a whole, but turns out, since it has nativeBuildInputs, the build results are different depending on where it was built even if target platform is the same.
Is this how it should work? I initially expected that the result produced should be the same independently of the build platform.
FWIW I’ve tried to remove nativeBuildInputs and now I get the same hash, but package from cache wants to build locally, anyway.
Nix is generally pretty pessimistic about differently built notionally-equivalent tools behaving the same. Just like GCC 14 built with different optimisation flags itslef will be treated as a different compiler producing different outputs (and theoretically the compiler bugs could be different!), cross toolchains running on different platforms are considered different toolchains with different bugs producing different outputs.
1 Like
Yes, this is how it should work. Nix gives a derivation a different name if any component of the build process is different. If you use a different version of the c compiler to build the package, you get different code out, so it gets a different name. Even if you just pass a different flag to the compiler, different name. This is what we mean when we say nix does reproducible builds.
Alas, this also means that you can’t reuse things that “should probably be the same” because they were built targeting the same arch, but building on different arches.
3 Likes
this definitely makes sense, I was just too hopeful. There is another thing that worries me: when I do cross compilation if I use default stdenv, I get derivations that always have system == build system, even if binaries inside are of different architecture, is that because I’m doing something wrong, or is it by design, too?
1 Like
No, you’re not doing anything wrong. The system value for a derivation is the system which can build the derivation. After all, to nix, a derivation is a set of instructions that produces some files. The system the files are then intended to be used on really isn’t vital information to nix. The system needed to run the build instructions is.
3 Likes
By the way, is that package the end-goal or an intermediate dependency? One option is to build it in CI, push to the cache, save the history of succesful builds, and use nix-store -r $latest_path --add-root $target --indirect then use $target/bin/command or whatever.
This is an intermediate dependency, but we’re using flakes to build the end-goal and I am not sure I understand how to translate this nix-store command to setting the dependency from cache. Does that mean, we would be able to override the dependency even if hash is different for local and cache? Is there a doc I can read about it?
An off-topic question is, where does the standard stdenv come from? The package is called in an overlay and without parameters, and has stdenvNoCC parameter that works, but if I use pkgsBuildHost which I though to be the right source of stdenv, it doesn’t work
OK, injecting an intermediate input is probably more complicated, not sure whether it is possible at all.
I would expect that stdenvNoCC is in a sense the pkgsBuildBuild version as it needs to be runnable on build, not on the host. But I might be oversimplifying yet, as there are some extra layers of cross-compilation orchestration logic in stdenv
1 Like