Help me understand two load-bearing `assert` statements

Development Nixpkgs Architecture
1

In all-packages.nix, there are as of this morning only two two assert statements, and they both appear to be “load-bearing” in the sense that if the asserts are removed, packages which used to compile no longer do.

One is for gccWithoutTargetLibc. 
# The GCC used to build libc for the target platform. Normal gccs will be
# built with, and use, that cross-compiled libc.
gccWithoutTargetLibc = assert stdenv.targetPlatform != stdenv.hostPlatform; let
    libcCross1 = binutilsNoLibc.libc;
    in wrapCCWith {
      cc = gccFun {
        # copy-pasted
        inherit noSysDirs;
        majorMinorVersion = toString default-gcc-version;

        reproducibleBuild = true;
        profiledCompiler = false;

        isl = if !stdenv.isDarwin then isl_0_20 else null;

        withoutTargetLibc = true;
        langCC = false;
        libcCross = libcCross1;
        targetPackages.stdenv.cc.bintools = binutilsNoLibc;
        enableShared =
          stdenv.targetPlatform.hasSharedLibraries

          # temporarily disabled due to breakage;
          # see https://github.com/NixOS/nixpkgs/pull/243249
          && !stdenv.targetPlatform.isWindows
          && !(stdenv.targetPlatform.useLLVM or false)
        ;
      };
      bintools = binutilsNoLibc;
      libc = libcCross1;
      extraPackages = [];
};
The other is for libcCross. 
libcCross = assert stdenv.targetPlatform != stdenv.buildPlatform; libcCrossChooser stdenv.targetPlatform.libc;

If either of these asserts are removed, wine (for instance) fails to build.

I’m trying to understand where, why, and how these asserts are “load-bearing”.

They’re pretty old. @Ericson2314 added it for gcc in bf17d6dacf24, which has a wonderfully long commit message.

My best guess at why it's load-bearing is this line of code in splice.nix. 
            # Get the set of outputs of a derivation. If one derivation fails to
            # evaluate we don't want to diverge the entire splice, so we fall back
            # on {}
            tryGetOutputs = value0:
              let
                inherit (builtins.tryEval value0) success value;
              in
              getOutputs (lib.optionalAttrs success value);

That would tie evaluation success or failure to a difference in which packages are chosen.

So, several questions:

  1. Is this analysis above right?
  2. Is there another way of doing this that doesn’t have an assert in all-packages.nix? What would an equivalent refactor look like?
  3. Is there a direction that this code should go in, other than this structure?

Your thoughts are wanted, especially @amjoseph and @emily and @Ericson2314 but all are desired.

6 Likes
2

From memory: Splicing has a tryEval. The assets I think are caught by that, which avoids infinite recursion.

Clean up cross bootstrapping, take 2 by Ericson2314 · Pull Request #321525 · NixOS/nixpkgs · GitHub I (or you? :)) should rebase soon which is a small step in the direction of making this stuff less confusing.

5 Likes
3

Thanks for your insight John (@Ericson2314)!

#321525 looks great, though I haven’t put it to the machine to see what it says yet. Is there anything that makes it unmergable today, either through incompleteness or through lack of function?

And yeah, continuing to caretake and evolve the stdenv remains one of my callings where I think I can contribute in nixpkgs. So I’d love to help with this big, old, and worthwhile mission.

3 Likes
4

No just some very trivial conflicts, which probably were made by me too.

Glad to hear it! :slight_smile:

5

Sorry for the slow reply.

At 0e0665d1e0a05ade073ff23e54b6e389c928af20 I was able to remove the first assert with no change to the instantiated derivation.

Regarding libcCross: the assertion here tickles (via wine, which is always the biggest headache for our cross-compilation machinery) the tryEval that @Ericson2314 mentioned. In this case, the gcc.libgcc output is throwing due to checking if libcCross==null. The same effect is had by simply setting libcCross to null in that situation, and this way we don’t have to lean on tryEval.

Here’s a commit that does so:

https://git.sr.ht/~amjoseph/nixpkgs/commit/fa8b6b8e2f4939c7c433ec3087cbc677e08fecca

I have not tested this too extensively, though – I’m running nix build tests.cross.sanity on it to make sure nothing breaks, so consider it a draft until that happens. built

2 Likes
6

Built everything except pkgs.pkgsCross.aarch64-multiplatform.qutebrowser-qt5, which seems to have been broken by something else already sigh.

So, if you want the assert gone, you can submit that patch and blame me in the unlikely event it breaks anything.

Glad to hear this. The libcCross spaghetti-pile was sort of next on the list of stuff to clean up when I moved on to other things. It is sorely in need of cleaning-up.

The native bootstrap used to be a mess like this (a bunch of mutually-recursive packages shoved into the top level) until @Ericson2314 factored them out into the separate stages that we currently have in pkgs/stdenv/linux/default.nix (which is not really as linuxgcc-specific as its name would have you believe).

The ultimate goal was to have that nice clean multistage bootstrap handle not just the native case, but the cross case as well, as a single codepath. That’s pretty ambitious though. Just separating the cross bootstrap into isolated stages would probably be a more feasible first step. Then we could try slowly merging the staged-cross-bootstrap with the native-cross-bootstrap.

Anyways basically anything will be an improvement on the current state of libcCross. It’s hideously ugly. It would be hard to make it any worse :smile:

4 Likes
7

Thank you for this work and analysis! I’ve opened libcCross: set to `null` rather than using `assert` in nonsense scenarios by philiptaron · Pull Request #340331 · NixOS/nixpkgs · GitHub to integrate this commit into nixpkgs.

1 Like