How does mkDerivation decide what build system to use

Help
1

mkDerivation has the ability to automagically take appropriate steps to build projects that use well-known build systems such as cmake and meson.

How does it decide which build system should be used?

I haven’t found the relevant documentation, but observations suggest that the order in which the build systems are listed in nativeBuildInputs is important!

Where can I find out more about this?

3 Likes
2

The nativeBuildInputs you add get their hooks ran, which then register themselves to get executed in the different phases.

This is a very rough and brief description, but essentially what happens.

5 Likes
3

If we take meson as an example meson package has this magic attribute:

pkgs/development/tools/build-managers/meson/default.nix:  setupHook = ./setup-hook.sh;

This causes pkgs/stdenv/generic/setup.sh for meson to install it as nix-support/setup-hook:

    if [ -n "${setupHooks:-}" ]; then
        mkdir -p "${!outputDev}/nix-support"
        local hook
        # have to use ${setupHooks[@]} without quotes because it needs to support setupHooks being a array or a whitespace separated string
        # # values of setupHooks won't have spaces so it won't cause problems
        # shellcheck disable=2068
        for hook in ${setupHooks[@]}; do
            local content
            consumeEntire content < "$hook"
            substituteAllStream content "file '$hook'" >> "${!outputDev}/nix-support/setup-hook"
            unset -v content
        done
        unset -v hook
    fi

And then on packages that include meson via nativeBuildInputs it gets sources in pkgs/stdenv/generic/setup.sh as:

# Add package to the future PATH and run setup hooks
activatePackage() {
...
    if [[ -f "$pkg/nix-support/setup-hook" ]]; then
        source "$pkg/nix-support/setup-hook"
    fi
}

And the rest of magic happens in pkgs/development/tools/build-managers/meson/setup-hook.sh itself:

mesonConfigurePhase() {
    runHook preConfigure
...
    runHook postConfigure
}

if [ -z "${dontUseMesonConfigure-}" -a -z "${configurePhase-}" ]; then
    setOutputFlags=
    configurePhase=mesonConfigurePhase
fi

It overrides default configurePhase.

4 Likes
4

So, in summary, adding a package to (native?)buildInputs will result in hooks for that package being run.

What happens if your nativeBuildInputs contain more than one package which overrides phases in the build process?

I am trying to use meson, but I need cmake too, because one of my dependencies does not provide a pkg-config, so meson has to use cmake to discover that dependency.

I stumbled across the ‘solution’ of swapping the order of meson and cmake in my build inputs to make sure the overall build uses meson rather than cmake, but is this something that can be relied on?

Also, you both write about the hooks being run if a package appears in the native build inputs, but it also seems to work in plain buildInputs. Is that expected?

5

For a cross-case it should be only nativeBuildInputs. For non-cross it’s only nativeBuildInputs if you set strictDeps = true;. Otherwise builtInputs are activated all the same.

I would say relying on the hook order is fragile as propagatedNativeBuildInputs for other packages might bring packages in unexpected ortder.

If the override does not do the right thing for you there are a few options:

  1. disable one of the overrides, like dontUseCmakeConfigure = true; (see pkgs/development/tools/build-managers/cmake/setup-hook.sh for specifics)

  2. write the code for incorrect phases explicitly. Something like:

configurePhase = ''
  mesonConfigurePhase
''

Or even call meson directly (and provide missing flags usually passed by default hook).

2 Likes
6

Exactly, that’s why I put “‘solution’” in scare quotes.

I’m not entirely sure what you mean by ‘cross-case’.

7

Native build is the build, host and target platforms are there same:

nix-repl> stdenv.buildPlatform == stdenv.hostPlatform && stdenv.buildPlatform == stdenv.targetPlatform
true

This is the simple usual case used when you build for local system.

cross-case is when you cross-compile (stdenv.buildPlatform == stdenv.hostPlatform && stdenv.buildPlatform != stdenv.targetPlatform) or cross-build (stdenv.buildPlatform != stdenv.hostPlatform && stdenv.buildPlatform == stdenv.targetPlatform) ther package.

The examples are: pkgsCross.riscv64.stdenv.cc.cc and pkgsCross.riscv64.gcc.cc.

1 Like