Building nixpkgs from scratch on non-nixos

Help
1

Hi. I’m experimenting with nix on an arch linux installation. (Nix version 2.5.1.) If I try to use build-use-substitutes = false in order to build all packages from source then I’m unable to build almost any package, with errors usually starting at either autoconf or c-ares. For example

> nix shell nixpkgs#autoconf
error: builder for '/nix/store/grfkz6z6fa9k4dg2gg7jxzn97307q4yv-autoconf-2.71.drv' failed with exit code 126;
       last 10 log lines:
       > unpacking sources
       > unpacking source archive /nix/store/g3pzpdicszf700d5xffyk939jzxa4r2k-autoconf-2.71.tar.xz
       > source root is autoconf-2.71
       > setting SOURCE_DATE_EPOCH to timestamp 1611869099 of file autoconf-2.71/.tarball-version
       > patching sources
       > applying patch /nix/store/4hcdpxjmr4nh625ry840g70xp00vdf5a-2.71-fix-race.patch
       > patching file lib/Autom4te/FileUtils.pm
       > configuring
       > configure flags: --prefix=/nix/store/khilbv2w0ixj55clcpsjzpaq0l1bhh2j-autoconf-2.71
       > /nix/store/lwcfw5vyjlzrs35k1hanv21j1q2s5c5w-stdenv-linux/setup: ./configure: /bin/sh: bad interpreter: No such file or directory
       For full logs, run 'nix log /nix/store/grfkz6z6fa9k4dg2gg7jxzn97307q4yv-autoconf-2.71.drv'.

I found 'Re: [Nix-dev] texinfo4 seems to have a /bin/sh dependency' - MARC

It seems there are some packages that are impure and require a specific environment setup to build (build-chroot-dirs/sandbox-paths), and that nixos will perform this setup automatically but non-nixos won’t. Is this understanding correct? Is it expected that some packages won’t build on non-nixos?

I tried setting the following config option to no success

sandbox-paths = /bin/sh:/nix/store/iqprjr5k5385bhf1dzj07zwd5p43py1n-bash-5.1-p12/bin/bash /nix/store/iqprjr5k5385bhf1dzj07zwd5p43py1n-bash-5.1-p12 /nix/store/lx7kf7spirg48miqhaj1in8ik75dr6ls-linux-headers-5.15.5 /nix/store/wl60dr9p15rwf53gxz61ijgisc1zdjc7-glibc-2.33-59

I tried strace but that didn’t show anything useful. Presumably because the actual building is initiated by the daemon on a worker?

I tried using nix develop -i nixpkgs#autoconf to debug the issue, in that environment I can successfully run unpackPhase and configurePhase. So the issue affects only nix build not nix develop.

I found Why is there no way to run `nix-shell` in a chroot and without the user's .bashrc? · Issue #903 · NixOS/nix · GitHub - seems to be not possible to get an environment identical to the one nix build uses. What would be the recommended way of debugging this issue?

Only other nix.conf settings I have are

build-users-group = nixbld
experimental-features = nix-command flakes
2

On NixOS i see following:

sandbox-paths = /bin/sh=/nix/store/5p4jga61v4mg9pdrglj2hj76y5rbz02k-busybox-static-x86_64-unknown-linux-musl-1.32.1/bin/busybox

Note the = instead of :.
Maybe that helps?

3

Thanks! Installing busybox-sandbox-shell and then

sandbox-paths = /bin/sh=/nix/store/2d0c1src48rcyc60hryp6pxgbynq9pkg-busybox-static-x86_64-unknown-linux-musl-1.34.1/bin/sh

did it!

On an unrelated question, is there a way of reversing nix hashes? (Obviously not possible in general, but for searching through known derivations in for example nixpkgs.) Say I wanted to install your 5p4jga61v4mg9pdrglj2hj76y5rbz02k-busybox-static-x86_64-unknown-linux-musl-1.32.1, is there a command that would do that for me? Or would I have to ask you for the derivation that produced that?

4

No, that path is known only after instantiation.

And hashing is a one-way function, so you can’t “unhash” things.

5

To clarify for my own understanding, the hash is only based on the input to the derivation? Nix package manager - NixOS Wiki

In that case, I assume it would be possible to search through the nixpkgs repo to find the exact derivation that lead to that hash, assuming it hasn’t been locally patched or modified. But it sounds like there isn’t any such search script.

6

Correct, for non-FODs (FODs have the sha256’s)

$ nix show-derivation $(nix-instantiate -A autoconf)
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
{
  "/nix/store/i9pv300qmpbwr9nv17mp4p64j4lxffx2-autoconf-2.71.drv": {
    "outputs": {
      "out": {
        "path": "/nix/store/xgzqli2s9kp8p2q42wfbz8qjvim3c9bm-autoconf-2.71"
      }
    },
    "inputSrcs": [
      "/nix/store/4hcdpxjmr4nh625ry840g70xp00vdf5a-2.71-fix-race.patch",
      "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
    ],
    "inputDrvs": {
      "/nix/store/2pw9yk95mvhw4y5d7pz1cjklvn7q6j52-perl-5.34.0.drv": [
        "out"
      ],
      "/nix/store/9j0a343p5x81d6bp3n2drw71hn2zf117-gnum4-1.4.19.drv": [
        "out"
      ],
      "/nix/store/cdbn7qd0asc57406q3vrb3pssljwc3v9-autoconf-2.71.tar.xz.drv": [
        "out"
      ],
      "/nix/store/ig7wbbgy94drqx3vgza6lq26ckkhlmwx-stdenv-linux.drv": [
        "out"
      ],
      "/nix/store/r8fpf45j4g634yvdb6sbnp4n62vq6p7x-bash-5.1-p12.drv": [
        "out"
      ]
    },
    "system": "x86_64-linux",
    "builder": "/nix/store/2kh3c4v2vf6d6xg6c9n8zvfpwf3zzwca-bash-5.1-p12/bin/bash",
    "args": [
      "-e",
      "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"
    ],
    "env": {
      "buildInputs": "/nix/store/fg4vzkv4a7gik8q8c0f7vr7vwv9ia2hl-gnum4-1.4.19",
      "builder": "/nix/store/2kh3c4v2vf6d6xg6c9n8zvfpwf3zzwca-bash-5.1-p12/bin/bash",
      "configureFlags": "",
      "depsBuildBuild": "",
      "depsBuildBuildPropagated": "",
      "depsBuildTarget": "",
      "depsBuildTargetPropagated": "",
      "depsHostHost": "",
      "depsHostHostPropagated": "",
      "depsTargetTarget": "",
      "depsTargetTargetPropagated": "",
      "doCheck": "1",
      "doInstallCheck": "",
      "dontPatchShebangs": "1",
      "enableParallelBuilding": "1",
      "enableParallelChecking": "1",
      "name": "autoconf-2.71",
      "nativeBuildInputs": "/nix/store/fg4vzkv4a7gik8q8c0f7vr7vwv9ia2hl-gnum4-1.4.19 /nix/store/d6j74hvlgqnv6c4ymallmdp5l5j8kccx-perl-5.34.0",
      "out": "/nix/store/xgzqli2s9kp8p2q42wfbz8qjvim3c9bm-autoconf-2.71",
      "outputs": "out",
      "patches": "/nix/store/4hcdpxjmr4nh625ry840g70xp00vdf5a-2.71-fix-race.patch",
      "pname": "autoconf",
      "preCheck": "export TESTSUITEFLAGS=\"-j$NIX_BUILD_CORES\"\n",
      "propagatedBuildInputs": "",
      "propagatedNativeBuildInputs": "",
      "src": "/nix/store/g3pzpdicszf700d5xffyk939jzxa4r2k-autoconf-2.71.tar.xz",
      "stdenv": "/nix/store/zi5qw4dab38kfi33svih50fl3kfkry6d-stdenv-linux",
      "strictDeps": "",
      "system": "x86_64-linux",
      "version": "2.71"
    }
  }
}

Assuming that only nixpkgs was taken into account, and you knew the checkout of nixpkgs, then yes.

It’s the inverse of how you want to think about nix usage in general, you nixpkgs to be your starting point, not a derivation. A derivation can be thought of as nix’s .o files, it’s a compiler/toolchain/ecosystem detail, not a user detail.