Extra packages?

yellowhat · February 16, 2024, 1:06pm

Hi,

I would like to move my homelab to NixOS (no GUI, only kubernetes).

For now I am:

creating a VM image using nix build .#nixosConfigurations.qcow2.config.system.build.qcow2.
running the VM using qemu-system-x86_64

But it is not clear to me how packages dependencies are calculated.

Even if I set:

  documentation.enable = false;
  documentation.dev.enable = false;
  documentation.doc.enable = false;
  documentation.nixos.enable = false;
  documentation.man.enable = false;
  environment.defaultPackages = [];

man pages for some packages are still installed:

$ nix path-info -r /run/current-system | grep man
/nix/store/ffqrphr0j6phc76lj4pwbwir3s0cgij7-jq-1.7.1-man
/nix/store/fybn8gry0ig40ji7xya5fzlirz9dd6nj-nix-2.18.1-man
/nix/store/apc0xxr94rxavsfb0x5jwfchybk5ld65-perl-5.38.2-man
/nix/store/m2arzrbfrscgsli0ahb59aky64jclmqc-nix-2.15.3-man

I do not understand why aws packages are installed:

$ nix path-info -r /run/current-system | grep aws
/nix/store/nvybrkwk0na8l3wk61rp093qjnr47r7q-aws-c-common-0.9.10
/nix/store/5p0dcwwmjs08qgffgq4v1s320w0wi3vj-aws-c-cal-0.6.9
/nix/store/g778l529lh7b0860fl43560s37r1qfb6-aws-c-io-0.13.36
/nix/store/rsaxmnm189mgyndfwbwcxqwkllimlmrk-aws-c-compression-0.2.17
/nix/store/2b0wfwr5rs9z8qqhzcssrsah6ynlbj79-aws-c-http-0.7.14
/nix/store/7vzvav5ycn5bsizfzm8q4d9ddj9c5fwr-aws-c-mqtt-0.9.10
/nix/store/g6szbhjrpfrk1411digir61vs2x37gxl-aws-c-sdkutils-0.1.12
/nix/store/p7fs3lnksl3rlr5kihljbdazqnva4ql7-aws-c-auth-0.7.10
/nix/store/s44lqifkb2pw3bkxyk7zdzdig5d9q7pr-aws-checksums-0.1.17
/nix/store/vja1ifw4k6ld3q4jdg7a3f65r4250ibz-aws-c-s3-0.4.0
/nix/store/zplxavxyszipwgjrxhrdim1iydax7x9g-aws-c-event-stream-0.3.2
/nix/store/qzjfn8rzk3vbnnk47sxa85628l4gfiyg-aws-crt-cpp-0.24.7
/nix/store/szawfx8vmd4fd4x8ryc2lnfyajg7mw3b-aws-sdk-cpp-1.11.207

$ nix why-depends /run/current-system /nix/store/qzjfn8rzk3vbnnk47sxa85628l4gfiyg-aws-crt-cpp-0.24.7
/nix/store/dz3pxh3x6zsi9bqz1zby5523s8iw0qa2-nixos-system-nixos-24.05.20240216.df54f6f
└───/nix/store/qa71j58x51bi3xc5f91dvkb94jidv0sg-system-path
    └───/nix/store/nzkd2mf4792r7aia5fw3f7libpk2rxyj-nix-2.18.1
        └───/nix/store/qzjfn8rzk3vbnnk47sxa85628l4gfiyg-aws-crt-cpp-0.24.7

Any suggestions?

Thanks

Atemu · February 16, 2024, 1:59pm

yellowhat:

$ nix why-depends /run/current-system /nix/store/qzjfn8rzk3vbnnk47sxa85628l4gfiyg-aws-crt-cpp-0.24.7
/nix/store/dz3pxh3x6zsi9bqz1zby5523s8iw0qa2-nixos-system-nixos-24.05.20240216.df54f6f
└───/nix/store/qa71j58x51bi3xc5f91dvkb94jidv0sg-system-path
    └───/nix/store/nzkd2mf4792r7aia5fw3f7libpk2rxyj-nix-2.18.1
        └───/nix/store/qzjfn8rzk3vbnnk47sxa85628l4gfiyg-aws-crt-cpp-0.24.7

As you can see here, Nix (as in: the package manager’s package) depends on the AWS packages.

Runtime dependencies aren’t “calculated”, they’re inferred. A built output path is scanned for Nix store path references (i.e. in executables’ rpaths) and any that are found become part of the runtime closure.

If some derivation of your system closure depends on explicitly depends on the man output of some package, your system will transitively depend on that man output.

The NixOS options you set here control mechanisms which explicitly try to add packages’ man pages to the global environment. They produce new dependency relations but disabling that won’t remove existing ones.

Use the same why-depends to find out what it is that depends on the man outputs of those packages and then use derivation show on the direct dependant to figure out why exactly.

yellowhat · February 16, 2024, 2:34pm

Thanks for the reply and the explanation.

But I am not sure how can reverse engineer why a package has been installed.

Let’s consider jq-*-man:

$ nix why-depends /run/current-system /nix/store/ffqrphr0j6phc76lj4pwbwir3s0cgij7-jq-1.7.1-man
/nix/store/g8piafj0jaj989z89w5fnq2bxy5hy08h-nixos-system-nixos-24.05.20240216.785ec14
└───/nix/store/0n514iqgfx6sa0gzv9x19ng6d29jlr5a-system-path
    └───/nix/store/02bcny6zbi9a2sl93v7dcm4snnar43px-nixos-rebuild
        └───/nix/store/0pvx8malgl8yw5f1bdzxxpjbrg0pa47l-jq-1.7.1-bin
            └───/nix/store/ffqrphr0j6phc76lj4pwbwir3s0cgij7-jq-1.7.1-man

$ nix derivation show nixpkgs#jq
{
  "/nix/store/g5zniv7c4ilcypfrs3hg1shpbdx2z6r3-jq-1.7.1.drv": {
    "args": [
      "-e",
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh"
    ],
    "builder": "/nix/store/5l50g7kzj7v0rdhshld1vx46rf2k5lf9-bash-5.2p26/bin/bash",
    "env": {
      "NIX_CFLAGS_COMPILE": "",
      "__structuredAttrs": "",
      "bin": "/nix/store/0pvx8malgl8yw5f1bdzxxpjbrg0pa47l-jq-1.7.1-bin",
      "buildInputs": "/nix/store/fdaqq1pz0w96ra3y2prspmmg12l14963-oniguruma-6.9.9-dev",
      "builder": "/nix/store/5l50g7kzj7v0rdhshld1vx46rf2k5lf9-bash-5.2p26/bin/bash",
      "cmakeFlags": "",
      "configureFlags": "--bindir=${bin}/bin --sbindir=${bin}/bin --datadir=${doc}/share --mandir=${man}/share/man LDFLAGS=-Wl,-rpath,\\${libdir}",
      "depsBuildBuild": "",
      "depsBuildBuildPropagated": "",
      "depsBuildTarget": "",
      "depsBuildTargetPropagated": "",
      "depsHostHost": "",
      "depsHostHostPropagated": "",
      "depsTargetTarget": "",
      "depsTargetTargetPropagated": "",
      "dev": "/nix/store/aa4cw3vy0vvaxi7jy8i6qp89vv10w15v-jq-1.7.1-dev",
      "doCheck": "",
      "doInstallCheck": "1",
      "doc": "/nix/store/jb68s0n2wr0cv9xffnh2i4gwv7dzv16n-jq-1.7.1-doc",
      "installCheckTarget": "check",
      "lib": "/nix/store/lnbk4y3dcl1pgwrld3wrzjybbh6m58sv-jq-1.7.1-lib",
      "man": "/nix/store/ffqrphr0j6phc76lj4pwbwir3s0cgij7-jq-1.7.1-man",
      "mesonFlags": "",
      "name": "jq-1.7.1",
      "nativeBuildInputs": "/nix/store/mkjqdhrv8l2hgigygkqlgn6g74ala5jg-remove-references-to /nix/store/387mwzr43v75ga53g2a1jmy60jn4hs9q-autoreconf-hook /nix/store/25p9rdlq32nxj1mjisln1dfc2vnlmjn3-bison-3.8.2",
      "out": "/nix/store/q0af2k8nwbzfr2awprlqgyfh03mhm0gn-jq-1.7.1",
      "outputs": "bin doc man dev lib out",
      "patches": "",
      "pname": "jq",
      "postFixup": "remove-references-to -t \"$dev\" \"$bin/bin/jq\"\n",
      "postInstallCheck": "$bin/bin/jq --help >/dev/null\n$bin/bin/jq -r '.values[1]' <<< '{\"values\":[\"hello\",\"world\"]}' | grep '^world$' > /dev/null\n",
      "preBuild": "rm -r ./modules/oniguruma\n",
      "preConfigure": "echo \"#!/bin/sh\" > scripts/version\necho \"echo 1.7.1\" >> scripts/version\npatchShebangs scripts/version\n",
      "propagatedBuildInputs": "",
      "propagatedNativeBuildInputs": "",
      "src": "/nix/store/vm0wql28kz1lc8cjs8x5350a2r2fhwx7-jq-1.7.1.tar.gz",
      "stdenv": "/nix/store/yq9qag5ak3w15nhcqplnhxpny6295l36-stdenv-linux",
      "strictDeps": "",
      "system": "x86_64-linux",
      "version": "1.7.1"
    },
    "inputDrvs": {
      "/nix/store/fjfk45i776qf9h7s5n7b7mwm1hfx3r1b-oniguruma-6.9.9.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "dev"
        ]
      },
      "/nix/store/l0hky0kg5r7xjf59gy8nhl0ia0kfa5w8-remove-references-to.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/nw35npbmh0x64r55plyi2fayzc33zhv2-jq-1.7.1.tar.gz.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/rhl3znab3ihy1zbnplm1g0k8asi4xhxq-bison-3.8.2.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/xcarlmvv7qlwnqcliswbfbsr04mdsb7m-bash-5.2p26.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/z0425brv3rqr58aq29zsv8qmc4g6v38j-stdenv-linux.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      },
      "/nix/store/zj8vrg7axxvb3311qprpmd9p0psh465j-autoreconf-hook.drv": {
        "dynamicOutputs": {},
        "outputs": [
          "out"
        ]
      }
    },
    "inputSrcs": [
      "/nix/store/v6x3cs394jgqfbi0a42pam708flxaphh-default-builder.sh"
    ],
    "name": "jq-1.7.1",
    "outputs": {
      "bin": {
        "path": "/nix/store/0pvx8malgl8yw5f1bdzxxpjbrg0pa47l-jq-1.7.1-bin"
      },
      "dev": {
        "path": "/nix/store/aa4cw3vy0vvaxi7jy8i6qp89vv10w15v-jq-1.7.1-dev"
      },
      "doc": {
        "path": "/nix/store/jb68s0n2wr0cv9xffnh2i4gwv7dzv16n-jq-1.7.1-doc"
      },
      "lib": {
        "path": "/nix/store/lnbk4y3dcl1pgwrld3wrzjybbh6m58sv-jq-1.7.1-lib"
      },
      "man": {
        "path": "/nix/store/ffqrphr0j6phc76lj4pwbwir3s0cgij7-jq-1.7.1-man"
      },
      "out": {
        "path": "/nix/store/q0af2k8nwbzfr2awprlqgyfh03mhm0gn-jq-1.7.1"
      }
    },
    "system": "x86_64-linux"
  }
}

How do I proceed?

Thanks

Atemu · February 16, 2024, 3:56pm

As I suspected, the bin output of jq somehow depends on the path to jq’s man pages which are in its man output. That could happen because an executable might reference the man page directory in its help output perhaps. You’d have to dig into the package’s build system to find out why it does this.

Also note that not all packages have separate outputs. In fact, I’d wager most packages in Nixpkgs only have a single combined output and it usually includes the man pages.

What is it that you are trying to achieve here?

yellowhat · February 16, 2024, 4:23pm

I am trying to understand the balance between how easy can a system be defined (*.nix files) and their flexibility to create a minimal system for a specific task.

I use gentoo therefore I understand that shipping binary packages means “you have to make a choice” on which flag to enable, otherwise you have to compile it.

Also I am trying to get more insight on what are the defaults.

Anyway thanks for you explanations.

Atemu · February 17, 2024, 12:44pm

Currently, Nixpkgs does not strive to be “minimal” or configurable. There’s nothing major stopping us from doing that in theory but it’s simply not being done in practice. Obviously, if you chose to make your derivations deviate from the defaults, that means compiling them yourself but that’s not a major burden on anyone using the defaults.

I’d generally consider it good practice to do things like separate outputs or configurable defaults but most packages in Nixpkgs don’t follow such practices yet.

One reason why this isn’t being done is that there’s no standard on it. We don’t have anything like standardised USE-flags. Some progress towards that is in [RFC 0169] Feature parameter names by KAction · Pull Request #169 · NixOS/rfcs · GitHub for instance.

The term you’re looking for btw is to reduce your “closure size”. You can find issues with that label in Nixpkgs.

The current sentiment towards that as I see it is that people aren’t opposed to closure size reductions but very few are going out of their way to do anything about it. If it’s convenience vs. slightly smaller closure, the former is usually chosen.

This is understandable because storage size is not really something we need to worry about these days in the vast majority of cases.

yellowhat · February 17, 2024, 3:43pm

Thanks for the clarification.