How to know which package in my HM config depends on another?

Help
1

I want to know which part of my Home Manager configuration requires pyopenssl which currently has some issue https://github.com/NixOS/nixpkgs/issues/175875

I can’t build my current HM config because of the above mentioned issue, so I tried taking the latest generation and…

$ nix why-depends /nix/store/vn406wg5p62m9cp9ljwh0kqk2zs44a6l-home-manager-generation nixpkgs#python310Packages.pyopenssl
'/nix/store/vn406wg5p62m9cp9ljwh0kqk2zs44a6l-home-manager-generation' does not depend on '/nix/store/wwy5jhcykcdv13a9b9rs0zbsf20np1fg-python3.10-pyopenssl-22.0.0'

It tried it with python39Packages.pyopenssl but same result. I find it strange that nothing in my HM generation would depend on either of these packages so clearly I’m doing something wrong here.

Or maybe I can somehow tell home-manager --some-cool-flag to show me a tree of dependencies for the broken package? --show-trace doesn’t do that and simply going by the order in which packages are built seems like a rather roundabout way of achieving this.

But more context

$ home-manager build --flake .#work-m1 --keep-failed --impure
building '/nix/store/b3a57gh0h3b15vk59v1x194ngk6w8gmx-hm_nvimafterftpluginjavascript.vim.drv'...
building '/nix/store/17lsimxb5xhb6wxpd8yx9g21jy4s8fdk-hm_nvimafterftplugintypescript.vim.drv'...
building '/nix/store/kv0206xi6y27r4f30rhni1z0vm050f88-man-pages.drv'...
building '/nix/store/ajig7ihww1kql7bfz7y4ixpfqibvq392-python3-3.10.5-env.drv'...
building '/nix/store/r1mfk1jjr9d8crd3gs7p8fi6jb4n3db7-man-pages-fish-completions.drv'...
building '/nix/store/x3gcky3i6c5iw6w3dmxd4cg32zf75y9h-python3-3.10.5-env.drv'...
building '/nix/store/z3sdhqlw7rvya73az185xy4572xzap5r-python3.10-awscrt-0.13.11.drv'...
building '/nix/store/5p6pib26psf832ydlz9sw4rknf9hwagv-python3.10-jmespath-0.10.0.drv'...
building '/nix/store/idinknph0kjppswdzinm2ccvwyx4sjcf-python3.10-pyopenssl-22.0.0.drv'...
building '/nix/store/8l2vz7ij8z1qvqppliis7kbqllf0fhlp-python3.10-skia-pathops-0.7.2.drv'...
note: keeping build directory '/private/tmp/nix-build-python3.10-skia-pathops-0.7.2.drv-0'
...

I then took the derivation it’s currently trying to build:

$ NIXPKGS_ALLOW_BROKEN=1 nix why-depends --impure /nix/store/mph0jkk74v7jmcpgnay12v5v0jr32ma0-home-manager-generation.drv nixpkgs#python310Packages.pyopenssl
building '/nix/store/5rimhv0v8c0df72brwqnvd0xhnw6xdn8-vscode-keybindings.drv'...
building '/nix/store/8l2vz7ij8z1qvqppliis7kbqllf0fhlp-python3.10-skia-pathops-0.7.2.drv'...
builder for '/nix/store/8l2vz7ij8z1qvqppliis7kbqllf0fhlp-python3.10-skia-pathops-0.7.2.drv' failed with exit code 1; last 10 log lines:
      self.build_extension(ext)
    File "setup.py", line 193, in build_extension
      libraries=self.get_libraries(ext),
    File "setup.py", line 210, in get_libraries
      library_dir = self._library_builders[library](self.build_temp)
    File "setup.py", line 279, in build_skia
      subprocess.run(build_cmd, check=True, env=env)
    File "/nix/store/6a94sw6yyc412y3a0qjnjdyljx3p1b7n-python3-3.10.5/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['/nix/store/6a94sw6yyc412y3a0qjnjdyljx3p1b7n-python3-3.10.5/bin/python3.10', 'src/cpp/skia-builder/build_skia.py', '--no-fetch-gn', '--no-virtualenv', '--gn-path', '/nix/store/g55whagjlih3zzpsd23i0gp5ysmjkifi-gn-unstable-2020-03-09/bin/gn', 'build/temp.macosx-11.0-arm64-3.10/src/cpp/skia-builder/skia', '--no-sync-deps']' returned non-zero exit status 1.
cannot build derivation '/nix/store/2jqjdvawbi8k76r4h0d89jspkfr8yrvy-python3.10-fonttools-4.33.3.drv': 1 dependencies couldn't be built
building '/nix/store/zwwkj80ynamf3k2g5zg7fawqzpyhwv07-python3.10-twisted-22.4.0.drv'...
cannot build derivation '/nix/store/sp98zawf2g9b0ylbhbyq6kn0ykrdjvq8-liberation-fonts-2.1.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/hgwy4qi29r1433crbd7g5bixr6ixdynx-home-manager-fonts.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/6pn4mcxrwvifjb2phx8948fmfgf4w2yg-home-manager-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/7iy25rr5ig27adqqm5wfdczk7p6z9460-liberation-fonts-2.1.0-fish-completions.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/mph0jkk74v7jmcpgnay12v5v0jr32ma0-home-manager-generation.drv': 1 dependencies couldn't be built
error: build of '/nix/store/mph0jkk74v7jmcpgnay12v5v0jr32ma0-home-manager-generation.drv' failed

So I guess I now at least see a narrowed list of dependencies and from this I gather:

liberation-fonts -> twisted
1 Like
2

I don’t think it’s hm that depends on it. It’s the nixpkgs that you most probably following in your hm config, are you using flakes? I pinned my nixpkgs for mac on a particular commit, that helps me rebuild my current config.

I don’t know, without flakes that might be not so straightforward to tackle that issue.

3

I didn’t mean HM literally rather my HM config. I do use flakes and I assume it’s because of some fonts and their dependency on twisted. I can pin it of course but then I don’t get the updates I’d like :\ but I’ll try to just get rid of the offending packages for now.

4

I do this: https://github.com/ivankovnatsky/nixos-config/blob/588f13fe49d5a1769101164c407cfa8c8afd5c5b/flake.nix#L11

5

That’s the dep chain, I don’t use these packages as well, not sure if you will be able to drop that deps. Though I would be really happy to see that dep tree.

6

Note that’s largely unnecessary if you use flakes, due to the lock file. Flakes are designed so that we don’t need to pin specific revisions anymore; if you want to pin to a specific revision, tell your lockfile about it, this allows you to manage the pin with nix commands instead of having to edit the source all the time.

For reference, flakes also automatically pin whatever dependencies you use, you must use nix flake update or nix flake --update-input if you want to update :wink:

Regardless of the version of nixpkgs used, some package in nixpkgs depends on pyopenssl, and even if this is caused by an update it’d be nice to know why. There must be some command that helps figure that out, I’d love to know what it is, so that if I need it in the future I know ;p

1 Like
7

It itches me to ask, even though it’s kind of off-topic. Somehow nixos-rebuild updates my lock file if I remove rev in nixpkgs-mac input. Any way to force no to?

Aha, seems like editing flake.lock manually after moving back to branch pinning solves the next rebuild.

8

Since you previously set the rev in the url, the lockfile has stored the rev being part of the url: https://github.com/ivankovnatsky/nixos-config/blob/588f13fe49d5a1769101164c407cfa8c8afd5c5b/flake.lock#L94

If you now remove that, nixos-rebuild will realize the url changed, and try to update the lockfile (because the url changed, and hence you clearly wanted to grab the new url location, not the old one).

I think in this case hand-editing the lock file, and removing line 94 would fix the issue.

1 Like
9

It is probably a build-time dependency: To show why derivation package has a build-time rather than runtime dependency on derivation dependency, use --derivation.

E.g.

nix why-depends nixpkgs#qemu nixpkgs#python3Packages.pyopenssl --impure --derivation
/nix/store/x3yhxfp796jyd3gmh259c35iwc7b6lc9-qemu-7.0.0.drv
└───/nix/store/h197j6nqkxvc2a597xv8dlhjrfmqnmxy-python3.10-sphinx-4.5.0.drv
    └───/nix/store/s92pshqppiypdg3gsk1i4zqdndfgljcp-python3.10-requests-2.28.0.drv
        └───/nix/store/5wlrnhr5li81gbpq4a8c0pqkv12wcsxd-python3.10-urllib3-1.26.9.drv
            └───/nix/store/idinknph0kjppswdzinm2ccvwyx4sjcf-python3.10-pyopenssl-22.0.0.drv

Note: urllib3 is fixed in https://github.com/NixOS/nixpkgs/pull/179159 and will land in master when https://github.com/NixOS/nixpkgs/pull/179844 is merged.

4 Likes
10

I’m so glad to learn how to better diagnose breakage like this, as I also couldn’t figure out how to track down the non-runtime dependency. I’m even happier to see that this is hopefully fixed now/soon! The breakage caused by these optional Python dependencies was really widespread, was it missed because darwin aarch64 is so low priority as a platform, or could the set of packages whose builds are checked on hydra use some tuning?

I really appreciate all the folks who pitched in to get this sorted!

11

This never works if I use my HM derivation where you used nixpkgs#qemu

$ NIXPKGS_ALLOW_BROKEN=1 nix why-depends --derivation --impure /nix/store/rcnaqv4l9vykm6pn7y37j0fhkhhy52f5-home-manager.drv nixpkgs#python310Packages.skia-pathops
'/nix/store/rcnaqv4l9vykm6pn7y37j0fhkhhy52f5-home-manager.drv' does not depend on '/nix/store/8l2vz7ij8z1qvqppliis7kbqllf0fhlp-python3.10-skia-pathops-0.7.2.drv'

even though building that HM .drv fails because of that Python package. Same if I use pyopenssl.

Just so it’s clear what I’m ultimately trying to do: identify all packages in my HM home.nix that depend on the broken packages so I can maybe try removing them from my HM config.

12

You need to make sure the home-manager generation is being built against the same Nixpkgs the nixpkgs flake currently refers to, otherwise the attribute might point to a different path. Looking at your initial log, something like this might work:

nix why-depends --derivation --impure /nix/store/rcnaqv4l9vykm6pn7y37j0fhkhhy52f5-home-manager.drv /nix/store/8l2vz7ij8z1qvqppliis7kbqllf0fhlp-python3.10-skia-pathops-0.7.2.drv

And since you use flakes, you can also use the attributes conveniently:

nix why-depends --derivation .#homeConfigurations.work-m1.activationPackage .#nixosConfigurations.work-m1.pkgs.python310Packages.pyopenssl

Similarly for NixOS configurations:

nix why-depends --derivation .#nixosConfigurations.azazel.config.system.build.toplevel .#nixosConfigurations.azazel.pkgs.ncurses

Just keep in mind that when the build is failing, you cannot use why-depends without --derivation flag, since the default mode of the command searches for references in the build outputs.

Also useful is nix-store --query --tree /nix/store/rcnaqv4l9vykm6pn7y37j0fhkhhy52f5-home-manager.drv but the ancestor–descendant relationships are not as clearly readable there.

5 Likes
13

You are a wizard. I tried nix flake show to get more info about the contents of homeConfigurations since I actually tried using .#homeConfigurations.work-m1, but I didn’t know that there’s activationPackage inside of it.

❯ nix why-depends --derivation --impure .#homeConfigurations.work-m1.activationPackage .#homeConfigurations.work-m1.pkgs.python310Packages.pyopenssl
warning: Git tree '/Users/fbs/dotfiles' is dirty
warning: Git tree '/Users/fbs/dotfiles' is dirty
/nix/store/qxiw3qgcch2x5yk5srkclczz4807pvqh-home-manager-generation.drv
└───/nix/store/9pwn4z8wab0fg66favf70z6zrx5san2c-home-manager-path.drv
    └───/nix/store/qk7xs5qk701jq411x54d44z0nv6hqwcy-awscli2-2.7.9.drv
        └───/nix/store/q0c8ar8kdlvhns2ai8hlqgnfn5p1p8hy-python3.10-urllib3-1.26.9.drv
            └───/nix/store/lc1zzb935z5lnwb2bfdjqjmr98m903j0-python3.10-pyopenssl-22.0.0.drv

I can’t even begin to thank you enough for helping me with this! :pray: :bowing_man:

I was now able to simply comment out liberation_ttf and awscli2 for now and build a new HM generation :relaxed:

14

I found it using nix repl and :lf . to load the flake.

15

Amazing, this should be in the docs.

16

The breakage caused by these optional Python dependencies was really widespread, was it missed because darwin aarch64 is so low priority as a platform, or could the set of packages whose builds are checked on hydra use some tuning?

Pyopenssl is not supported on aarch64-darwin and led to strange, hard to reproduce downstream errors in other packages. Therefore, it was recently marked as broken. See https://github.com/NixOS/nixpkgs/issues/175875#issuecomment-1166359240 for more info.