Should we support encoding runtime dependencies without requiring a specific implementation?

Development
1

Right now the only way to specify runtime dependencies is to hard-code an implementation using something like makeWrapper. Although, there are a lot of cases I think it would be useful to encode that a specific file must exist or that a specific binary must exist in $PATH, without tying down the package to a specific implementation.

For example:

{ stdenv }

stdenv.mkDerivation {
  name = "example";

  ...

  meta = {
    runInputs = {
      paths = [ "/run/opengl-driver" ];
      executables = [ "wine" ];
    };
  };
}

Then nixos-rebuild could use this metadata to verify that the final resulting profile satisfies these dependencies.

1 Like
2

Can you link/cite a concrete example?

3

I think it would make a lot of sense!

It even applies much more broadly, you could e.g. delay choosing a libc implementation, or a docker/container runner, or a Python implementation, for an application. Conceptually you’d want to build a derivation against an API, then decide on an implementation of that API in a separate derivation. You can then only build the first derivation once, then have a (cheap) build for each combination with an implementation.

1 Like
4

Requiring something in $PATH etc is kinda annoying IMHO because it breaks expectations for things like nix run. However you can do a two-level, where the wrapper is in a separate derivation, so you don’t have to rebuild the ‘base’ to change the ‘wrapper’. IIUC things like neovim and clang already use this approach.

5

xdg-utils is already a good concrete example of this as apps which open URLs assume xdg-open will be on the path, and they can call xdg-open https://example.org/ to open it. handlr was already linked as an example replacement.

Another example is replacing xdg-open with an implementation which uses dbus to ask the portal to launch apps like this https://github.com/LunNova/nixos-configs/blob/1c7264b860ca18c582d2cdf18b119c862b053fa7/packages/xdg-open-with-portal/default.nix Currently there’s a mix of packages picking it up from path and wrapping it themselves.

I don’t know how often this applies in the general case, and I can’t remember other examples off the top of my head right now.

1 Like
6

Oh yep here are some examples where I think this would make sense:

7

That’s true for most cases, but I think it makes sense in certain circumstances. I feel like for nix run we could just give the same kind of error message pointing out that a runtime dependency is missing.

8

For certain common tools which one might want to replace, like xdg-open, it seems to me like nixpkgs config, combined with the two-level structure @dramforever suggested, is the right way to handle it. That way you can set in one place what you want to use for that, but you still keep the self-contained nature of a nix derivation closure.

1 Like
9

This sounds similar to npm’s concept of “peer” dependencies.

2 Likes
10

Yep, I think that’s the ideal, but the problem comes when there’s no good default, and it makes more sense to take what the user set on $PATH.

For example, I mentioned doing the exact same thing for texlab (that it should depend on a TeX distribution), but the other reviewers decided against it because they mentioned that many users may have a different TeX distribution installed, and expect that to be used instead:

https://github.com/NixOS/nixpkgs/pull/85700#issuecomment-617438601

For these situations I’m suggesting that we have some other mechanism to determine when these dependencies are satisfied, instead of just leaving the dependency off altogether.

11

Another similar existing example is /run/opengl-drivers. Packages build against mesa, and then at runtime depend on whatever is symlinked in /run/opengl-drivers.

1 Like
12

This sounds similar to npm’s concept of “peer” dependencies .

Kind of, but more general, instead of requiring a specific nix dependency, I think we want to require a specific path or executable to exist in $PATH

13

I’ve updated the description to give a better sense of how I think something like this would be implemented.

1 Like
14

For /run/opengl-drivers/lib specifically, there is a addOpenGLRunpath hook which will add the RUNPATH to all elf files:

  nativeBuildInputs = [ addOpenGLRunpath ];

and then:

objdump -x ./result/bin/<elf> | grep opengl
15

Yeah, but that hook can’t actually verify that /run/opengl-drivers actually exists in the final system profile during build time, it will just fail to load the library at runtime.

16

correct, on the consumer side there’s GitHub - nix-community/nixGL: A wrapper tool for nix OpenGL application [maintainer=@guibou]. The problem is trying to add some impurity, without opening the floodgates of /usr/local/lib

17

Do you think it would help to have some formal way to describe these runtime impurities like what I mentioned in the main description though? Right now we assume these exist at runtime and fail at runtime if they don’t, but I feel like we could verify this earlier during profile generation.

18

The problem is c-style dynamic linking. There’s only so much we can do when an environment “assumes” that libraries will be co-located with everything else. :frowning:

You might be able to do something with /etc/nix-ld.so.preload https://github.com/NixOS/nixpkgs/blob/8e36f0c4d18a55630954ff2206b1c05ec3fb8bb5/pkgs/development/libraries/glibc/dont-use-system-ld-so-preload.patch, if it only contained the paths of what was meant to be in /run/opengl-driver/lib.

This probably isn’t a good answer, but I don’t think there is one either; otherwise some crazy nix contributor would probably have already done it :slight_smile:

19

Part of the problem is that even nixos doesn’t actually know for sure what’s in PATH at runtime. It comes from multiple sources, including home-manager and nix-env for user-installed stuff.

Part of the overall philosophy of the nix ecosystem is to use build-time resolution of resources wherever possible. This philosophy isn’t incidental. It’s an important part of getting many of the benefits we get from it. PATH lookups are an exception to this philosophy, because you don’t want to reboot just to get a new package installed, but this doesn’t seem like it qualifies for an exception to me. If the current UX is bad, we need to make the build-time resolution’s configuration process easier, not start trying to guarantee runtime resolution will work. It can’t even be done in environments like nix-env and nix profile anyway… not without a major overhaul to the architecture of nix itself, which I would view as quite detrimental.

20

With CA derivations, it might be possible to add a hardware drivers stub, and then people don’t have to rely on any runtime hacks to find the more specific hardware libraries.

The stub is just there so that the actual drivers only have a single downstream package. This should make it cheap for the contents to be replaced with something else without having to rebuild the actual immediate downstream dependencies.