Limited interface to system/nix-external dependencies?

Development Nix
1

Edit: I went ahead and opened an issue for this: Limited interface to system/nix-external dependencies? · Issue #8192 · NixOS/nix · GitHub

Packages that need run-time access to executables that aren’t/can’t be in the Nix store tend to be awkward if not broken. I have a vague proposal that might be an improvement; looking for people with ~ecosystem perspective to poke holes or help refine…

The main cases are setuid (like sudo) and system/platform executables (like pbcopy on macOS).

Background/problem

You can skip this section if you're familiar with the issues here.

I’ll focus on sudo since it’s a good example of both cases, and it’s a common problem for Nix users on both NixOS and non-NixOS systems.

The store can’t contain setuid executables for security reasons. sudo will refuse to run if it isn’t setuid 0, so pkgs.sudo is a common a stumbling block. You can use it to embed a path to a sudo executable, but it’ll fail if it ever runs. Testing/review will catch this if you’re lucky, but if it’s a rare code path you’ll end up with a time-bomb. I see some in nixpkgs that probably don’t work as the packager intended. The sudo package is marked as platform linux, so another side effect of this is that Nix will refuse to install packages using pkgs.sudo on non-Linux systems like macOS by default.

Aside: pkgs.sudo builds on macOS if you force the issue, but I don’t personally see much point in “fixing” the platform just to increase the number of people who get tricked into depending on it…

Some people try to work around this if they know/realize it, but we don’t have good tools so results are inevitably mixed:

  • Since setuid executables on NixOS are implemented as wrappers located in /run/wrappers/, some packages hardcode this path (I currently see 73 instances of /run/wrappers in 53 files within nixpkgs/pkgs/). These won’t be working outside of NixOS.
  • Some packages patch sudo out of invocations. This is probably the right call if the sudo invocation is in a build/test/install script or is otherwise superfluous, but it wouldn’t shock me if this occasionally bites someone.

Other uses of sudo in packages probably go unnoticed, or get left alone by people who know pkgs.sudo is a trap.

  • If these are bare commands (I could find at least one of these in my own store), this will be fine if the package is run with a normal PATH–but it’s probably also causing some people grief if they use them in service definitions, test frameworks, or other contexts where a weird PATH is common.
  • If these are abspaths like /usr/bin/sudo, they’re presumably failing on NixOS (not sure if any other Nix-supported systems have a sudo that isn’t @ /usr/bin/sudo?)

Proposal

This is ~sparse. I don’t grok all of the pressures/interests here and don’t want to fret details that may be moot if everyone hates this :slight_smile:

  • Add an eval and/or build-time oracle to Nix that can, given an arbitrary executable name, hand out a stable path under Nix’s control. To ensure the contents of the path don’t affect the build, the path itself shouldn’t be accessible in the build sandbox.

    (To avoid people hardcoding these paths, “stable” should probably not mean a priori “predictable”?)

  • Preserve these declared external dependencies (in the derivation? within nix-support?) for Nix to handle at realisation time.

  • At realisation time, attempt to resolve these external dependencies to paths present on the system. If they are found, symlink the stable path to the system executable. If they are not, error and ask the user to help disambiguate. (The user might be able to give Nix the correct path, obtain the dependency and retry, report a packaging problem, etc.)

    I don’t have a strong opinion on how implicit vs. explicit resolution should be. AFAIK, ~system executables should usually be locatable with something equivalent to PATH="$(getconf PATH)" type -p <executable>, and I imagine that’ll almost always be the expected behavior. But some kind of config/lockfile mechanism for edge cases also feels inevitable, so it wouldn’t be the end of the world if an ~implicit mechanism just suggests additions and keeps the user in the loop?

    Aside: This naive getconf approach won’t actually work on NixOS atm, but I’m unsure if that’s a mistake or not. I noticed that /run/wrappers/bin is in my PATH on NixOS but is not in the PATH returned by getconf PATH (/run/current-system/sw/bin:/bin:/usr/bin). Anyone have perspective on the pressures, here?

    There’s some context from 2013 in the PR that added /run/current-system/sw/bin. The last comment by @peti suggests maybe patching getconf to just return the existing PATH env (when it is set) might be better. That sounds like it would fix this issue on NixOS without adding additional PATH components for non-NixOS Linuxes, but I assume it would also meaningfully change the behavior of getconf PATH on those systems. IDK if the splash damage from that would be manageable or not.

  • Add some mechanism(s) for ~managing the links (GC unused links? re-resolve broken ones? update links per user changes to config/locking mechanism? maybe list existing links?)

For thematic brevity, I nicknamed this affordance rime.

3 Likes
2

Relevant work I can think of:

  • Distri has a concept that I think is similar in spirit called ‘exchange directories’. I think that abstraction would be worth looking at.
  • @shlevy at some point had an interest in creating a mechanism for adding selective or restricted impurities to the Nix store. I think this is part of what he wanted to achieve with ‘Nomia’.

Work I think might be relevant but can’t gather the details for at the moment:

  • I think Denxi tries to offer abstractions like Nix but without strictly requiring Nix’s discipline all the time
  • A couple of years ago, when Qt packages were broken due to some incompatibilities, there was a proposal to patch our Qt libs to look for plugins in a different way, which was rejected by the maintainer of those packages at the time in favor of fixing up their existing solution, which involved performing lookups based on what could be discovered in the PATH variable. One of their motivations was that everything in Nixpkgs can normally be run directly from the Nix store, not requiring packages to be ‘installed’ to any kind of profile in order to work. I think the discussion surrounding those issues might help us figure out some pros and cons.

Prima facie, I think something like rimes or exchange directories is a good idea. A use case which strikes me as important for adoption and maybe other things is better GPU acceleration support on foreign Linux systems.

I need to think about this more in order to really make up my mind. But I like that this proposal exists and look forward to hearing what others think.

1 Like
4

You mentioned some broader concepts, which reminds me that I’ve seen another (nix-specific) proposal in this space: pivot-notes.md · GitHub (but it didn’t sound like anyone involved currently has interest in advancing it).

1 Like
5

Shameless bump, in case it spares me from having to bug the Nix team with something this ~raw :innocent:

1 Like
6

I went ahead and opened an issue for this:

https://github.com/NixOS/nix/issues/8192

7

Don’t know if I’m too late, but I just found an instance of this problem that is really bugging me and could (hopefully) build some context for people coming here and not knowing what this is all about, and for users encountering the same problem and being left in utter confusion.

Emacs constructs its remote path for executing commands on remote servers via getconf PATH. Since this command does not return /run/wrappers/bin, I cannot execute any commands that require elevated privileges on NixOS servers, forcing me to take the long way and spawn an actual remote shell in a terminal emulator. I think there are many applications that rely on getconf returning a correct path, and this seems to me a misbehaviour and major source of incompatibility.

In addition, I have a suspect that issues like pam_mount: `/run/wrappers/bin` seems not to be in $PATH so `fusermount` fails · Issue #201368 · NixOS/nixpkgs · GitHub might be related to the same cause.

@abathur did you have any other exchange with the Nix team surrounding this problem? I see that the discussion seems to have stagnated, somewhat.

2 Likes
8

No. It apparently got triage in a nix team meeting. I responded to the confused reaction but haven’t heard anything else.

We might need to be a squeaky wheel.

2 Likes
9

I’m still interested in this. Being able to manage impurity via Nix instead of just talk about it vaguely would be great.

There are places in Nixpkgs that already make room for a bit of impurity deliberately, too. Managing them with something like this might be better.

Being able to do away with stuff like nixGL would also be great.

Setting a precedent for systematizing all that by thinking through the case of foreign executables would be great.

2 Likes
10

There’s a new issue broadly related to managing impurity: Fine-grained impurity · Issue #8865 · NixOS/nix · GitHub

1 Like