I (unfortunately) discovered today that lib.getExe and lib.getExe' are not “safe” in the sense that they are purely string manipulation tools, and their successful execution does not certify that the underlying executable exists.
Is there an alternative that ensures that the underlying executables exist?
The result of a derivation is not knowable without building it, and to check you’d either have to solve the halting problem or do IFD. I’d love a version of getExe which doesn’t fallback to bin/${pname}, but a lengthy deprecation process is required before we get there.
Yes, I’m building the underlying derivation anyways, so I’d like to do IFD
lib.getExe and lib.getBin ?
Check the title of your thread 
Please ignore me !
Shouldn’t be too hard to implement if you’re ok with IFD. It won’t get used in nixpkgs though. And I think you’re downplaying the downsides to IFD here. It can be extremely disruptive to evaluation.
An innocuous little function like this could end up being used everywhere, exacerbating the problem to a terrible degree.
That’s fair, and I could be naive in reaching for IFD. I’m mostly interested in using something for my own personal purposes and not thinking that this would be relevant to nixpkgs at large.
I think the best solution is to test whether meta.mainProgram exists during installCheckPhase
I have one derivation that has three executables and I’m looking to create writeShellApplications derivations, one for each of the executable.
that could work, or maybe even just a
getExe2 = drv: progName: (runCommand drv.name {} ''
test -f ${lib.getBin drv}/bin/${lib.escapeShellArg progName}
ln -s ${lib.getBin drv} $out
'') + "/bin/${lib.escapeShellArg progName}";
It’s a neat idea, not that i’d ever use it
if what you want is a lib.forEach based on the contents of a derivation, then IFD is the only choice.
Very clever, thank you! (20 charaaaaaaaaaaaacters)
I do not think it is desirable for Nix functions to force realization when evaluating a function call. But it would be nice if we could do something like nix-instantiate --eval-realized --expr 'let pkgs = import ./. {}; in pkgs.lib.getExe pkgs.hello' and we have a feature request for that.
In that context, I wonder if we could get a feature upstreamed into ShellCheck that lets us specify an assumed PATH and would check that every executable is in that PATH. The maintainer doesn’t seem to think too highly of this approach but in Nixpkgs we have a real shot at using it robustly, and maybe we could build a convincing argument for it.
All of this amounts to sanity checking at build time, with different ergonomics and runtime costs. I’d prefer a stricter getExe' which checks if the program name is a member in a declared whitelist at e.g. drv.meta.allPrograms which is checked to be correct while building drv.
Somewhat related to all this, i’ve been wanting to have meta.mainProgram=null; be a sanctioned way to dissallow using nix run and lib.getExe, if there either are no binaries or none that can be considered the main binary.
How is this not just getExe with extra steps?
It will still not realize the path when pure getExe would not:
$ nix-instantiate --eval --strict --expr 'let pkgs = import ./. {}; getExe2 = drv: progName: (pkgs.runCommand "${drv.name}-wrap" {} "test -f ${pkgs.lib.getBin drv}/bin/${pkgs.lib.escapeShellArg progName}; ln -s ${pkgs.lib.getBin drv} $out") + "/bin/${progName}"; in getExe2 pkgs.hello "hello"'
"/nix/store/629h9n0r2wi926sndrhn5ybnsjj1vqsm-hello-2.12.1-wrap/bin/hello"
$ file /nix/store/629h9n0r2wi926sndrhn5ybnsjj1vqsm-hello-2.12.1-wrap/bin/hello
/nix/store/629h9n0r2wi926sndrhn5ybnsjj1vqsm-hello-2.12.1-wrap/bin/hello: cannot open `/nix/store/629h9n0r2wi926sndrhn5ybnsjj1vqsm-hello-2.12.1-wrap/bin/hello' (No such file or directory)
(Nix does not need to realize runCommand derivation to get its output path, evaluation is enough, just like with the derivation interpolation in getExe.)
$ nix-instantiate --eval --strict --expr 'let pkgs = import ./. {}; in pkgs.lib.getExe pkgs.hello'
"/nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1/bin/hello"
$ file /nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1/bin/hello
/nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1/bin/hello: cannot open `/nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1/bin/hello' (No such file or directory)
And getExe will be sufficient in cases where the derivation is realized:
$ nix-build --expr 'let pkgs = import ./. {}; in pkgs.runCommand "foo" {} "echo ${pkgs.lib.getExe pkgs.hello} > $out"'
this derivation will be built:
/nix/store/338xcx1p809hlnkffv8al7g1qh8zq57d-foo.drv
this path will be fetched (0.05 MiB download, 0.22 MiB unpacked):
/nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1
copying path '/nix/store/63l345l7dgcfz789w1y93j1540czafqh-hello-2.12.1' from 'https://cache.nixos.org'...
building '/nix/store/338xcx1p809hlnkffv8al7g1qh8zq57d-foo.drv'...
/nix/store/73mkdnil6qcng3knj7915gfmvjnn4m5l-foo
I do not understand what @samuela is trying to achieve that does not work. Unless they are using nix repl for prototyping (but then it should be expected that repl does just evaluation).
Yea, that getExe2 function is not useful. But what @samuela is trying to achieve is an eval-time way to get the correct exe name. e.g. if a package named foo has an exe that is not named foo but instead named bar and the package does not include a meta.mainProgram attribute, then the derivation should be built, and it should eval the necessary readDir calls to find a suitable exe name. Something like this:
{
getExe2 = drv: let
regFiles = lib.mapAttrsToList (f: _: f) (lib.filterAttrs (_: t: t == "regular") (builtins.readDir "${drv}/bin"));
mainProg = drv.meta.mainProgram or lib.head regFiles;
in "${drv}/bin/${mainProg}";
}