How to get a list of executable files in a specific directory (not including subdirectories) with nix?

I tried, but nix is broken unrecoverable.
I just wanted to do builtins.exec…

https://github.com/NixOS/nixpkgs/blob/ceac21cff7f6cc875647515a56ef98d431200035/pkgs/applications/misc/houdini/default.nix

executables = [ “bin/houdini” “bin/hkey” “houdini/sbin/sesinetd” ];

In this way, without listing all executable files in advance,
If I just set a directory, I want to extract only executable files from it and put it in a list.
builtins.exec did not work,
Is there any way to do it using at least bash ?
Should I choose a method by making a specific program and creating a file such as json?
I wish there was a way to do it using only nix.

Nix isn’t really designed to do that. It’s a functional DSL for managing build scripts. Even if you could manage file paths with it this way, it would probably be painful and inefficient.

That said, I’m a bit confused by your derivations here. You put in a lot of effort to create “wrapped” “executables”, but the only difference between un-wrapped and wrapped is that the wrapped ones have a shebang of "#!${stdenv.shell}".

For reference, “wrapped” in nixpkgs usually refers to binaries that need to have special environment variables set, and therefore have “wrapping” scripts that set those variables and then execute the binaries. Patching shebangs usually happens directly on the scripts, so I’m not sure how we ended up with this.

The “un-wrapped” binaries are provided from a nix derivation in the first place anyway - are you sure they don’t already have the correct shebangs set? Nix will automatically patch shebangs after installPhase, see the fixupPhase section in the stdenv manual entry.

If it they don’t already have the correct shebangs, that is most likely because your scripts aren’t set to be executable at the end of the install phase. So rather than trying to hack around the paths in default.nix, I would instead put this at the end of installPhase in runtime.nix.

Now, all of that said, the problem you’re describing is very hard to solve! Since you’re rewriting shebangs, I assume these are all scripts, but either way, the only way to identify whether a file is “executable”, unless it has the executable permission set, is to check if it starts with a shebang. To my knowledge, you can’t easily do this with a single command in bash, and no matter which language you do it in it will be somewhat slow because you need to check file contents.

If the shebangs are really not already correct when you’re creating your fhs env, and you’re sure the patchShebangs hook runs in fixupPhase, I’d suggest trying to go upstream and seeing if they’d be willing to make their install script make files executable before trying to hack this. But I’d be very surprised if it doesn’t do that already.

If you’re absolutely positive you need to hack this, we can work out a grep or a find or an awk or something, but that should be a last resort.

2 Likes

Instead of trying to use nix to find files and then again build shell values from the nix value, why not use find?

1 Like

I suspect it’s because the executables aren’t marked as executable, so find by itself can’t do that. If they are executable, then you want something like this:

find <dir> -executable -exec sed 's/#!.*/#!${stdenv.shell}/' {} > $out/{} \; 

Except you can’t specify the found binary twice, of course, so that takes some hacking.

But then stdenv.mkDerivation should already be doing that in fixupPhase, and more accurately, so doing this is pointless.

1 Like

Though then I do not see how nix would be able to help with that, by using builtins.exec, which I understand as the eval time version of pkgs.runCommand

1 Like

ls | bb -i ‘(vec (filter #(-> % io/file .canExecute) *input*))’ > executable-files-list

hmmm
I am looking for a way to load immediately after converting to a nix list. What was the best way… I was a bit surprised that there was no executable access.
In particular, flake does not have access to environment variables on Mac.

exclude symbolic link, And subtracting the number 7(chmod) , etc…
I’m thinking of a certain way & simple & comfortable.

Can you please elaborate on the why you need a list of executables in a derivation?

In theory you could get it at the cost of IFD, but again, why?

let
  executablesTxt = runCommand "hello-executables.txt" {} ''
    ${findutils}/bin/find ${hello} -executable -not -type d | tee $out
  '';
  executablesPre = lib.strings.split "\n" (builtins.readFile executablesTxt);
  executables = lib.lists.filter (e: ! (e == "" || e == [])) executablesPre;
in { inherit executables; }

Removing any unwanted prefixes is left as an exercise to the reader.

1 Like

If they aren’t marked as executable, nothing really could help, as the “executables” aren’t executable then.

If find can’t say if the file is meant to be executable, then nix can’t say either.

2 Likes

houdini is also unfree, so there’s not much we can do in the official repository to support this use case. The closest thing would be meta.mainProgram which can help you list which program to run with nix run.

For normal packages, if your root user follows a nixos-* channel, you can do something like:

$ sqlite3 /nix/var/nix/profiles/per-user/root/channels/nixos/programs.sqlite "select name from Programs where system = 'x86_64-linux' and package = 'binutils';"
as
ld
ld.bfd
ld.gold
strip
1 Like

I made it like this…
with bb(babashka & bash) get redirected to files-list (mono-cmds).
but, not automatic…
anyway, (When executing makewrapper with reference to external binaries with many execution commands…)
I’m still in the introductory stage to nix, and it’s not bad, but (I’ll be able to add comments… within that’s nix script)
,
In the future… If there is a newly updated executable binary, it automatically finds and registers it (it searches all sub-directories, then it would be better.)

mono-cmds = ["ClassInitGenerator" "IronRuby.Tests" "al" "al2" "aprofutil" "asp-state" "asp-state2" "asp-state4" "autopoint" "cairo-trace" "caspol" "cccheck" "ccrewrite" "cert-sync" "cert2spc" "certmgr" "chktrust" "cjpeg" "crlupdate" "croco-0.6-config" "csc" "csharp" "csi" "csslint-0.6" "dbsessmgr" "dbsessmgr2" "dbsessmgr4" "disco" "djpeg" "dmcs" "dtd2rng" "dtd2xsd" "envsubst" "fastcgi-mono-server" "fastcgi-mono-server2" "fastcgi-mono-server4" "fax2ps" "fax2tiff" "fc-cache" "fc-cat" "fc-list" "fc-match" "fc-pattern" "fc-query" "fc-scan" "freetype-config" "fsharpc" "fsharpi" "fsharpiAnyCpu" "gacutil" "gacutil2" "gapi2-codegen" "gapi2-fixup" "gapi2-parser" "gdbus" "gdbus-codegen" "gdk-pixbuf-csource" "gdk-pixbuf-pixdata" "gdk-pixbuf-query-loaders" "genxs" "gettext" "gettext.sh" "gettextize" "gif2epsn" "gif2ps" "gif2rgb" "gifasm" "gifbg" "gifburst" "gifclip" "gifclrmp" "gifcolor" "gifcomb" "gifcompose" "giffiltr" "giffix" "gifflip" "gifhisto" "gifinfo" "gifinter" "gifinto" "gifovly" "gifpos" "gifrotat" "gifrsize" "gifspnge" "giftext" "gifwedge" "gio-querymodules" "glib-compile-resources"  "glib-compile-schemas" "glib-genmarshal" "glib-gettextize" "glib-mkenums" "gobject-query" "gresource" "gsettings" "gtester" "gtester-report" "gtk-builder-convert" "gtk-query-immodules-2.0" "gtk-update-icon-cache" "httpcfg" "icon2gif" "ikdasm" "ilasm" "illinkanalyzer" "installvst" "intltool-extract" "intltool-merge" "intltool-prepare" "intltool-update" "intltoolize" "ipy" "ipy64" "ipyw" "ipyw64" "ir" "ir64" "irw" "irw64" "jpegtran" "lc" "libglade-convert" "libpng-config" "libpng14-config" "llc" "llc64" "macpack" "makecert" "mconfig" "mcs" "mdassembler" "mdbrebase" "mdoc" "mdoc-assemble" "mdoc-export-html" "mdoc-export-msxdoc" "mdoc-update" "mdoc-validate" "mdvalidater" "mkbundle" "mod" "mod-mono-server" "mod-mono-server2" "mod-mono-server4" "mono" "mono-api-html" "mono-api-info" "mono-boehm32" "mono-cil-strip" "mono-configuration-crypto" "mono-find-provides" "mono-find-requires" "mono-fpm" "mono-hang-watchdog" "mono-hang-watchdog32" "mono-hang-watchdog64" "mono-heapviz" "mono-package-runtime" "mono-service" "mono-service2" "mono-sgen" "mono-sgen32" "mono-sgen64" "mono-shlib-cop" "mono-symbolicate" "mono-test-install" "mono-xmltool" "mono32" "mono64" "monodis" "monodis32" "monodis64" "monodocer" "monodocs2html" "monodocs2slashdoc" "monolinker" "monop" "monop2" "mozroots" "mprof-report" "mprof-report32" "mprof-report64" "msbuild" "msgattrib" "msgcat" "msgcmp" "msgcomm" "msgconv" "msgen" "msgexec" "msgfilter" "msgfmt" "msggrep" "msginit" "msgmerge" "msgunfmt" "msguniq" "ngettext" "nuget" "opt" "opt64" "pal2rgb" "pango-querymodules" "pango-view" "pdb2mdb" "pedump" "pedump32" "pedump64" "permview" "peverify" "pkg-config" "ppm2tiff" "raw2gif" "raw2tiff" "rdjpgcom" "recode-sr-latin" "resgen" "resgen2" "rgb2gif" "rsvg-convert" "secutil" "setreg" "sgen" "sgen-grep-binprot" "sgen-grep-binprot32" "sgen-grep-binprot64" "shim" "signcode" "sn" "soapsuds" "sqlite3" "sqlmetal" "sqlsharp" "svcutil" "text2gif" "tiff2bw" "tiff2pdf" "tiff2ps" "tiff2rgba" "tiffcmp" "tiffcp" "tiffcrop" "tiffdither" "tiffdump" "tiffgt" "tiffinfo" "tiffmedian" "tiffset" "tiffsplit" "vbc" "vbnc" "vbnc2" "wrjpgcom" "wsdl" "wsdl2" "xbuild" "xgettext" "xml2-config" "xmlcatalog" "xmllint" "xmlwf" "xsd" "xsp" "xsp2" "xsp4"];
 
 mac-linux-mono = let
   linux-mono = pkgs.mono;
   P = " mkdir -pv $out/bin/ ; ";
   L = lib.lists.forEach mono-cmds (x: ''  
     makeWrapper "$src/${x}" "$out/bin/${x}" --add-flags "$@"     
   '');
 in if stdenv.isDarwin then (stdenv.mkDerivation rec {
   name = "mono";
   src = "/Library/Frameworks/Mono.framework/Commands";
   buildInputs = [ makeWrapper ];
   dontUnpack = true;
   installPhase = lib.concatStrings ([ P ] ++ L);    
 }) else linux-mono;

For the above reasons, (there are too many executable files or there are times when you want to configure symbolic links of pre-installed packages by registering only binaries.)
.
In the flake method, it will be more difficult because of the IO problem…,
IFD of the old nix method? by
Also, I wish there was a way to do it automatically (without even to create a list in advance). (Isn’t there an answer other than to return the monitoring/file creation runtime program that accesses the same folder as /etc/ and interact with nix?)
.
It would be great if nix could go beyond dsl and interact with any language. (Then, beyond the reward of learning nix, it will revolutionize the computing industry? haha)