I keep running into too many cases where binaries just don’t work. It quite a limitation. Although what Nix already does is really neat, it can be quite a hinderance for someone who wants to try a new OS and get to work.
Even after learning Nix, it’s a hassle to have to work around non-patched binaries, especially coming from workflows involving other package managers.
So this got me thinking: is it possible to make any binary magically work in NixOS? Some mechanism by which they can automatically be patched after being loaded into memory but before being executed?
What about some sort of kernel module or init system plugin that can attempt to do this, on the fly, whenever someone runs an executable? It’d be sort of like autoPatchelfHook, but at a low level that can patch binaries when you run them.
Imagine, you download a binary anywhere, then you run it, and it just works.
Something like Rpmsearch maps liibs to packages. Autorun ldd and autopatchelf using that Rpm search. Don’t know if it will work. There is something called nix search btw
Is it possible to hook into the Linux kernel (or something else) somehow, so that the same logic in nix-autobahn can be applied automatically to the binary being executed?
To install it for example, it would be something like
What about some sort of kernel module or init system plugin that can attempt to do this, on the fly, whenever someone runs an executable? It’d be sort of like autoPatchelfHook, but at a low level that can patch binaries when you run them.
Imagine, you download a binary anywhere, then you run it, and it just works.
What this wonderful system would do when you need a library that is not installed? Can you specify that the library is actually optional for the binary? Can you override the version used (a build with non-default build-time feature selection)?
Maybe better tooling to autodefine, amend, and debug FHS env construction is a better first step, as most systems with Nixpkgs kernels support unprivileged user namespace creation, and that approach would have less conflict with Nix approach in general.
The main reason why those app doesn’t work is because they refer to a static path that nix doesn’t use (for the purpose that software don’t have undeclared dependancies). This may be workable around with some symlink. Also, nixpkgs provide steam-run, that can run a majority of app into the FHS of steam.
You can use the steam-run tool (obviously designed to run steam) to run many binaries without any sort of patching. It uses user namespaces to create a chroot environment where the standard FHS is there. There’s also the buildFHSUserEnv function if you need to make a custom version of this with other libs or something.
Yeah echoing a few comments here already… but when I have a binary I don’t feel like putting any time and effort into the first thing I do is steam-run ./prog and it almost always just works.
Sure, I agree. It is fantastic in a pragmatic sense. Especially for newcomers. Let’s not kid ourselves into thinking nix has a low bar to entry, because it most certainly does not.
I installed NixOS having never used the nix package manager or knowingly using the nix language. I was sold on the idea of a reproducible OS managed declaratively by a single configuration before even I finished a (very) light reading through the manual. I ran though an install and configured my system in no time… never having the need to write a single package or do anything outside of enter values into nixos module options. I continued that way happily for about a year.
steam-run is the best friend for a NixOS user who has no interest in becoming a developer
You mean for a developer that has no interest in becoming (or maybe no time to become) a NixOS developer.
In my free time outside of my full-time dev job I am always writing code in my personal projects. I just can’t wait to get rid of the full time job.
With that said, I have so many issues in various projects now that I’m in NixOS. The amount of work needed to get to work is too high and on-going, required for each new project that might have limitations in NixOS.
I’ve been giving NixOS an honest try, and although I like the concept very much, I might have to switch to another distro just to get productive again so I can work on my own projects and not on wrangling binaries with Nix (despite being a good learning experience about Linux in general). I don’t regret the time I’ve spent these past few weeks, but now I need to get to work without continual problems popping in when I switch to various projects in NixOS for the first time).
Today I’m working on a project that runs testing with Karma and Electron, and although I did the trick of replacing electron with the one from nixpkgs and deleted the one in node_modules of the project, the environment still hangs on loading Electron for some reason. I wish I had the time to iron it out, but I don’t.
Maybe steam-run is my last chance. I’ll give that a try.
$ git fetch
/nix/store/g4jpqmsrw22ksv4j664sdlpps1zkmqda-openssh-7.9p1/bin/ssh: relocation error: /nix/store/j2q7fmj0b30yi2fydc10r58l3sjig6k8-libkrb5-1.17/lib/libcom_err.so.3: symbol k5_os_mutex_destroy version krb5support_0_MIT not defined in file libkrb5support.so.0 with link time reference
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
These little things are eating away from my productivity and I’m not sure yet if I am convinced that the benefit of Nix/NixOS outweighs, for example, the occasional manual intervention that would result from weekly invocation of pacman -Syu in Manjaro linux (or Arch).
I’m curious what about your workflow is making NixOS difficult. Is it your personal projects that aren’t working? Do you have a lot of different ones? What languages are they written in? What do you think is making you need to use a lot of unpatched binaries?
Doing nix-autobahn automatically like that seems a little bit surprising since it could end up downloading random libraries that you haven’t actually authorized. Binfmt might be able to do this though.
A better improvement that I have been thinking about is making an “LSB” module that would include all of the libraries listed in LSB (without truly being LSB compliant though). So users are at least no worse off than they are in a normal Debian or RedHat distro. Along with https://github.com/NixOS/nixpkgs/pull/69057, we can get pretty close to letting users opt-in to run their random binaries.
I wonder if we could have a custom ld-linux.so wrapper that would possibly suggest a simple FHS nix shell script on error, at least for dynamically libraries loaded at start (and perhaps a wrapper around dlopen()?)
Basically something like nix-autobahn but perhaps (a) don’t automatically do anything just print what nix shell expression you might use, and (b) useable as a dynamic linker.
Yep, that’s it. A bunch of Node/Electron/Web projects, each with differing requirements binary wise. In plain Linux I can work on them all fine without hiccups.
Mostly NPM packages that ship with binaries. electron is the big one. I got local electron working with steam-run, but then git command broke.
