@fzakaria Would there be any gain enabling these tools inside the Nix build sandbox? It wouldn’t affect reproducibility, right? Because everything that the tools make available is still constrained by the derivation’s inputs.
We’re calling this setup Mic92OS 
For new users, this should probably be the default. It reduces a lot of the friction developers are getting when switching to NixOS. For example when pulling npm packages with pre-built binaries in them.
at what point should new users turn this feature off? when they have drank enough nix kool-aid that self-flagellation is an appealing prospect?
in all seriousness - i am asking you if you think we’ve got it “wrong” with nixos and the /run/current-system pattern was a bad idea? should we instead be linking software into /bin, /lib, etc… instead?
curious to hear any and all opinions on this 
Hell no. I would much rather have the actual sanity of completely avoiding this kind of global state and all the problems that come with it. If people don’t see the value in that, Nix clearly isn’t for them, and NixOS shouldn’t turn away from that philosophy to appease them. It’s a false promise.
Maybe not default in the sense of modules, but we’d do well to advertise these options in every “quick start” guide, perhaps include them (even if set to false) in nixos-generate-config
I have encountered quite a few things that break with a different un-understandable error messages that refer to nix-ld only my colleagues setup, with the only advice I was able to give to deactivate it.
Once deactivated and many many cached directories cleared later, you can once again encounter correct library not found errors and work on packaging successfully
That’s an interesting article, thanks!
In fact, I was fighting a related issue at work where we have plenty virtualenvs running on NixOS VMs: Managing virtualenvs with dynamic libraries on NixOS (Part 1)
should we instead be linking software into /bin, /lib, etc… instead?
I’m not sure if this is desirable: glibc falls back to the system dirs if it can’t find shared libraries in the rpath/runpath sections of the ELF header (or LD_LIBRARY_PATH).
This seems prone to be hiding packaging bugs in nixpkgs and having more “works on my machine” situations again.
I’m curious what @Mic92 thinks but I really like the FUSE idea;
Could nix-ld be replaced with a FUSE at /lib /lib64 instead of the custom ld.so ?
It’s not clear what’s the trade-off.
There’s also an active discussion about the syscall filtering done by envfs.
I’d like the default to allow all so that ls and stuff just works and you can easily navigate the tree.
(Either way @Mic92 has created so many useful tools in Nix ty!)
Correct; they are only seeing what’s on your PATH so it should be safe to do so;
We could symlink join add it to the mkDerivation –
FUSE is significantly slower.
I guess it’s fine for /bin but yea maybe libraries would be too much overhead?
If it’s only for “non-Nix” software though it might be an acceptable trade-off for the simplicity of using a “normal interpreter” and browsing the /lib
(anyways, just a thought)
Quite the opposite; it’s when using FHS feels like self-flagellation. Not being facetious here - If I wanted /lib (or /usr/lib or whatever), I’d still use Arch. I can’t tell you the endless headaches I encountered because of that very design, which meant every library update required a manually-initiated rebuild of the reverse dependencies. [1]
Correct deployments that nix-based systems (and by extension, NixOS) offer are so much more appealing. Even if I use, say, system.replaceDependencies, then I’m merely opting into headaches for only that limited list of packages for convenience’s sake.
And using /lib on NixOS means the software is simply packaged incorrectly, and will only work on that one system at one specific point in time - so even for my own sake, if I want to deploy said software on another system, I cannot guarantee it to work as I would hope.
I continue to observe in amazement why NixOS, the distro designed for correct deployments (i.e., write code once, deploy basically anywhere, modulo some hardware and networking config and secrets deployment), is constantly twisted to ensure incorrect deployments (write code, shrug and hope it works again?).
And any tooling that made that easier, like AUR helpers, were officially excoriated. Though, even that mess was heavenly compared to other distros’ packaging tools. ↩︎
Continuing the discussion because it’s Friday evening and I’m excited for PlanetNix 
Just to be clear – this is for cases where you’ve downloaded binaries prebuilt off the internet.
They are clearly not “nixified” so all bets are off; nix-ld & envfs just let you move forward in the meantime before getting it as an official package.
I think everyone who is using Nix, especially flakes, is falling for a large sense of security that doesn’t exist.
Searching on GitHub, the majority of flake users are doing nixpkgs.follows = which changes the versions of everything that particular flake was tested/built against.
At that moment, you’ve just gone off the rails and have done something like searching /lib.
tl;dr; Most people aren’t the purists they think are (especially if they use follows), why not be a little more pragmatic where it counts
No, because you’ve rebuilt the software against the correct library version.
Software off the internet is, well, not.
(Closed-source software is another beast entirely, and in that case, you probably don’t have any choice but to use hacks like patchelf or the ones mentioned in your post. But avoiding /lib means the remaining open-source software doesn’t suffer for it.)
I’d like to reiterate, my desire for correct (as much as possible) deployment is a pragmatic concern, not a purist one. I want my software to work so I can get on with my day, and writing code that may break at some unspecified time in the future (usually at the most inconvenient time) is not pragmatic to me.
Also worth mentioning: another approach to this problem is patchelf. An advantage of patchelf is you can still use it if you want to use nix on a host where you don’t have admin privileges and where user namespaces are disabled.
At that moment, you’ve just gone off the rails and have done something like searching
/lib
I disagree.
nixpkgs.follows =is a conscientious choice made on per-flake input, not globally for the whole system- In most cases, the reason for adding
followsis to reduce closure size, not to “make things work” which is a completely different reasoning. - If the flake you modified doesn’t work as expected, all you need to do is to remove the
follows, get bigger closure size, but pretty much guaranteed working software.
I can only imagine a nightmare of first enabling nix-ld and envfs on the system, run it for a few years, and then try to remove it later. I’d rather take a pain of fixing software gradually as it comes than all at once a few years down the line.
Also worth mentioning: another approach to this problem is patchelf. An advantage of patchelf is you can still use it if you want to use nix on a host where you don’t have admin privileges and where user namespaces are disabled.
That is how I tend to do it. Either patchelf or a wrapper script that sets the library paths (well, and patchelf to change the ld-loader). 95% of the work is anyways to find out which libraries are needed.
Would there be any gain enabling these tools inside the Nix build sandbox? It wouldn’t affect reproducibility, right? Because everything that the tools make available is still constrained by the derivation’s inputs.
Wouldn’t you run the risk then that running the results of the build also requires these paths to be available?
Custom ld.so has fewer dependencies – FUSE requires user namespaces enabled.
Oh this is really cool. Some NPM projects I work with ship a lot of prebuilt stuff, and devcontainers or FHS just seems a bit excessive (and breaks the 1password agent)
And this just magically lets it work!