The FAQ mentions the difficulties of compiling software on NixOS:
I installed a library but my compiler is not finding it. …
I understand that the accessibility of libraries is intentionally limited when building packages, in order to achieve reproducible builds. However, why hide libraries in non-standard locations like that when the user is just building his own code (not creating a package)?
It’s for the same reason: to achieve reproducibility.
There’s never a moment when a library is the correct one for the environment, because different packages can use different versions of the same library, or the library may be compiled with different build inputs.
Nix provides us with environments for our convenience; /run/current-system/sw/bin/firefox points to the “current” Firefox browser. Libraries don’t get the same treatment, but Nix provides a way to create an isolated normal-looking environment which does include libraries.
As I understand it, reproducibility is a question of trust: if three devs build a package and it’s identical, as it should be, then no one is slipping malware into the binary.
However, if you are building your own code, and not distributing it to others, is reproducibility important enough to to make your life more difficult like that?
It’s not about reproducibility of the artifact, but about reproducibility of the build environment.
Create your shell.nix once, use it everywhere.
No nvm+rbenv getting out of sync anymore for a full stack project, just a single shell.nix managing both, and with a level of detail neither of the other 2 could do.
To me it looks like you have this upside down:
is reproducibility important enough to to make your life more difficult like that?
It’s not making life more difficult - quite to the contrary. IMHO, what is more difficult is when you can have only 1 global environment and everything has to work with that.
If you have your own application(s) that for whatever reason only works with an older version of a particular library - possibly with known vulnerabilities but you know that given how it’s used, those vulnerabilities are not an issue. Then you build against that until you have time to fix it properly.
Having to specify all your dependencies makes life so much easier.
As a side note: I remember reading that only a fraction of “known” vulnerabilities are declared as such and written up for the vulnerability databases – most are just fixed in the latest version, which is why “stable releases” are dangerous: An exploit author doesn’t even have to look for vulnerabilities.
What you are illustrating is that the trade-off is worth it in some cases. But there is a trade-off?
Note you also do not get optional dependencies that you only need in your other projects, which is good for reducing the surface.
I think a frequent practice is to have shell.nix and actually keep track if Nixpkgs updates until something breaks. This still allows you to pick from multiple available stable branches where applicable. And then you get to bisect without messing with your entire system. And of course not everything treats malicious input all the time, so you have an option to finish processing the by-now-known-non-malicious-although-garbage data before debugging the update.
What you are illustrating is that the trade-off is worth it in some cases. But there is a trade-off?
It depends on your perspective. What you see as a trade-off, I see as bonus.
Hmm, is pkg-config & co patched not to resolve hand-compiled libraries or something? Presumably the “standard” library paths in /usr aren’t resolved, so you’ll have to set your own prefix path, but I don’t think it’s fundamentally impossible to do this?
I believe the problem here is that the wiki FAQ reads as if nix was the only way to do anything, which might indeed get in the way of hacking on a local lib/application. I think this is not true (but I haven’t compiled anything without nix in a while)?
If you can just set $PREFIX and have it work the answer to this question is purely technical: doing it any other way would have a chance of polluting your running system with your hand-compiled library, especially for things like Python that are totally happy to resolve the first library they find.
I know you’re on your way out, but I’d like to answer this anyway since I think it could be of interest for others, and because I don’t think others here are likely to say something similar.
While many Nixers will not be interested (perhaps because Nixpkgs is now huge, perhaps because they’ve developed Nix packaging skills after longtime usage), I actually think this kind of one-way impurity could be valuable. But I think that for it to really make sense, we would need more long-lived NixOS/Nixpkgs releases, otherwise users would have to constantly rebuild their ‘foreign’ software after NixOS updates, as is the (unfortunate) custom on Arch Linux. I described what I have in mind here.
I agree with the consensus opinion that it’s best to actually use Nix to build whatever programs you need and integrate them properly. And I like that in the Nix world, there aren’t really ‘end users’ to the same degree as elsewhere; most end users end up positioned to make small contributions at some point. But I think that if we had releases that lasted a little longer (something that’s not on the table atm and shouldn’t be a high priority for individuals in the community, since it would require corporate resources), linking (or using some GoboLinux-y magic, if build tools might resolve symlinks all the way down to store paths in a problematic way) base NixOS components into some versioned path under /usr somewhere could be a good way to provide an experience that has more of an ‘on ramp’ like Nix on non-NixOS but that still allows comprehensive Nix-driven configuration for services like you get on NixOS.
Allowing a little bit of impurity in this way could be good for new users so they can learn to package things for Nix more gradually. For better and for worse, it would also allow NixOS to have true ‘end users’ who could enjoy using Nix as a simple configuration for the base system and easily install software outside its confines (and outside its guarantees 
) rather than having to learn large pieces of the whole system just to move forward with unpackaged software or whatever. That could prove difficult to support, and I’d hate to see NixOS fall into the same kind of twilight as Arch where there’s this thing a ton of users find necessary but must remain nominally unsupported because it runs against the grain of the base system’s design.