Why hate?
If it is the way to boost the number of NixOS guides from less than a dozen to “countless”, it is worth to try. And then fix the purity.
4 Likes
Once something is supported it is very hard to revert.
I find some of the “pain” in Nix packaging to be very helpful to projects themselves as it helps find impurities in them. Most projects welcome the feedback. I also find a common correlation between project maturity and ease of Nix integration.
Here is a typical interaction with a project:
https://github.com/bash-my-aws/bash-my-aws/pull/247
4 Likes
I was thinking of nix-shell: Bus error (core dumped) · Issue #3597 · NixOS/nix · GitHub for example. Using np.memap in a nix-shell or buildFHSUserEnv triggers a Bus error, but works fine in a python interpreter (using the same nixpkgs for all).
3 Likes
The custom buildFHSUserEnv chroot script should probably be replaced with something that’s supported by the industry. There are plenty of options out there like firejail that would do a better job and have been battle-tested.
6 Likes
I agree with @danieldk that there a bigger obstacles than not having /bin/bash. If a package is generated by a derivation patchShebangs already solves the problem. For binary packages that can not be packaged in a conventional nix fashion we use buildFHSUserEnv. Can someone maybe give an example, which does not fall in the two categories mentioned above, where the lack of /bin/bash is a problem (or at least a big annoyance)?
I would argue that what’s exposed in / should be consistent between NixOS and the build environment, which already isn’t the case. Adding more variations only makes these kind of build issues more confusing and difficult to reproduce.
- nixos
/bin/shand/usr/bin/env - builds
/bin/shand no env
A great example of the kind of issues that implicit impurities can introduce git: 2.16.2 -> 2.17.0 by layus · Pull Request #38636 · NixOS/nixpkgs · GitHub.
5 Likes
There’s certainly an argument to be made that issues/PRs replacing shebangs or asking people not to use /bin/bash are causing somewhat of an effect outside of NixOS and perhaps it’s up to this community to try and alleviate that, as I fear that it might give NixOS some unpopularity.
1 Like
I’m currently on day 3 of trying to figure out what is wrong with my build. It clearly has to do with the shebang thing, but in spite of patchShebang’ing all over the place, I still get “bad interpreter”.
Somewhere in the list of scripts both static and generated something is not good.
I don’t think one can anticipate all the possible uses and don’t see why one should.
/bin/bash is the most minimal expectation.
/bin/bash is still impure.
if you really want a “quick and dirty”, you can always do:
nix-shell -p steam-run --run "steam-run <program>"
4 Likes
I don’t want quick and dirty. I just want a minimum of unsolvable puzzles.
Like:
[ 85%] Generating asn1.cert.pem, coordinates.bin, ec_cert_with_ext.pem, ec_cert_crl_distribution.pem, intermediate.crl.der, intermediate.cert.pem, intermediate.ec.cert.pem, intermediate2.cert.pem, leaf.key.pem, leaf.cert.pem, leaf.ec.cert.pem, leaf.public.key.pem, leaf_modulus.hex, leaf2.cert.pem, root.crl.der, root.cert.pem, root.ec.cert.pem, root.ec.key.pem, root.ec.public.key.pem, root2.cert.pem, self_signed.cert.der, test_ec_signature, test_rsa_signature, time.txt
cd /nix/store/78rvpsgfp9g92sl30620ywr4239mwwpa-openenclave-sdk/tests/crypto/data && /nix/store/k8p54jg8ipvnfz435mayf5bnqhw4qqap-bash-4.4-p23/bin/bash -c /nix/store/myjxqm5qbx9gsq6spvcy73hac5rwj89w-source/tests/crypto/data/make-test-certs\ /nix/store/myjxqm5qbx9gsq6spvcy73hac5rwj89w-source/tests/crypto/data\ /nix/store/78rvpsgfp9g92sl30620ywr4239mwwpa-openenclave-sdk/tests/crypto/data\ –bash
/nix/store/k8p54jg8ipvnfz435mayf5bnqhw4qqap-bash-4.4-p23/bin/bash: /nix/store/myjxqm5qbx9gsq6spvcy73hac5rwj89w-source/tests/crypto/data/make-test-certs: /nix/store/6h0xx5d8wwl8zf7g4x2sjl0q4hr3nnb4-bash-interactive-4.4-p23/bin/bash: bad interpreter: No such file or directory
make[2]: *** [tests/crypto/data/CMakeFiles/crypto_test_data.dir/build.make:103: tests/crypto/data/asn1.cert.pem] Error 126
make[2]: Leaving directory ‘/nix/store/78rvpsgfp9g92sl30620ywr4239mwwpa-openenclave-sdk’
make[1]: *** [CMakeFiles/Makefile2:19204: tests/crypto/data/CMakeFiles/crypto_test_data.dir/all] Error 2
make[1]: Leaving directory ‘/nix/store/78rvpsgfp9g92sl30620ywr4239mwwpa-openenclave-sdk’
This expression works fine in nix-shell --pure but something is wrong in nix-build…
I have more details in the other post (sudden EOF)…
I’ve tried adding patchShebang everywhere but no luck.
Development:
I removed the shebang altogether and it built. Apparently patchShebangs gets confused?
--pure makes the environment pure but does not chroot your shell (does not run it in a sandbox without file system access). If the project tries to use impure paths on the file system instead of relying on PATH, it will be able to access them. It is a compromise between fully hermetic build environment (like the one inside nix-build) and fully impure one with access to the environment variables set by your system.
2 Likes
I just ran into this while trying to use emacs org babel mode with tramp; i.e. interactive “notebook style” computing.
At first it looked like a silent failure, but I eventually found env: ‘/bin/bash’: No such file or directory in a nearby buffer.
I note this because I see plenty of discussion of interactions within a machine but not interactions between machines.
1 Like
maybe related, but what’s the proper solution for things like PHP which executes /bin/sh like here https://github.com/php/php-src/blob/9bbeb0555b6b842ebd44e08510ff3f3226237544/ext/standard/proc_open.c#L1210-L1215
Patching the source file.
3 Likes
This is an alternative that can add arbitrary programs to /usr/bin and /bin while not hurting purity: GitHub - Mic92/envfs: Fuse filesystem that returns symlinks to executables based on the PATH of the requesting process.
10 Likes
envfs looks really interesting.
Is it feasible to use a mechanism like that inside the build sandbox while also preserving purity there?
As I understand it that could work for build-time-only dependencies, which could be great to make things like build scripts run without tedious patching.
I would imagine for that use case it could be better to provide an explicit whitelist of paths that should be resolved from the derivation’s nativeBuildInputs than to magically resolve everything. For example:
envfsResovle = [ "/bin/bash", "/usr/bin/env" ];
@Mic92 cntr/breakpointHook uses FUSE inside the sandbox as well, right?
It would be probably easier to add them via the sandboxPath option:
nix.sandboxPaths = ["/opt" "/usr"];
I’m a bit late, but couldn’t /bin/bash just forward to the bash found in $PATH? Is a sandbox really needed for purity?
2 Likes
Actually, since in many cases the upstream change we would require would be to replace #!/bin/bash with #!/usr/bin/env bash, would there be any problems with simply putting this as /bin/bash?
#!/bin/sh
/usr/bin/env bash $@
It’s true that doing this is impure, but only as much as the #!/usr/bin/env bash hashbang would be in the first place. I’m not overly familiar with the details and gotchas with shells and hashbangs, so I’m not sure if there are more problems I can’t foresee.
7 Likes