I’m trying to hack on python3Packages.torch to hopefully get MPS working on aarch64-darwin (issue ).
When I build with nix develop -i .#python310Packages.torch, I can see in the build log that there are issues due to impurity; specifically the PATH contains /usr/bin, which means the xcrun commands return a path to the system’s SDK as opposed to frameworks in nixpkgs#darwin.apple_sdk.
Seeing this post on the same topic, I tried building with nix develop -i .#python310Packages.torch --command bash --norc, followed by source $stdenv/setup and genericBuild.
However, building in this way I get a failed build with a ton of errors about missing protobuf header paths, for example:
fatal error: 'google/protobuf/any.h' file not found
#include <google/protobuf/any.h>
The files in question are present:
$ fd any.h
source/third_party/protobuf/src/google/protobuf/any.h
What about running with --norc might prevent these header files from being found, wheres they are found when I run without --norc? Does that suggest that there’s another source of impurity in this derivation?
Perhaps this is a darwin specific issue?
Thanks for any ideas.
1 Like
–norc is clobbering the mechanism nix is using to initialize the shell environment.
if (!found)
throw Error("package 'nixpkgs#bashInteractive' does not provide a 'bin/bash'");
} catch (Error &) {
ignoreException();
}
// If running a phase or single command, don't want an interactive shell running after
// Ctrl-C, so don't pass --rcfile
auto args = phase || !command.empty() ? Strings{std::string(baseNameOf(shell)), rcFilePath}
: Strings{std::string(baseNameOf(shell)), "--rcfile", rcFilePath};
// Need to chdir since phases assume in flake directory
if (phase) {
// chdir if installable is a flake of type git+file or path
auto installableFlake = installable.dynamic_pointer_cast<InstallableFlake>();
if (installableFlake) {
auto sourcePath = installableFlake->getLockedFlake()->flake.resolvedRef.input.getSourcePath();
if (sourcePath) {
if (chdir(sourcePath->c_str()) == -1) {
throw SysError("chdir to '%s' failed", *sourcePath);
I actually was digging through the source and saw that, so I manually put in something like ... command bash --norc -c 'source /path/to/my/bashrc; source $stdenv/setup; genericBuild as an experiment, and it still failed.
Currently AFK though, and I wasn’t reading that carefully through the source, so I’ll have to try again.
Is there a recommended way to get a clean PATH then? It complains if I try to use -i --unset PATH.
Currently using some awk silliness to manually remove all paths that are not /nix/*.
I’m not as familiar with the new cli and am not at a system to check, but I think the -i/–ignore-environment flag does this?
No, unfortunately PATH is preserved even with -i, see the comment from @edolstra that I linked above.
nix-shell has a --pure flag… that could be an option?
That’s what I get for naively trusting the option’s description.
I did find this partial ~explanation of why, though I’m not sure how authoritative it is: https://github.com/NixOS/nix/issues/6439#issuecomment-1107619941
It looks like --unset PATH works without -i/--ignore-environment, so I suppose it’s feasible if tedious to knock out other variables that way.
$ nix develop --unset PATH github:nixos/nixpkgs#jq
$ echo $PATH
/nix/store/dllvnqsylfc63sb7rzmcqsvak83rkkvm-autoconf-2.71/bin:/nix/store/q55wq9hk75hcdv9jvyzz1fv21x7agw7g-automake-1.16.5/bin:/nix/store/zbg0cm06br4j0li6qxhkp5y0c87c9yvw-gettext-0.21.1/bin:/nix/store/h38ndsn90fjp5qcp1bhyc41lrz4ml651-libtool-2.4.7/bin:/nix/store/1f54fk70vk76gj2zcg1gsd2asigjb3g0-gnum4-1.4.19/bin:/nix/store/s3x10q1z697dvpcd2l2cw9a21j71xaa2-file-5.44/bin:/nix/store/yypjyqzyd4j6knphklrqq75vn80x831v-bison-3.8.2/bin:/nix/store/21apr3mbmghnkd6zy2axxli2s36js3n8-clang-wrapper-11.1.0/bin:/nix/store/j979im1nim264y14d6k26dlbiwrnpijs-clang-11.1.0/bin:/nix/store/jfkxszd5mclhbqygx9lm1jy5gq2a5rw6-coreutils-9.3/bin:/nix/store/f0is0g9aafkjd7vyhzia7axcp45rcv9g-cctools-binutils-darwin-wrapper-11.1.0-973.0.1/bin:/nix/store/wnj5203979qy5i69zr9jznc6mncybpap-cctools-binutils-darwin-11.1.0-973.0.1/bin:/nix/store/970gpxq6ig7zk3pc34c8hfvj9gjmfq6f-oniguruma-6.9.8-dev/bin:/nix/store/jfkxszd5mclhbqygx9lm1jy5gq2a5rw6-coreutils-9.3/bin:/nix/store/hkjwkyvspdyz5v6bbdmzpgkqy38sk3id-findutils-4.9.0/bin:/nix/store/fac5y9gsaij3vsywyix99ni2bidbdvbl-diffutils-3.10/bin:/nix/store/0915q6jj8wdym586zjfijhrw3aqflibf-gnused-4.9/bin:/nix/store/qpk9fb3wn6qkvjpnznjkcx0hb7lpm41w-gnugrep-3.11/bin:/nix/store/m223lvwiz7v47djkm861524r350lwj71-gawk-5.2.2/bin:/nix/store/6baws0dbshlirv7dqqkmcrc7jpnxqri6-gnutar-1.35/bin:/nix/store/cwgy31xxhnxgf4hjahas02xa1bn1ji36-gzip-1.12/bin:/nix/store/hlmv5j1kwx737d6p4c8pblqbfr6f5yck-bzip2-1.0.8-bin/bin:/nix/store/ac9577gx85az323gjws1n52ar7w4b3x1-gnumake-4.4.1/bin:/nix/store/vvga016yc68k1y6n08qym93l5zna305q-bash-5.2-p15/bin:/nix/store/mmnkwyfc8rhddv6pqvmcmm2yxg54pq2p-patch-2.7.6/bin:/nix/store/skfb9v88g69mad3x5jvsj4vlrjv7i3g7-xz-5.4.3-bin/bin:/nix/store/s3x10q1z697dvpcd2l2cw9a21j71xaa2-file-5.44/bin:/no-such-path
In theory you should be able to clear the environment with env, but I’m having a little trouble making it work.
For example, this is basically the equivalent of --unset:
env -u PATH $(type -p nix) develop github:nixos/nixpkgs#jq
Clearing the whole environment, with env is just as easy:
env -i $(type -p nix) develop github:nixos/nixpkgs#jq
When I try this, however, nix uses its ~normal PATH again. I wonder if something else this is clearing out might be causing the dev shell to initialize differently.
I’m hoping to find a way with nix develop; I’m a relative newcomer and have used flakes, empty channels, and new-style commands since day one, so I don’t really grok the -A attribute '<nixpkgs>' way of calling things.
1 Like
FWIW, I’ve just been running this after entering the nix develop -i shell, just seems like a bit of a hack!
export PATH=$(awk -v RS=: -v ORS=: '$0 ~ /^\/nix\/.*/' <<<"$PATH")
I’m not sure the env -i business is working as expected, I still get a bunch of extraneous paths (scroll down in the codeblock, extra paths at the bottom of output):
$ env -i $(type -p nix) develop -i ~/git/nixpkgs#python310Packages.torch
$ printf "${PATH//:/'\n'}\n"
/nix/store/bq1q4gk52gsx4fg4pf07f2kxqgazkcls-python3-3.10.12/bin
/nix/store/gdh5vg1j8b4qmri26hzl520asq9j3h8a-python3.10-wheel-0.38.4/bin
/nix/store/wsxwsff0ldm2dig8cib9snw5qrrjby6b-python3.10-pip-23.0.1/bin
/nix/store/08nrfzzy1jamc21qj6pfrcn0q2h334bl-cmake-3.26.4/bin
/nix/store/vg7kzjmq32781bzdgqbjzb3c28v1f5zd-ps-adv_cmds-119/bin
/nix/store/9c2h25wb5qf83w0hn2f5mk87vvp1z02i-which-2.21/bin
/nix/store/wb5ib13rsyrcjkpmxwgmpimqplq2v20p-ninja-1.11.1/bin
/nix/store/yqp51kbjl2a2xx134j5wira49x95qpjq-python3.10-pybind11-2.10.4/bin
/nix/store/1r5zq1gy9y1vk5sygvnfk37r2yjijpa9-remove-references-to/bin
/nix/store/3lp8dbkhk6gq3zkj8ny8myxgvrs2s40m-protobuf-3.21.12/bin
/nix/store/y432q34aj5h9qn5qzm4w066j5l5dhmd2-clang-wrapper-11.1.0/bin
/nix/store/gcbmg7a0fhdcda42g7qp7ngqcix7bax5-clang-11.1.0/bin
/nix/store/xyagncilqx57cljac32w9ld3kkn276d3-coreutils-9.3/bin
/nix/store/qridnykybxbk74v5akvbpmyz66rhkxzk-cctools-binutils-darwin-wrapper-11.1.0-973.0.1/bin
/nix/store/0i2sn40fp7mncwzah2mryjkdy3wx4z8r-cctools-binutils-darwin-11.1.0-973.0.1/bin
/nix/store/xyagncilqx57cljac32w9ld3kkn276d3-coreutils-9.3/bin
/nix/store/9fy9qpcgdvg407pmkhfmh9afpw88vzxq-findutils-4.9.0/bin
/nix/store/qvl0kczzl4s2379bw2v4kp80zi5yhqc9-diffutils-3.10/bin
/nix/store/bcwjrxn6zl1pbxqkif4gifwv29msr5yf-gnused-4.9/bin
/nix/store/y6gkihd76zvaby3givk27s5mmbr027s1-gnugrep-3.11/bin
/nix/store/f6lixwd5dn5lynd759ff50r9zy71ndsh-gawk-5.2.2/bin
/nix/store/dz9j69sf1fgf7z2p4pbssf9agbi9b06z-gnutar-1.34/bin
/nix/store/6k108njfvixbyj3yaxpk6051jpla4y2s-gzip-1.12/bin
/nix/store/1dcf69y0bbmnzbrmvhn7kmwlss5hjbk7-bzip2-1.0.8-bin/bin
/nix/store/p68hd006asnl478cwvqls77wkn9dqi9k-gnumake-4.4.1/bin
/nix/store/0hhqhyxj34z4bjzy8r37zl05nqjlvjrm-bash-5.2-p15/bin
/nix/store/a7iv26ilaalw8qd1zh6wmgcm2b1mhqxy-patch-2.7.6/bin
/nix/store/dc9z0p1r7ifi73kjk4wn4jivdax43q3z-xz-5.4.3-bin/bin
/nix/store/k3i8aqcj4izfwn701i68xnclgz1hdz4x-file-5.44/bin
/.cargo/bin
/go/bin
/.rbenv/shims
/opt/homebrew/sbin
/opt/homebrew/bin
/.nix-profile/bin
/etc/profiles/per-user//bin
/run/current-system/sw/bin
/nix/var/nix/profiles/default/bin
/usr/local/bin
/usr/bin
/usr/sbin
/bin
/sbin
/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources
Neither does the --norc --noprofile trick from the post you linked (scroll down in the codeblock, extra paths at the bottom of output):
(HEAD)$ env -i bash --norc --noprofile
bash-3.2$ # PS1 loses color and my git branch, indicating the loss of env
bash-3.2$ /run/current-system/sw/bin/nix develop -i .#python310Packages.torch
Executing setuptoolsShellHook
Finished executing setuptoolsShellHook
(HEAD)$ # prompt color is back, this seems to be back in my usual environment
(HEAD)$ printf "${PATH//:/'\n'}\n"
/nix/store/bq1q4gk52gsx4fg4pf07f2kxqgazkcls-python3-3.10.12/bin
/nix/store/gdh5vg1j8b4qmri26hzl520asq9j3h8a-python3.10-wheel-0.38.4/bin
/nix/store/wsxwsff0ldm2dig8cib9snw5qrrjby6b-python3.10-pip-23.0.1/bin
/nix/store/08nrfzzy1jamc21qj6pfrcn0q2h334bl-cmake-3.26.4/bin
/nix/store/vg7kzjmq32781bzdgqbjzb3c28v1f5zd-ps-adv_cmds-119/bin
/nix/store/9c2h25wb5qf83w0hn2f5mk87vvp1z02i-which-2.21/bin
/nix/store/wb5ib13rsyrcjkpmxwgmpimqplq2v20p-ninja-1.11.1/bin
/nix/store/yqp51kbjl2a2xx134j5wira49x95qpjq-python3.10-pybind11-2.10.4/bin
/nix/store/1r5zq1gy9y1vk5sygvnfk37r2yjijpa9-remove-references-to/bin
/nix/store/3lp8dbkhk6gq3zkj8ny8myxgvrs2s40m-protobuf-3.21.12/bin
/nix/store/y432q34aj5h9qn5qzm4w066j5l5dhmd2-clang-wrapper-11.1.0/bin
/nix/store/gcbmg7a0fhdcda42g7qp7ngqcix7bax5-clang-11.1.0/bin
/nix/store/xyagncilqx57cljac32w9ld3kkn276d3-coreutils-9.3/bin
/nix/store/qridnykybxbk74v5akvbpmyz66rhkxzk-cctools-binutils-darwin-wrapper-11.1.0-973.0.1/bin
/nix/store/0i2sn40fp7mncwzah2mryjkdy3wx4z8r-cctools-binutils-darwin-11.1.0-973.0.1/bin
/nix/store/xyagncilqx57cljac32w9ld3kkn276d3-coreutils-9.3/bin
/nix/store/9fy9qpcgdvg407pmkhfmh9afpw88vzxq-findutils-4.9.0/bin
/nix/store/qvl0kczzl4s2379bw2v4kp80zi5yhqc9-diffutils-3.10/bin
/nix/store/bcwjrxn6zl1pbxqkif4gifwv29msr5yf-gnused-4.9/bin
/nix/store/y6gkihd76zvaby3givk27s5mmbr027s1-gnugrep-3.11/bin
/nix/store/f6lixwd5dn5lynd759ff50r9zy71ndsh-gawk-5.2.2/bin
/nix/store/dz9j69sf1fgf7z2p4pbssf9agbi9b06z-gnutar-1.34/bin
/nix/store/6k108njfvixbyj3yaxpk6051jpla4y2s-gzip-1.12/bin
/nix/store/1dcf69y0bbmnzbrmvhn7kmwlss5hjbk7-bzip2-1.0.8-bin/bin
/nix/store/p68hd006asnl478cwvqls77wkn9dqi9k-gnumake-4.4.1/bin
/nix/store/0hhqhyxj34z4bjzy8r37zl05nqjlvjrm-bash-5.2-p15/bin
/nix/store/a7iv26ilaalw8qd1zh6wmgcm2b1mhqxy-patch-2.7.6/bin
/nix/store/dc9z0p1r7ifi73kjk4wn4jivdax43q3z-xz-5.4.3-bin/bin
/nix/store/k3i8aqcj4izfwn701i68xnclgz1hdz4x-file-5.44/bin
/.cargo/bin
/go/bin
/.rbenv/shims
/opt/homebrew/sbin
/opt/homebrew/bin
/.nix-profile/bin
/etc/profiles/per-user//bin
/run/current-system/sw/bin
/nix/var/nix/profiles/default/bin
/usr/local/bin
/usr/bin
/usr/sbin
/bin
/sbin
/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources
Correct. It ~should be capable of working, but I saw the same. The fact that clearing just the PATH works at this level is why I wonder if something else thrown out by this causes either nix develop or the shell init inside to run differently. (Perhaps some sanity check that causes it to fall back to defaults?)
samuela
August 19, 2023, 10:22pm
11
This is such a common need. We should really have a flag for it!