Hey all,
I would like to escape some of the restrictions Nix puts on a package build. Namely;
- is it possible to switch to a different build directory from within the derivation? E.g. by a simple
cd $SOME_PLACE_I_HAVE_WRITE_ACCESS_TO or is this only possible with TMPDIR?
- in the
installPhase can I additionally copy outputs to another directory, that is not managed by Nix? Some arbitrary directory, that I have access to? Like /home/phillip/out?
I have tried a cd and copy to other directories with a nix-build --no-sandbox and a stdenvNoCC.mkDerivation but so far no luck.
Context:
I am doing physical simulations with numerical codes and usually one simulation step depends on some others. The chain of dependencies can be long and difficult. A calculation usually produces very large outputs distributed over many files (too big to put them in the store).
My idea is to use Nix to model the dependencies between the calculations, e.g. not building a software, but the output of a simulation program. I can put the files relevant for the next simulation step in the store, but would also like to keep the others on a different filesystem for large files. It also would be helpful if I could specify a scratch directory from within the derivation, that is not TMPDIR, but a fast parallel network filesystem.
Is this somehow possible with Nix and some hacks or am I bending the architecture too far here?
Best wishes
Phillip
In principle that’s probably possible with sandbox-paths. But then you need to be sure that you prevent conflicting writes and manage what needs and what needs not to be retained extra carefully.
For example, if you delete this extra output but leave the things in Nix store, Nix will see no reason to rebuild the large extra output.
I am putting together a similar thing and probably have some of the answers. It’s still very nascent. Care to chat about our approaches? My thought is that this use-case might become more viable with a better understanding how people would use it and put together a helper library or some docs/guide.
The sandbox-paths sound like a good first step. On a multi-user system those would be accessed by the nix-build users, correct? On a HPC cluster this would mean that if you make them available in /etc/nix/nix.conf, everyone could override others people calculation? Would it somehow be possible to keep the permissions of the user, that is invoking nix-build to only write to directory the user has permissions to write to?
@tomberek That sounds very interesting. I will send you a direct message 
@domenkozar That’s also an interesting approach. Looks like I could run something on my local machine but it is probably not the possible to do this on a non-NixOS cluster, that just runs multi-user Nix?
The sandbox-paths sound like a good first step. On a multi-user system those would be accessed by the nix-build users, correct?
Yes.
On a HPC cluster this would mean that if you make them available in /etc/nix/nix.conf, everyone could override others people calculation? Would it somehow be possible to keep the permissions of the user, that is invoking nix-build to only write to directory the user has permissions to write to?
That sounds hard; Nix tries not to take into account who asked for the build…