Is it possible to specify a path where multiple derivations can read/write, a cache, inside nix/store?
Scons is a build tool and uses CacheDir=<path>. Since my code changes are minor, cache cuts down build times exponentially. I use nix mkDerivation and give CacheDir=./cache. So, cache is created inside nix/store/<hash>/cache
However when I change the source code <hash> changes and I get a different nix/store/<hash>/cache dir. I can try to specify previous build as dependency and copy its cache to current build dir before compilation but this not only duplicates the cache but also I have to bookkeep previousHash.
Any ideas?
This would be a violation of hermeticity and nix’s sandboxing. You can do it, with extra-sandbox-paths in /etc/nix/nix.conf (the nix.settings option in configuration.nix), but proceed at your own risk. If your cache isn’t really well designed to consider all input parameters, you will end up with inexplicably broken builds. For instance, you can use this kind of of trick with ccache, but ccache takes extreme care to make sure it’s using exactly the same compiler, flags, etc. when it uses a cached product.
3 Likes
After a couple of hours of research, I have found out that scons caching is really primitive and not suitable for nix.
A possible scons cache for nix requires hashing all nix derivation inputs without src except the Sconscript files inside the src and using the hash as a parent directory to CacheDir. So, if any changes, the parent directory of the cache will be different.
Another problem is If src changes, its path also changes in Nix. Scons is using the individual file paths as an input to hash build outputs. I think it is safe to remove this from scons hashing algorithm since file contents are hashed together with all other Nix guarantees.
I think I can achieve all by just patching their cache signature function.
I will give this a go if I can find time.
2 Likes