I’m trying to build and contribute a Rust package called sequoia and I was wondering whether the process could be improved somehow. Most of the errors I get are from the installPhase which runs make PREFIX=$out install. There were several reasons for these errors but I’ve managed so far to update the package’s function’s arguments and I’m getting through them.
The problem is, that every nix-build -A sequoia ends up taking an hour or so while my CPU gets really hot during which and besides the fact I feel my computer is wearing out, It’s just taking too much time.
Back on my unpure Arch Linux installation, I’ve had sccache set up and naturally it has reduced the CPU usage and time consuming Rust compilations. The setup for this is pretty simple - export RUSTC_WRAPPER=sccache.
I intend to contribute a sccache package as well, but I’m not sure I’ll even know how to use this package as a compiler cache for Rust compilations of nix-build as well. Ideally, it should be enabled in /etc/nixos/configuration.nix just like the programs.ccache module.
I was wondering, perhaps someone could just give me an idea for where should I start? Would it be possible to make nix-build use sccache temporarlily at least until I’ll merge a module for it? Say sccache will be compiled and stored in /nix/store/, Would I be able to use it if I’ll just put RUSTC_WRAPPER = ${sccache}/bin/sccache; in my package’s input?
You’ll probably have to disable the nix sandbox or sscache won’t be able to use its cache.
The sandbox prevents filesystem and network access. https://nixos.org/nixos/options.html#sandbox
can’t you run it in a nix-shell and set out/PREFIX so that it installs outside the store ? it worked for me in the past with some programs (you need to define one variable per output for multioutput derivations)
Thanks a lot for your comments everyone. I’ve added a PR for the sccache package and I tried to use it in my nix-build tests with the following changes:
Setting RUSTC_WRAPPER and SCCACHE_DIR variables as environment variables in the package’s attributes.
Setting nix.useSandbox = false; in the global nix config because sccache uses a client-server architecture. Thanks for pointing that out @tokudan!
With the above, it seemed that my $SCCACHE_DIR got successfully used and got filled with some files by sccache. Unfortunately, after I changed a little bit the derivation, and I ran nix-build again, it didn’t seem as if the cache was actually used. it seems that this might be due to the lack of an equivalent CCACHE_BASEDIR feature for sccache which I should have used. There’s a bug open for that upstream.
Naturally, since I don’t use the sandbox in my nix-build, sccache gets a different path for every .rs file it compiles since the derivation’s build path is different (e.g /tmp/nix-build-sequoia-0.8.0.drv-1 vs /tmp/nix-build-sequoia-0.8.0.drv-2).
Would it be possible to overcome that? Perhaps it would have been possible if the sandbox had enabled networking capabilities for the programs running in it but I couldn’t find an evidence for such option…
I’m not sure I understand exactly what you mean @teto, I’m only a beginner NixOS user who’d like to contribute some packages back from my Arch Linux days .
Are you talking about cloning the Git repo, adding a default.nix file there and in a nix-shell running make PREFIX=${out} install? Would that actually create a ‘package’ that I’ll be able to use outside that nix-shell?
I’ve read about setupHook in the manual. It seems this is mainly useful for packages which may be dependencies of others, right? Are you suggesting that as for the implementation of a programs.sccache option, it will be possible to define a setup hook for all packages which use rustPlatform.buildRustPackage?
sry, you seemed too competent for a beginner
nix-build rebuilds from scratch and as you witnessed with some packages, it can be very frustrating. In those cases, you can debug your derivation in a nix-shell (instead of a nix-build) and run each phase (configure/build/install) manually. The good thing is you won’t have to restart from scratch.
so you do nix-shell -A sequoia ~/nixpkgs and then once in the shell run the code nix-build would. It can be tricky to find these commands, you better follow Nixpkgs/Create and debug packages - NixOS Wiki and ask on #IRC as I am not familiar with the rust phases and have to close my computer to take the plane
Wow thanks a lot! This sort of resolves the issues I’ve had with sccache as well I mentioned in my previous comment. It doesn’t solve issues which may arise only when building in the sandbox but it’s good enough I guess.
.