Ccache does not work with c/c++ compiles - /homeless-shelter error message

I have been trying to use ‘ccache’ under NixOS. I have successfully used it on Debian Linux and it reduces the compile time from an hour to less than five minutes for my projects. I have set the following options in configuration.nix:

   nix =
   {
      maxJobs = 2;
   
      sandboxPaths = [
                        (toString config.programs.ccache.cacheDir)
                     ];

      nixPath =
         options.nix.nixPath.default ++
            [ 
               "nixpkgs-overlays=/etc/nixos/overlays/"
            ];
   };

and have confirmed that this adds ‘sandbox paths’:

build-users-group = nixbld
max-jobs = 2
cores = 0
sandbox = true
extra-sandbox-paths = /nix/var/cache/ccache
substituters = https://cache.nixos.org/
trusted-substituters = 
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
auto-optimise-store = false
require-sigs = true
trusted-users = root
allowed-users = *
builders =

system-features = nixos-test benchmark big-parallel kvm
sandbox-fallback = false

to /etc/nix/nix.conf. I have also added:

programs.ccache =
      {
         enable = true; 

         cacheDir = "/nix/var/cache/ccache"; 

         packageNames =
         [
            "iup-cd"

            "iup-ftgl"

            "iup-im"

            "iup-iup"

            "iup-pdflib7"
         ];
      };

to configuration.nix. Then I changed my derivation so that stdenv was replaced with ccacheStdenv and when I run the derivation, the following error is displayed:

/nix/store/5b62ia6vj72nf1mmdlfbxawpb9hjr8w9-stdenv-linux/setup: line 1512: /homeless-shelter/.cache/ccache/tmp:: No such file or directory
builder for '/nix/store/bhh4cskzdc3rz5vxlq5qg4jm836dx9pb-iup-im.drv' failed with exit code 127
cannot build derivation '/nix/store/dlsqaxhv21665yzr9m3cm3s6cyxfhz9a-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/k5g1x9a2811d0lskz7gyw0yhlk3mcmrc-nixos-system-Delbert-21.11.337514.4c560cc7ee5.drv': 1 dependencies couldn't be built
error: build of '/nix/store/k5g1x9a2811d0lskz7gyw0yhlk3mcmrc-nixos-system-Delbert-21.11.337514.4c560cc7ee5.drv' failed

Is ccache for NixOS broken or are there more configuration changes that I need to make.

ccache is trying to reference $HOME, which doesn’t exist for build sandboxes.

You can try to set some environment variable to a path in the sandbox for the cache to use. If that doesn’t work a common idiom is export HOME="$TMP" in an early phase, I usually add it to postUnpack in derivations.

2 Likes

@Growpotkin - thank you for your reply and explanation. I have found that adding the following code:

  export CCACHE_COMPRESS=1

  export CCACHE_DIR=(toString config.programs.ccache.cacheDir)

  export CCACHE_UMASK=007

  export HOME="$CCACHE_DIR"     #"(toString config.programs.ccache.cacheDir)"

to the pre configure phase allows my derivations to compile the cross platform GUI kit that all of my applications use. Is there any chance that the owner of the ccache package could add some more documentation (or the wiki entry be updated) so that others may benefit from this ?

1 Like

I have no idea who maintains ccache, but I’m a strong proponent of “be the change you wanna see my dude!”

I’d recommend popping up a quick fork of the repo, add a section to the ccache README, and send it in as a PR. Put the snippet you just sent me in the README with a heading like “Troubleshooting HOME references” with generalized “problem” and “solution” paragraphs based on the contents of this thread.

Odds are whoever manages ccache will tweak a few lines or add a comment or two, so don’t sweat making it perfect, but sending a PR is the simplest way to get the maintainer’s attention. I assure you that they’ll appreciate your contribution, and you’ll save others from a similar headache.

While this might sound like a lot of effort, the truth is that GitHub’s “edit file” interface makes this pretty painless - it’ll probably take you longer to read this message than it will to make your contribution :smiley: