Set runCommand to do the build locally and never ask cache

Help
1

I folks,

I have a few functions that look like this:

  do_something =
    x:
    fromJSON (
    builtins.readFile (
      runCommand "do_something" {
        inherit x;
        allowSubstitutes = false;
        preferLocalBuild = true;
        }
            ''
            ...

And I use the function on many places so they produce a lot of derivations (files) in my Nix store.

And looking at terminal after running nixos-rebuild ... on a new system or after a lot of changes, I see a lot of do_something blinking. I think that means that Nix is asking the global Nix cache for derivation which obviously is not there since the shell script and input for the script is custom to my setup.

  1. I don’t want to waste Nix cache resources / bandwidth for stuff that clearly is not there.
  2. I think I can speed up rebuild process - especially since I have slow internet and bad latency.

Is there something I can do to just tell runCommand to NEVER ask cache for this?

Thank you.

2

Use runCommandLocal instead. That sets the flags the way you want.

Edit: That said, passing the flags manually should override them anyway.

3

Thank you @lilyball

I’m looking at the documentation and runCommandLocal sets allowSubstitutes to false (it doesn’t change preferLocalBuild.

So regardless which builder I use (runCommand with allowSubstitutes to false or runCommandLocal)
does it mean that this is never pulled from a cache? In other words the progress I see when nixos-rebuild doesn’t mean it’s asking cache?

4

It sets both allowSubstitutes and preferLocalBuilder

Are you sure it’s asking the global cache, and that you aren’t simply seeing it show up because it’s building the script?

5

FYI, what you’re doing is going to have a weird effect on evaluation. If you’re using that do_something function a lot, it’s going to look like it’s doing one do_something derivation, waiting for a tiny fraction of a second, then doing another, and so on in a rapid stop-and-go fashion. In case you’re mistaking this for it checking the cache, it’s not.

The problem is using builtins.readFile on the output of a derivation. This is an instance of what’s called “import from derivation” (IFD), and it means a derivation has to be built before evaluation of the nix expression can continue, because the contents of that output are part of the next evaluation. So it looks like very rapid stop-and-go, because it’s stopping to build one derivation, reading its output, evaluating some more, then stopping for another one, and so on.

6

To expand on that: evaluation is single-threaded, so IFD is effectively a blocking operation.

The wiki for more info: Import From Derivation - NixOS Wiki

7

No, I didn’t use strace or something like that.

That explains a lot. Thank you.

It means each time I use do_something. Nix has to build a derivation and store the output as a file in /nix/store. And if change something inside the runCommand, Nix has to create all those files again. And If I use the function 1000x (I could be mapping over it or something similar), Nix has to create 1000 new files.

Thank you all for the explanation!