How to exclude derivation in a package closure from the binary cache?

Development
1

I’m trying to package a game in Nixpkgs with a large assets directory (~ 400MBs). Ideally, Hydra should build the game and cache it without including the assets: these can be fetched directly by the user, no need to put them on cache.nixos.org.

The problem is the game derivation has a reference to the assets derivation, so they’re part of the closure and will be cached by default. Is there a way to prevent this?

2

You could solve this using the wrapper pattern: Build the game without assets and then combine it with the assets in a separate drv (i.e. buildEnv { name = "veloren-wrapped"; paths = [ veloren assets ]; }

If the assets are free though, I wouldn’t bother. 400MB isn’t nothing but it’s not that much. In this case, hydra will have to cache the assets anyways by the looks of it because they’re in the src. (Also: Yuck, binaries in git…)

I guess you could filter them out after the fetchgit for src and have a second FOD for the assets. Using a FOD for assets should be preferred anyways since the assets are just data and FODs aren’t affected by rebuilds. (If you opt to simply not bother, symlinking assets/ to src would be the best option.)

2 Likes
3

(i.e. buildEnv { name = "veloren-wrapped"; paths = [ veloren assets ]; }

Interesting, I’ll see if I can make it work.

400MB isn’t nothing but it’s not that much

The problem is that the size has doubled in two years, it’s not unlikely they’ll grow more and start to cause trouble (is the 2GB limit still a thing?).

4

Maybe derivations marked with allowSubstitutes=false should be automatically excluded from cache.nixos.org.

Obviously they still have to be built by Hydra.

But since they are marked with allowSubstitutes=false, they will never be fetched from the binary cache (unless users go out of their way to override this with always-allow-substitutes, which AFAICT was only added very recently).

5

allowSubstitutes=false only means that users wouldn’t attempt to substitute but it’d still be in the cache.

That’d be the worst of both worlds; you’d take up storage space on cache.nixos.org and users wouldn’t be able to make use of it.

6

Correct. I am proposing that, in addition to that, these derivations should not be uploaded to the cache.nixos.org binary cache.

I am proposing that we change that.

I am very confused by this comment. I am specifically proposing that they not be uploaded to cache.nixos.org. If they aren’t uploaded, how would they take up space there?

7

Having them not be uploaded is not a functionality that exists (to my knowledge) or could easily exist as it would go counter to how the basic formats of Nix work. I have no idea how you forsee this to ever work with the already quite fragile remote builds.

8

Yes that’d be great. Regardless of the fact that this doesn’t actually help with the legalese, I’d love to have this feature when dealing with certain kinds of unfree software.

it would go counter to how the basic formats of Nix work. I have no idea how you forsee this to ever work with the already quite fragile remote builds.

I expect Nix would crumble, but it really shouldn’t: it has the recipe for provisioning the missing output by building at all times

9

After a Hydra builder finishes a build, it must be copied explicitly to the AWS S3 cache which forms the backing for cache.nixos.org. Here’s the line that does that copy:

That line appears in copyClosureTo(), which is called from only one place:

By adding a single line right before the call to copyClosureTo():

if (basicDrv->substitutesAllowed())

… and an opt-in for this new behavior, so the change doesn’t affect preexisting Hydra installations.

Caveat: I don’t use Hydra; it isn’t necessary anymore since cppnix is capable of simply nix-building the entire release attrset.

2 Likes