In case it’s a useful data point, I tried nix-community/cache-nix-action for a project last month, then ended up deciding to switch to DeterminateSystems/magic-nix-cache-action for my projects instead; here are the notes I wrote at the time:
After thinking about it a bit more, I think I’m actually just gonna switch to the Magic Nix Cache:
According to the GitHub docs, “the total size of all caches in a repository is limited to 10 GB.” Each cache entry for my website is one gigabyte, so this means they’ll start being deleted after I reach ten of them.
Cache eviction works “by deleting the caches in order of last access date, from oldest to most recent.” So if I have a branch with ten commits, the cache for the main branch gets deleted.
Each cache is bigger than the last, because no garbage collection happens by default.
The garbage collection just works by setting a cap on the size of each cache, not by deleting things unreachable from the current build.
So, the Magic Nix Cache just seems like the better solution: since each /nix/store path is its own cache entry, the ones that do overlap with main will just be reused over and over, so they won’t get deleted. The only way for stuff from main to actually get evicted from the cache would be for the sum of everything in all the other branches to exceed several gigabytes beyond what’s already in main, with nothing using anything from main in between. And at that point it seems reasonable to evict anyways. It just seems like fundamentally the better approach.
Yeah, that part is annoying. Part of me wonders whether it’s trying to upload all the caches serially and thus wasting time by not uploading multiple in parallel. But I haven’t dug into it so I don’t know.
In any case, in my experience, even though the post-run step where it uploads remaining caches can sometimes take longer than the build itself, it can still be worth it if usually you’re reusing most of the work. So for instance, on one of my projects, a from-scratch rebuild with no cache might take 45 minutes, and in some cases a CI run with the cache might take 5 minutes to build and 10 minutes to upload the remaining cache, but that’s still faster than doing it from scratch. Of course, there are situations where you don’t reuse most of the work, and in those cases the Magic Nix Cache seems like just a net negative.
It’s also worth noting that it’s already uploading caches during your other CI steps, it just does so silently. The post-run step is only printing logs for the ones it didn’t have enough time to finish in the background earlier.
I will say, it’s unclear to me how much value you’re adding by opening DeterminateSystems/magic-nix-cache-action#155; you already linked to the existing DeterminateSystems/magic-nix-cache#154 in which the person who opened the issue provided details explaining exactly what is happening that seems unnecessary, and why they expected better. Your issue says that the Magic Nix Cache bumped up build times, but doesn’t provide any reason for why you expect that shouldn’t happen. It’s definitely possible that it could be improved, but the issue you opened doesn’t provide any reasoning as to why you believe that to be the case.
