Two (or more) nix stores?

Help
1
  • My local machine has an existing /nix/store. It’s big (50 GB). It’s copied over from another (multi-core) build server on a monthly basis.
  • For developing local projects on a daily basis, I’d like to use /nix/store and also a secondary store in ~/my_nix_store. Two stores.
  • During local development, new builds not available in /nix/store should always go into ~/my_nix_store.

Is this possible?

2 Likes
2

I’m not sure that exactly what you want is possible, I certainly don’t think it’s trivial though.

That said, is your goal to avoid “overwriting” what you sync from the build server? If so, consider the following:

  1. The nix store paths are by definition unique; locally built packages can never overwrite what’s in there already. You should not need to worry about sharing that nix store whatsoever, just rsync-ing in the remote paths (and not overwriting local ones) should never cause issues and in fact reduce disk wear, speed up transfers and waste less energy.
  2. Rather than doing manual rsyncs, have you considered using your build machine as a remote cache? It’s really easy to set up. Your local host will just grab whatever the remote builder has previously built and put it in your local cache. The only downside is that you’ll need an active connection to the server whenever you want to use its cache, but if both machines are local that’s pretty easy (and you can still do an rsync whenever you think you might be without a direct connection for a while).
  • @Solene has nice blog posts on cache hosting or using peerix
    • Also, if you want to keep your private keys outside the cache, use something like sops-nix. I remember that @Solene asks for suggestions on that in the peerix blog post, never gotten around to emailing about that :wink:
3 Likes
3

Spongix may be a nice candidate for local caching remote derivations from caches + your built derivations GitHub - input-output-hk/spongix: Proxy for Nix Caching

4 Likes
4

@TLATER @Solene My use case is an offline (and not powerful) machine. Should have mentioned that, sorry. So the idea is, on a monthly basis I get a fresh large /nix/store copied over from a powerful online machine - but locally I also want to benefit from keeping the nix builds that I built offline (and not losing them after the next monthly copy).

3 Likes
5

In that case, why would you lose the old store contents during the copy?

6

Just use overlayfs with your lower readonly layer a NFS mount from the build-server

7

You may be interested in the efforts to create an overlayfs for the nix store: Super Colliding Nix Stores