Sharing Nix store between containers

Technically it can work if both the host and the container have the same store paths available. This can be ensured by adding a GC root on the host so that the container paths never get collected.

It means that the host needs to keep the GC root and deployed containers in sync. Let’s say that an updated container image is being shipped. Now the host also have to be upgraded. Should the old imager references be kept around? What mechanism is being used to handle these pins?

The second thing is that the Docker image will ship with a bunch of /nix/store entries, which will never be used because /nix is hidden by the host bind-mount. This is not super critical but it makes the image larger than necessary.

One strategy to work around that is to embed a static version of the Nix binary in the container. Put it in /bootstrap/nix or something. And then bootstrap all of the rest using it. Even the version of Nix that will be used after the bootstrap can be determined by it.

  1. Setup a binary cache
  2. Build and publish the development environment to the cache. Keep note of that store path.
  3. In the container:
    a. Run (exec -a nix-store /bootstrap/nix --realize /nix/store/<hash>-my-env) and add this to the PATH.
    b. Run the rest of the CI commands.

That’s one way to do it that relies heavily on the store. Another way is to ship the container with the bootstrap nix and the nix code that is used to setup the environment. Then replace the nix-store realization with a nix-build.

1 Like