{}).system'`
it will fail with
`error: changing ownership of path '/builds/srghma/nix-docker-cached-store-reproduction/.mycache/nix/store': Invalid argument`
Question:
Why it changes GID? I expect it to leave GID of 30000
N.B. I have tried to make some debugging here [test (#153809309) · Jobs · Serhii Khoma / nix-docker-cached-store-reproduction · GitLab](https://gitlab.com/srghma/nix-docker-cached-store-reproduction/-/jobs/153809309)
N.B.
```auto
$ id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
$ nix run --file ./nix/pkgs.nix nix --command id
uid=0(root) gid=0(root) groups=0(root),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody)
# ???
```
---
### Author:  [@danbst](https://discourse.nixos.org/u/danbst)
#### Post date: [January 31, 2019, 1:34am UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/2 "2019-01-31T01:34:36Z")
Maybe not quite the answer… Is it possible to move `/nix` to `/cache` after tests are run, and populate `/nix` from `/cache` at start?
---
### Author:  [@srghma](https://discourse.nixos.org/u/srghma)
#### Post date: [January 31, 2019, 7:03am UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/3 "2019-01-31T07:03:39Z")
I would avoid that cause it takes some time
---
### Author:  [@srghma](https://discourse.nixos.org/u/srghma)
#### Post date: [February 2, 2019, 8:52pm UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/4 "2019-02-02T20:52:19Z")
It became clear for me that i can’t use `nix-build` inside `nix run --store...` (chroot) inside docker, so I decided to try `nix-build --store ...` inside docker
Now i can’t access network
[https://github.com/NixOS/nix/issues/2663](https://github.com/NixOS/nix/issues/2663)
---
### Author:  [@srghma](https://discourse.nixos.org/u/srghma)
#### Post date: [February 7, 2019, 6:46pm UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/5 "2019-02-07T18:46:53Z")
Current state of research
> **[Nix Hydra Pipeline (#17854) · Issues · GitLab.org / GitLab · GitLab](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/31312)**
>
> Description Right now the Shared Runners feature let's users run build processes on shared servers and save the build artifacts. The...
---
### Author:  [@pimiddy](https://discourse.nixos.org/u/pimiddy)
#### Post date: [January 20, 2020, 2:24pm UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/6 "2020-01-20T14:24:20Z")
Did you make any further progress on this? I’m in the same situation and don’t really know what to do.
---
### Author:  [@srghma](https://discourse.nixos.org/u/srghma)
#### Post date: [January 20, 2020, 3:58pm UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/7 "2020-01-20T15:58:19Z")
Hi, in comment above you can find my solution that works without copying
Its a hack on how gitlab makes caching volume for docker runner
Cons:
- gitlab knows nothing about /cache/nix, so you are not able to clean to on UI
- it doesn’t remove stale packages once a week
---
### Author:  [@turion](https://discourse.nixos.org/u/turion)
#### Post date: [March 21, 2020, 3:45pm UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/8 "2020-03-21T15:45:59Z")
I ended up using [https://cachix.org/](https://cachix.org/) for caching. The setup is basically:
```auto
$ cat .gitlab-ci.yml
image: nixos/nix:latest
before_script:
- nix-env -iA nixpkgs.cachix
- cachix use insertyourcachixnamehere
build:
stage: build
script:
- nix-build
```
Before I push build on CI, I do `nix-build | cachix push insertyourcachixnamehere` on my dev machine. It uploads store paths to my personal cache, and CI downloads them instead of rebuilding them. I could cut down the pipeline time from 30 minutes to 3 that way.
---
### Author:  [@kamadorueda](https://discourse.nixos.org/u/kamadorueda)
#### Post date: [August 22, 2021, 6:10am UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/9 "2021-08-22T06:10:01Z")
We also use Cachix on Gitlab, though we use a [framework](https://github.com/fluidattacks/makes#cache) to avoid the boilerplate
- Cache is configured like this: [makes.nix · 9ab7a0cc74360a3246b4e39a51765bcc5c60f17e · Fluid Attacks / universe · GitLab](https://gitlab.com/fluidattacks/product/-/blob/9ab7a0cc74360a3246b4e39a51765bcc5c60f17e/makes.nix#L8)
- Jobs look like this: [.gitlab-ci.yml · 9ab7a0cc74360a3246b4e39a51765bcc5c60f17e · Fluid Attacks / universe · GitLab](https://gitlab.com/fluidattacks/product/-/blob/9ab7a0cc74360a3246b4e39a51765bcc5c60f17e/.gitlab-ci.yml#L101)
The side benefit of cachix is that when you build locally it’ll also use the cache, and when you build locally you are also populating the cache for the CI. So win-win
The infrastructure is simply deployed with terraform:
- Example: [makes/makes/ci · a25d59662e3f7b63809166b56899b88ef6d756fa · Fluid Attacks / universe · GitLab](https://gitlab.com/fluidattacks/product/-/tree/a25d59662e3f7b63809166b56899b88ef6d756fa/makes/makes/ci)
- Using [GitHub - cattle-ops/terraform-aws-gitlab-runner: Terraform module for AWS GitLab runners on ec2 (spot) instances](https://github.com/npalm/terraform-aws-gitlab-runner)
The arch is this:
It runs thousand of pipelines a day where each pipeline has 100+ jobs. The arch auto-scales automatically, making it pretty fast and cost-effective
---
### Author:  [@volth](https://discourse.nixos.org/u/volth)
#### Post date: [August 24, 2021, 9:14am UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/10 "2021-08-24T09:14:49Z")
Anyone tried to mount read-only a network storage as /nix/store on the workers?
---
### Author:  [@domenkozar](https://discourse.nixos.org/u/domenkozar)
#### Post date: [September 14, 2021, 11:44am UTC](https://discourse.nixos.org/t/docker-nix-gitlab-ci-cache-nix-store-per-machine/2003/11 "2021-09-14T11:44:13Z")
That’s so great! 🙂
I’ve implemented 5min caching for narinfo files as Cachix is getting 1000 req/s at peaks, which has reduced the load significantly. There were a few short bursts of high load since Friday.
In other news, Nix 2.4 reduces number of requested narinfos quite a bit so this will all get better soon: [Short-circuit querying substituters on success by Kha · Pull Request #5109 · NixOS/nix · GitHub](https://github.com/NixOS/nix/pull/5109)