dockerTools.buildImage fails with "rsync:...Cannot allocate memory"

Help
1

I am trying to build an image and getting this error message:

...
Adding contents...
Adding /nix/store/XXXXX...
Adding /nix/store/YYYYY...
rsync: [sender] readdir("/nix/store/ZZZZZ.../some/file"): Cannot allocate memory (12)

Are there any workarounds?

2

How much RAM do you have and how much is it filled right before the error?

3

32G total. I am on a shared server with cgroups so I can reserve the memory for my shell session, but I guess the actual build process is running in its own cgroup since it is orchestrated by nix-daemon. Is there anything that can be done to give the build process more memory? Or is it possible to have buildImage use disk instead of collecting the layers in memory?

4

As a workaround, you could use dockerTools.streamLayeredImage or nix2container.buildImage which don’t store layers at all :slight_smile:

5

I am finally starting to experiment with dockerTools.streamLayeredImage! So far my two main questions are:

  1. If packages sitting in different layers have one big dependency, would the final image have just one copy of that dependency or will it be stored as many times as I have layers depending on it? In my case it is MKL library and I have a layer with R closure and another one with Python. Both are built with MKL support.

  2. I use runAsRoot clause to run dockerTools.shadowSetup to create a user, then a bunch of chown and chmod commands. Since runAsRoot is not supported by streamLayeredImage, can fakeRootCommands be used as a replacement? Will it be able to run all those commands?