Nixos image on dockerhub has 100 layers, close to limit of 127

Development
1

The nixos/nix image on dockerhub has 100 layers. Several of them just a few kilobytes in size.

In 2.3.16 it was 7 layers. Then something changed in the build process, and in the 2.5.1 it was already 67 layers. It has been 100 layers since at least 2.8.0.

As a smaller issue, this makes the layer download take much longer than needed since every layer is fetched individually. But the bigger problem is that in practice, there is a limit to the maximum amount of layers that AUFS will allow. It’s not clear what that limit is, since it may vary between versions and systems, that’s also why this is not documented. However, we found docker erroring out at 127 layers in our build pipeline, so the nixos/nix base image limits us to 27 layers.

If there is no technical reason to have so many layers, I would welcome a change to reduce this.

Relatedly: I was unable to find more information on where, how, and by whom the docker images on dockerhub are built, so I could submit this for discussion there. Perhaps I just missed it, pointers would be appreciated :slight_smile:

3 Likes
2

The image is probalbly built from this expression:

It has a default maxLayers of 100, which matches your description.

You can build your own image from this with a lower layer max. Though the relevant output sadly doesn’t seem to be overrideable, so you have to import it manually when trying to build it.

A bit of a nasty thing.

3

Hey, thanks for your reply! That at least clears up where this image comes from :+1:

Wait, so it intentionally fills 100 layers? Or spreads the data over those layers? Or is it just using exactly 100 layers by coincidence and that is also specified as max?

I don’t quite get it, could you (or someone else) elaborate what’s going on there?

4

You might want to read about the ideas behind dockerTools.buildLayeredImage.

It does indeed intentionally build as many layers as possible, even though the heuristics when it combines layers and how are weird.

In my opinion, for small images (up to 20 or 30 store paths) the idea is good, but it sucks for everything bigger than that.

2 Likes
5

The images are built here: GitHub - nix-community/docker-nixpkgs: docker images from nixpkgs [maintainer=@zimbatm]

6

I’m fairly sure it’s actually here. Thanks for your input folks, I’ll raise in issue in that repo :slight_smile:

1 Like
7

fyi: Reduce maxLayers to 70 in docker build by Valodim · Pull Request #13155 · NixOS/nix · GitHub

2 Likes