Docker image produced by dockerTools.buildImage not bit identical

Help
1

I wrote a Nix recipe for building a package that produces a docker image. It packages together Postgres, a Parquet foreign data wrapper module for Postgres, some sql script that runs on Postgres, gcloud cli tools and some bash entrypoint code glueing it all together producing a docker image. I built this recipe on two different machines, one a fedora 6.4.7 VMWare guest and another a VM on GCP running Debian 12, both x86_64. The nix store path hash of both builds are identical but the sha256sum is different for the docker image. The Parquet foreign data wrapper artifact OTOH had the same hash. Is this a known limitation of docker tools or is this not expected? I can share the recipe if needed. Thanks.

2

Is either the tag, name or the created date( seen with ’docker images’)different? Do both machines follow the same nix channel/nixpkgs revision?

3

Thanks for pointing me to that direction. After fiddling a little bit, I was able to find the difference with the tool dive. I am attaching the screenshot of the two images highlighting the difference, which seems like the debian generated image’s postgresql-14.8 has the man page folder but the fedora generated one does not.

dive-debian.PNG
dive-debian.PNG1920×1150 244 KB

dive-fedora.PNG
dive-fedora.PNG1920×1150 244 KB

It is the same clean repo and same values of flake.lock file etc. I am unclear why this might happen.

4

It can very easily happen, if one doesn’t pin one’s imports then the container will be build with whatever the build system has on-hand…

I made this mistake myself… and then reproduced the problem I was having by commenting and un-commenting out blocks in the imports below:

# { pkgs ? import <nixpkgs> {} }: # FIXME: non-deterministic build - this will use whatever system packages are installed
# NixOS 23.11 - FIXME: Shipped with zlib 1.3 which has CVE-2023-45853
# { pkgs ? import (builtins.fetchGit {
#   url = "https://github.com/nixos/nixpkgs.git";
#   rev = "219951b495fc2eac67b1456824cc1ec1fd2ee659";
#   shallow = true;
# }) {} }:
# NixOS 24.05 - FIXED: zlib 1.3.1 - No more CVE-2023-45853
{ pkgs ? import (builtins.fetchGit {
  url = "https://github.com/nixos/nixpkgs.git";
  rev = "63dacb46bf939521bdc93981b4cbb7ecb58427a0"; # Deterministic: Pin NixPkgs at this rev for our build tools
  shallow = true;
}) {} }:
5

This is a pretty old post, but I think this just comes down to Nix not being able to guarantee bit-for-bit deterministic builds.

Note, the OP indicates the the store hashes where the same, so it’s not really due to lack of unpinned inputs. (That would appear as differences in the store hash, as well as likely bitwise different outputs.)

6

@colemickens @deftdawg Thanks for your replies. I am using nix flakes so pinning was taken care of automatically (and or rather because of it the nix store hash was the same). Anyway I let it be and moved on with actual code changes and after that the issue just went away. So I too strongly suspect that had something to do with Nix’s inability to guarantee reproducible builds. I should mention, it never reappeared afterwards.

FYI fast forward a year, when I switched the Python part of the project to Poetry and poetry2nix, the non reproducibility came back again and this time permanently. But that has nothing to do with docker tools.