Push docker tgz images to registry directly

Guides
1

Anyone who has used the dockerImage.buildImage derivation has this same problem: how do I push these images to the registry?

Before:

docker load -i $(nix-build -A my-image --no-out-link)
# somehow know the docker image name
docker tag $oldname $remotetag
docker push $remotetag

This has multiple problems: it takes time to load and unpack the image in docker, it means that docker needs to be running, and it also means that now there is a need for a garbage-collection mechanism for the loaded images.

Thanks to @lewo for showing me the light, there is a new tool called skopeo that solves all of this (since version 0.1.19):

skopeo copy docker-image://$(nix-build -A my-image --no-out-link) docker://$remotetag

As a bonus, here is a wrapper script that makes the upload a noop if the tag already exists: skopeo-push-maybe · GitHub

13 Likes
2

Nice. I don’t suppose you have a nifty solution for taking a docker-compose.yml file and turning that into nixos containers?

1 Like
3

I suppose it would involve generating a NixOS configuration and then running nixos-rebuild switch. Generally I try to keep projects self-contained and avoid requiring system configuration changes. Unless it involves installing Nix obviously :slight_smile:

1 Like
4

How to do the same with buildah (and a .nix file)?

1 Like
5

Nix outputs a docker image so buildah would be redundant in that context.

2 Likes
6

But how to do it (.nix file to image build) with buildah [instead of docker] - so that docker is not needed anymore

1 Like
7

Docker is already not needed. If you use pkgs.dockerImage.buildImage from nixpkgs it’s possible to build a docker image using just Nix. It outputs an image tarball that can be pushed to the registry with skopeo. I assume that podman could also be used to run the image instead of docker.

You can try it by writing this file to hello-docker.nix, and then run nix-build hello-docker.nix.

{ pkgs ? import <nixpkgs> {} }:
pkgs.dockerTools.buildImage {
  name = "hello";
  config.Cmd = [ "${pkgs.hello}/bin/hello" ];
}
2 Likes
8

dockerImage needs docker as a dependency, right?

My question is to build (from a .nix file) without docker [daemon] → e.g. to build with buildah

1 Like
9

I can see where the confusion is coming from.

pkgs.dockerTools.buildImage uses its own build process and doesn’t depend on the Docker daemon at all. The only docker-related thing is that it outputs docker-compatible images.

4 Likes
10

I just want to add that in latest versions of skopeo the command should be something like:

# ${oci} is what nix build returns
# ${tag} is any docker registry tag like "redis:latest"
skopeo \
  --insecure-policy \
  copy \
  --dest-creds "${user}:${password}" \
  "docker-archive://${oci}" \
  "docker://${tag}"
7 Likes
11

Worth checking out:

It abstracts pushing to Docker Hub, Github Registry and Gitlab Registry

5 Likes
12

A bit late to the party, but has anyone else experience an issue where it seems Skopeo is somehow dropping the manifest “Config” file so that it looses settings like Cmd and Env?

I build the image with nix build .#image and then inspect the result:

$ nix run nixpkgs#skopeo -- inspect tarball:./result
{
    "Digest": "sha256:c50138832b58e8d660a69d35651819458f59ce203fa9c58e49ea60b3e533d25f",
    "RepoTags": [],
    "Created": "1970-01-01T00:00:01Z",
    "DockerVersion": "",
    "Labels": null,
    "Architecture": "amd64",
    "Os": "linux",
    "Layers": [
        "sha256:12db62cd03a0ff98b8d779ab866db2d34e3bb2c18bb52d73b341d63695ea28b5"
    ],
    "LayersData": [
        {
            "MIMEType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "Digest": "sha256:12db62cd03a0ff98b8d779ab866db2d34e3bb2c18bb52d73b341d63695ea28b5",
            "Size": 26764352,
            "Annotations": null
        }
    ],
    "Env": null
}
$ tar -xf ./result manifest.json
$ cat manifest.json
[
    {
        "Config": "d8c535233242b89edd7e8edef6244f8c750bd37aa9d48c1b79f9708ca62a3bc2.json",
        "RepoTags": [
            "corpnet:s3lyvbnlr5mwmqd1hmab5w0pv36ppp23"
        ],
        "Layers": [
            "b5ac3ea78b33b1596ce42ad115568f032dfb6a8887e680aae4d4e7319cb69076/layer.tar",
            "3e20206b75db42deafcbd7b5b620260b69ed3a604622a4c52bf2f5f60db80fd2/layer.tar",
            "552a2254d1bb2f0b5893a1532b8023815f9d7260b8a57879b665f09d262e2814/layer.tar",
            "4d8655c9b9b0a162fc19a65f482267854d1a5d7fcd9ca1ab04b56b47e0114acd/layer.tar",
            "fa4948bee5a5916f9262b6ba61da9eb35938b6acfea60a01a0cf156cbfc7fb5e/layer.tar",
            "9bd12eaf50a785dee7c827e200d3dbc0be8b7dd3268757f54b43800cf93a11ef/layer.tar",
            "ebd4921b10f83481b24207ee53b9f536932529a24596f2f9b6c27d11f8e7295c/layer.tar",
            "67feee1be51bb9643a0fa3d399f9426bbc7560e4ef2cd548be5cfd2ec6a640a8/layer.tar",
            "1287cd89b69b0db9a8f4dc0a5b203be01f7a9971a6b2a359a8552b967fecf35b/layer.tar",
            "b4480b85c0c6730975107c83befdd23c127aa19e400cba5b8f6d574f16f69547/layer.tar",
            "392d35f5e01bbdf8b6a13bc1e950357511ad224f22d891db4deee4fd0edf40e3/layer.tar",
            "a7fd44dca9bc8c65688f897bdfca850f6d5fe8c36b9c145f8eec5b8511f57a1e/layer.tar",
            "8dec630b2eda2af7ea23fc822f401dd47a5cc0ed6878800559868517e5209d60/layer.tar",
            "19159e8e5de77d181def5ab28f5a0263a5d33db446a06db81eecbf6b7052df92/layer.tar",
            "41b4dbdb7ae7625644f3db00403819868e35d546ec683c49d05cb5a523a2bcab/layer.tar",
            "f802cdc09650236fca1c4d3389ed153f17f8113b9befd8ea1972ce33056b0efa/layer.tar"
        ]
    }
]
$ CONFIG=$(cat manifest.json | nix run nixpkgs#jq -- .[0].Config -r)
$ tar -xf ./result $CONFIG
$ cat $CONFIG
{
    "created": "1970-01-01T00:00:01+00:00",
    "architecture": "amd64",
    "os": "linux",
    "config": {
        "Cmd": [
            "/nix/store/kb03aks2fwya4np9wlgnvn35py5q7y2q-corpnet-0.0.0/bin/corpnet"
        ],
        "Labels": {
            "org.opencontainers.image.description": "Auth-injecting MITM"
        }
    },
    "rootfs": {
        "diff_ids": [
            "sha256:b5ac3ea78b33b1596ce42ad115568f032dfb6a8887e680aae4d4e7319cb69076",
            "sha256:3e20206b75db42deafcbd7b5b620260b69ed3a604622a4c52bf2f5f60db80fd2",
            "sha256:552a2254d1bb2f0b5893a1532b8023815f9d7260b8a57879b665f09d262e2814",
            "sha256:4d8655c9b9b0a162fc19a65f482267854d1a5d7fcd9ca1ab04b56b47e0114acd",
            "sha256:fa4948bee5a5916f9262b6ba61da9eb35938b6acfea60a01a0cf156cbfc7fb5e",
            "sha256:9bd12eaf50a785dee7c827e200d3dbc0be8b7dd3268757f54b43800cf93a11ef",
            "sha256:ebd4921b10f83481b24207ee53b9f536932529a24596f2f9b6c27d11f8e7295c",
            "sha256:67feee1be51bb9643a0fa3d399f9426bbc7560e4ef2cd548be5cfd2ec6a640a8",
            "sha256:1287cd89b69b0db9a8f4dc0a5b203be01f7a9971a6b2a359a8552b967fecf35b",
            "sha256:b4480b85c0c6730975107c83befdd23c127aa19e400cba5b8f6d574f16f69547",
            "sha256:392d35f5e01bbdf8b6a13bc1e950357511ad224f22d891db4deee4fd0edf40e3",
            "sha256:a7fd44dca9bc8c65688f897bdfca850f6d5fe8c36b9c145f8eec5b8511f57a1e",
            "sha256:8dec630b2eda2af7ea23fc822f401dd47a5cc0ed6878800559868517e5209d60",
            "sha256:19159e8e5de77d181def5ab28f5a0263a5d33db446a06db81eecbf6b7052df92",
            "sha256:41b4dbdb7ae7625644f3db00403819868e35d546ec683c49d05cb5a523a2bcab",
            "sha256:f802cdc09650236fca1c4d3389ed153f17f8113b9befd8ea1972ce33056b0efa"
        ],
        "type": "layers"
    },
    "history": [
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/cjnmdi9chk0fhjy8p8dq8inxqr4p3vk7-libunistring-1.2']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/04hj9k1gyv26b9gdmcmw3l3bbi124xxi-libidn2-2.3.7']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/mrsnnz2nmnb4mrma1bfv6inmy2k3k3k8-xgcc-13.3.0-libgcc']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/3ilxy06yvz70vg6ld86kyjr6qb248vd6-gcc-13.3.0-libgcc']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/zx9zgvy2nsxcmpn0fzx277m5lk352cdf-gcc-13.3.0-lib']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/p610ybx1pp15lnxxdggm3nmaq31whnvi-attr-2.5.2']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/0l52axcc0k0pn1qpwyhnjz7pipidrwvr-ncurses-6.4.20221231']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/kb03aks2fwya4np9wlgnvn35py5q7y2q-corpnet-0.0.0']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/gda8zjgw0s8zzg5ijza7p12zf5c4b0b0-acl-2.3.2']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/cxz2iaaaipqbf2slznyaazx81d27imlh-gmp-with-cxx-6.3.0']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/sh78727j89acid5dw9xk7nylrb0p0wkj-readline-8.2p10']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/17avbnwwl5r1r9x7fjridjfafdbgmhjd-bash-interactive-5.2p32']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/vb8mdklw65p9wikp97ybmnyay0xzipx3-coreutils-9.5']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/q35mwzks79767rz6agab7919c6w22s4l-nss-cacert-3.101.1']"
        },
        {
            "created": "1970-01-01T00:00:01+00:00",
            "comment": "store paths: ['/nix/store/z3s01qjljpx1ggfg7g7ymz3664d19g6j-corpnet-customisation-layer']"
        }
    ]
}

Note how the d8c535233242b89edd7e8edef6244f8c750bd37aa9d48c1b79f9708ca62a3bc2.json contains a Config with Cmd and Labels. But after publishing the image to a JFrog docker repository:

$ nix run nixpkgs#skopeo -- copy --insecure-policy tarball:./result docker://company.jfrog.io/docker/corpnet/$PLATFORM:$(git describe --tags)

This is the manifest.json:

{
    "schemaVersion": 2,
    "config": {
        "mediaType": "application/vnd.oci.image.config.v1+json",
        "digest": "sha256:362e44cc4823c39e9ddd0cabd57a4eae1a49d590474bc24aea99130485140466",
        "size": 389
    },
    "layers": [
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": "sha256:6608edc16eb84f5e562d69ac4f056396d1f0ea1ea15a1bcba7effc779c8cb0f1",
            "size": 26702877
        }
    ]
}

And the configuration file now looks like this:

{
    "schemaVersion": 2,
    "config": {
        "mediaType": "application/vnd.oci.image.config.v1+json",
        "digest": "sha256:362e44cc4823c39e9ddd0cabd57a4eae1a49d590474bc24aea99130485140466",
        "size": 389
    },
    "layers": [
        {
            "mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
            "digest": "sha256:6608edc16eb84f5e562d69ac4f056396d1f0ea1ea15a1bcba7effc779c8cb0f1",
            "size": 26702877
        }
    ]
}

Or is this a JFrog issue? Our CI pipelines that produce docker images with BuildKit work fine however.

13

I just had the same problem with a similar setup. I replaced tarball:./result with docker-archive:./result and it did sync multiple layers instead of only one layer previously and it works!

14

Thanks a lot @h2horn , this works!

15

related Using skopeo copy from Nix source to docker-daemon target produces no image · Issue #2204 · containers/skopeo · GitHub

2 Likes