Building Container Images

Two make targets exists to build container images automatically based on the locally checked out branch:

Developer images

Run make dev-docker-image to build a cilium-agent Docker image that contains your local changes.

  1. ARCH=amd64 DOCKER_DEV_ACCOUNT=quay.io/myaccount DOCKER_IMAGE_TAG=jane-developer-my-fix make dev-docker-image

Run make docker-operator-generic-image (respectively, docker-operator-aws-image or docker-operator-azure-image) to build the cilium-operator Docker image:

  1. ARCH=amd64 DOCKER_DEV_ACCOUNT=quay.io/myaccount DOCKER_IMAGE_TAG=jane-developer-my-fix make docker-operator-generic-image

The commands above assumes that your username for quay.io is myaccount.

Race detection

See section on compiling Cilium with race detection.

Official release images

Anyone can build official release images using the make target below.

  1. DOCKER_IMAGE_TAG=v1.4.0 make docker-images-all

Experimental Docker BuildKit and Buildx support

Docker BuildKit allows build artifact caching between builds and generally results in faster builds for the developer. Support can be enabled by:

  1. export DOCKER_BUILDKIT=1

Multi-arch image build support for arm64 (aka aarch64) and amd64 (aka x86-64) can be enabled by defining:

  1. export DOCKER_BUILDX=1

Multi-arch images are built using a cross-compilation builder by default, which uses Go cross compilation for Go targets, and QEMU based emulation for other build steps. You can also define your own Buildx builder if you have access to both arm64 and amd64 machines. The “cross” builder will be defined and used if your current builder is “default”.

Buildx targets push images automatically, so you must also have DOCKER_REGISTRY and DOCKER_DEV_ACCOUNT defined, e.g.:

  1. export DOCKER_REGISTRY=docker.io
  2. export DOCKER_DEV_ACCOUNT=your-account

Currently the cilium-runtime and cilium-builder images are released for amd64 only (see the table below). This means that you have to build your own cilium-runtime and cilium-builder images:

  1. make -C images runtime-image

After the build finishes update the runtime image references in other Dockerfiles (docker buildx imagetools inspect is useful for finding image information). Then proceed to build the cilium-builder:

  1. make -C images builder-image

After the build finishes update the main Cilium Dockerfile with the new builder reference, then proceed to build Hubble from github.com/cilium/hubble. Hubble builds via buildx QEMU based emulation, unless you have an ARM machine added to your buildx builder:

  1. export IMAGE_REPOSITORY=${DOCKER_REGISTRY}/${DOCKER_DEV_ACCOUNT}/hubble
  2. CONTAINER_ENGINE="docker buildx" DOCKER_FLAGS="--push --platform=linux/arm64,linux/amd64" make image

Update the main Cilium Dockerfile with the new Hubble reference and build the multi-arch versions of the Cilium images:

  1. make docker-images-all

Official Cilium repositories

The following table contains the main container image repositories managed by Cilium team. It is planned to convert the build process for all images based on GH actions.

Github Repository

Dockerfile

container image repository

Architectures

Build process

amd64

aarch64

github.com/cilium/cilium

images/runtime/Dockerfile

quay.io/cilium/cilium-runtime

Y

Y

GH Action

images/builder/Dockerfile

quay.io/cilium/cilium-builder

Y

Y

GH Action

images/cilium/Dockerfile

[docker|quay].io/cilium/cilium

Y

Y

GH Action

images/cilium-docker-plugin/Dockerfile

[docker|quay].io/cilium/docker-plugin

Y

Y

GH Action

images/hubble-relay/Dockerfile

[docker|quay].io/cilium/hubble-relay

Y

Y

GH Action

images/operator/Dockerfile

[docker|quay].io/cilium/operator

Y

Y

GH Action

images/operator-aws/Dockerfile

[docker|quay].io/cilium/operator-aws

Y

Y

GH Action

images/operator-azure/Dockerfile

[docker|quay].io/cilium/operator-azure

Y

Y

GH Action

images/operator-generic/Dockerfile

[docker|quay].io/cilium/operator-generic

Y

Y

GH Action

images/clustermesh-apiserver/Dockerfile

[docker|quay].io/cilium/clustermesh-apiserver

Y

Y

GH Action

github.com/cilium/proxy

Dockerfile.builder

quay.io/cilium/cilium-envoy-builder

Y

Y

GH Action

Dockerfile

quay.io/cilium/cilium-envoy

Y

Y

GH Action

github.com/cilium/image-tools

images/bpftool/Dockerfile

docker.io/cilium/cilium-bpftool

Y

Y

GH Action

images/iproute2/Dockerfile

docker.io/cilium/cilium-iproute2

Y

Y

GH Action

images/llvm/Dockerfile

docker.io/cilium/cilium-llvm

Y

Y

GH Action

images/compilers/Dockerfile

docker.io/cilium/image-compilers

Y

Y

GH Action

images/maker/Dockerfile

docker.io/cilium/image-maker

Y

Y

GH Action

images/startup-script/Dockerfile

docker.io/cilium/startup-script

Y

Y

GH Action

Image dependency:

  1. [docker|quay].io/cilium/cilium
  2.  depends on:
  3.   quay.io/cilium/cilium-builder
  4.    depends on:
  5.     quay.io/cilium/cilium-runtime
  6.      depends on:
  7.       docker.io/cilium/cilium-iproute2
  8.       docker.io/cilium/cilium-bpftool
  9.       docker.io/cilium/cilium-llvm
  10.   quay.io/cilium/cilium-envoy
  11.    depends on:
  12.     quay.io/cilium/cilium-envoy-builder
  13.      depends on:
  14.       quay.io/cilium/cilium-builder
  15.        depends on:
  16.         quay.io/cilium/cilium-runtime
  17.          depends on:
  18.           docker.io/cilium/cilium-iproute2
  19.           docker.io/cilium/cilium-bpftool
  20.           docker.io/cilium/cilium-llvm

Update cilium-builder and cilium-runtime images

The steps described here, starting with a commit that updates the image versions, build the necessary images and update all the appropriate locations in the Cilium codebase. Hence, before executing the following steps, the user should have such a commit (e.g., see this commit) in their local tree. After following the steps below, the result would be another commit with the image updates (e.g,. see this commit). Please keep the two commits separate to ease backporting.

If you only wish to update the packages in these images, then you can manually update the FORCE_BUILD variable in images/runtime/Dockerfile to have a different value and then proceed with the steps below.

  1. cilium-builder depends on cilium-runtime so one needs to update cilium-runtime first. Steps 4 and 7 will fetch the digest of the image built by GitHub actions.

    1. $ make -C images/ update-runtime-image

  2. Commit your changes and create a PR in cilium/cilium.

    1. $ git commit -sam "images: update cilium-{runtime,builder}"

  3. Ping one of the members of team/build to approve the build that was created by GitHub Actions here. Note that at this step cilium-builder build failure is expected since we have yet to update the runtime digest.

  4. Wait for cilium-runtime build to complete. Only after the image is available run:

    1. $ make -C images/ update-runtime-image update-builder-image

  5. Commit your changes and re-push to the PR in cilium/cilium.

    1. $ git commit --amend -sa

  6. Ping one of the members of team/build to approve the build that was created by GitHub Actions here.

  7. Wait for the build to complete. Only after the image is available run:

    1. $ make -C images/ update-runtime-image update-builder-image

  8. Commit your changes and re-push to the PR in cilium/cilium.

    1. $ git commit --amend -sa

  9. Update the versions of the images that are pulled into the CI VMs.

  • Open a PR against the Packer-CI-Build with an update to said image versions. Once your PR is merged, a new version of the VM will be ready for consumption in the CI.

  • Update the SERVER_VERSION field in test/Vagrantfile to contain the new version, which is the build number from the Jenkins Job for the VMs. For example, build 119 from the pipeline would be the value to set for SERVER_VERSION.

  • Open a pull request with this version change in the cilium repository.

Nightly Docker image

After each successful Nightly build, a cilium/nightly image is pushed to dockerhub.

To use latest nightly build, please use cilium/nightly:latest tag. Nightly images are stored on dockerhub tagged with following format: YYYYMMDD-<job number>. Job number is added to tag for the unlikely event of two consecutive nightly builds being built on the same date.

Image Building Process

Images are automatically created by a GitHub action: build-images. This action will automatically run for any Pull Request, including Pull Requests submitted from forked repositories, and push the images into quay.io/cilium/*-ci. They will be available there for 1 week before they are removed by the ci-images-garbage-collect workflow. Once they are removed, the developer must re-push the Pull Request into GitHub so that new images are created.