From 5994ce4049afc28dd5f5548315194ecf9d2d75a4 Mon Sep 17 00:00:00 2001 From: Clark Boylan Date: Tue, 9 May 2023 08:45:55 -0700 Subject: [PATCH] Gracefully handle use of intermediate registry in container upload role For symmetry and ease of transition between the docker specific jobs/roles and generic container jobs/roles it is advantageous to have the container upload role skip pushing artifacts to the final registry location if we are relying on the intermediate registry instead. Update the container upload role to skip pushing to the actual registry if the promote var is set to intermediate registry. This allows us to avoid reshuffling all of our jobs as we migrate between the two implementations. Change-Id: I3cae9e03517cb0a5ce8e9369bf43fd052cac97ff --- playbooks/container-image/README.rst | 14 +++--- roles/build-container-image/common.rst | 12 +++++ roles/upload-container-image/tasks/main.yaml | 51 +++++++++++--------- 3 files changed, 47 insertions(+), 30 deletions(-) diff --git a/playbooks/container-image/README.rst b/playbooks/container-image/README.rst index b788e0da2..8ec4bc8af 100644 --- a/playbooks/container-image/README.rst +++ b/playbooks/container-image/README.rst @@ -58,13 +58,13 @@ Summary: *Promotion via intermediate registry* -Note that as of 2023-03, this path is not fully implemented. It is -documented here for compeleteness. +The :zuul:job:`build-container-image` runs in the `check` pipeline. +It will build images then upload them to an intermediate registry. -The :zuul:job:`build-container-image` runs in the `check` pipeline, -but also in the `gate` pipeline. Usually in both cases the job builds -and uploads the images to an intermediate registry; but at least the -`gate` pipeline job must.. +The :zuul:job:`upload-container-image` job runs in the `gate`. With +this promotion method it will build and upload images to an intermediate +registry. No images will be pushed to the upstream registry until +promotion occurs. The :zuul:job:`promote-container-image` job is designed to be used in a post-merge `promote` pipeline. It requires no nodes and run on the @@ -94,7 +94,7 @@ between upload and promote steps in this model. Summary: * :zuul:job:`build-container-image` in `check` -* :zuul:job:`build-container-image` in `gate`. This must push to an +* :zuul:job:`upload-container-image` in `gate`. This must push to an intermediate registry. * :zuul:job:`promote-container-image` in `promote` with ``promote_container_method: intermediate-registry`` diff --git a/roles/build-container-image/common.rst b/roles/build-container-image/common.rst index cca4d4e59..93123c920 100644 --- a/roles/build-container-image/common.rst +++ b/roles/build-container-image/common.rst @@ -45,6 +45,10 @@ registry. It can be used in one of two modes: to by ```` will now reflect the underlying code closing the out-of-sync window. + When running in this mode uploads are only made if + ``promote_container_image_method`` is unset or set to ``tag``. + Otherwise we skip upload to the registry. + 2. The second mode allows for use of this job in `release` and `tag` pipelines to directly upload a release build with the final set of tags. @@ -266,4 +270,12 @@ promote job assumes `skopeo` is available on the executor. A dictionary of key value pairs to add to the container build environment. This may be useful to enable buildkit with docker builds for example. +.. zuul:rolevar:: promote_container_image_method + :default: tag + + A string value indicating whether or not we upload images to the upstream + registry pre merge then promote that upload via a retag (``tag``) or we + upload to a downstream registry and later fetch and promote that to the + upstream registry post merge (``intermediate-registry``). + .. _anchors: https://yaml.org/spec/1.2/spec.html#&%20anchor// diff --git a/roles/upload-container-image/tasks/main.yaml b/roles/upload-container-image/tasks/main.yaml index 063097588..9239a436f 100644 --- a/roles/upload-container-image/tasks/main.yaml +++ b/roles/upload-container-image/tasks/main.yaml @@ -1,25 +1,30 @@ -- name: Verify repository names - when: | - container_registry_credentials is defined - and zj_image.registry not in container_registry_credentials - loop: "{{ container_images }}" - loop_control: - loop_var: zj_image - fail: - msg: "{{ zj_image.registry }} credentials not found" +- name: Control when we push to the upstream registry + # We only want to push upstream if we are in a release / tag pipeline or + # if we are using the tag promotion method. + block: + - name: Verify repository names + when: | + container_registry_credentials is defined + and zj_image.registry not in container_registry_credentials + loop: "{{ container_images }}" + loop_control: + loop_var: zj_image + fail: + msg: "{{ zj_image.registry }} credentials not found" -- name: Verify repository permission - when: | - container_registry_credentials[zj_image.registry].repository is defined and - not zj_image.repository | regex_search(container_registry_credentials[zj_image.registry].repository) - loop: "{{ container_images }}" - loop_control: - loop_var: zj_image - fail: - msg: "{{ zj_image.repository }} not permitted by {{ container_registry_credentials[zj_image.registry].repository }}" + - name: Verify repository permission + when: | + container_registry_credentials[zj_image.registry].repository is defined and + not zj_image.repository | regex_search(container_registry_credentials[zj_image.registry].repository) + loop: "{{ container_images }}" + loop_control: + loop_var: zj_image + fail: + msg: "{{ zj_image.repository }} not permitted by {{ container_registry_credentials[zj_image.registry].repository }}" -- name: Upload image to container registry - loop: "{{ container_images }}" - loop_control: - loop_var: zj_image - include_tasks: push.yaml + - name: Upload image to container registry + loop: "{{ container_images }}" + loop_control: + loop_var: zj_image + include_tasks: push.yaml + when: not upload_container_image_promote|default(true) or promote_container_image_method|default('tag') == 'tag'