zuul/zuul-jobs
Code Issues Proposed changes

build-container-image: expand docs

Browse Source 
This goes into a bit more detail on the advantages/disadvantages of
each method.

Change-Id: Ie90a52f1c0e205e9f8552156aded871b6fd30214
This commit is contained in:
Ian Wienand 2023-03-30 08:29:16 +11:00
parent 51e437c2f1
commit d7e5559e58
No known key found for this signature in database
1 changed files with 77 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

96
roles/build-container-image/common.rst
View File

@ -6,33 +6,91 @@ context:
* :zuul:role:`upload-container-image`: Upload the images to a registry.
* :zuul:role:`promote-container-image`: Promote previously uploaded images.
All roles accept the same input data, principally a list of
dictionaries representing the images to build. YAML anchors_ can be
used to supply the same data to all three roles.
*Building*
The :zuul:role:`build-container-image` role is designed to be used in
`check` and `gate` pipelines and simply builds the images. It can be
used to verify that the build functions, or it can be followed by the
use of subsequent roles to upload the images to a registry.
*Uploading*
The :zuul:role:`upload-container-image` role uploads the images to a
registry. It can be used in one of two modes: by default it will
upload with a single tag corresponding to the change ID. In this
mode, the role role is designed to be used in a job in a `gate`
pipeline so that the build produced by the gate is staged and can
later be promoted to production if the change is successful. The
other mode allows for use of this job in a `release` pipeline to
directly upload a release build with the final set of tags.
registry. It can be used in one of two modes:
The :zuul:role:`promote-container-image` role is designed to be used
in a `promote` pipeline. It requires no nodes and runs very quickly
on the Zuul executor. It simply re-tags a previously uploaded image
for a change with whatever tags are supplied by
:zuul:rolevar:`build-container-image.container_images.tags`. It also
removes the change ID tag from the repository in the registry, and
removes any similar change ID tags. This keeps the repository tidy in
the case that gated changes fail to merge after uploading their staged
images.
1. The default mode is as part of a two-step `promote` pipeline. This
mode is designed to minimize the time the published registry tag is
out of sync with the changes Zuul has merged to the underlying code
repository.
They all accept the same input data, principally a list of
dictionaries representing the images to build. YAML anchors_ can be
used to supply the same data to all three jobs.
In this mode, the role is intended to run in the `gate` pipeline.
Zuul will build and upload the resulting image with a single tag
prefixed with the change ID (e.g. ``change_12345_<tag>``). Thus at
the completion of the `gate` job, all the layers of the new
container are uploaded, but the ``<tag>`` in the remote repository
will not have been updated.
Once the gate queue is successfully finished Zuul will merge the
change to the code-repository. At this point, a small window opens
where the ``<tag>`` is pointing to a container that does not
reflect the state of the code-repository. The merge of the change
will trigger the `promote` pipeline which uses a very quick,
executor-only job to retag ``<tag>`` to ``change_12345_<tag>``.
Since this step does not require any nodes or upload any data, it
generally takes only a few seconds. The remote container pointed
to by ``<tag>`` will now reflect the underlying code closing the
out-of-sync window.
2. The other mode allows for use of this job in a `release` pipeline
to directly upload a release build with the final set of tags.
In this mode, the completion of the `gate` jobs will have merged
the code changes, and the role will now have to build and upload
the resulting image to the remote repository. Once uploaded, the
tags will be updated.
The alternative `promote` method can be thought of as a
"speculative" upload. There is a possibility the `gate` job
uploads layers and creates a temporary tag, but either the
container upload or another co-gating job fails, causing the `gate`
jobs to fail overall. This causes extra uploads, unsued layers and
unused tags that require cleaning up. Since changes have merged
before the `release` pipeline starts, the upload will simply not
run if the gate jobs fail. This avoids uploading or tagging
anything that will not be used. The trade-off is a higher latency
between merging code and publishing final tags.
Transient network failures can cause upload errors in both cases.
Although the `promote` job may fail, leaving the tag incorrectly
unmodified, the `promote` job's relatively simplicity minimises
potential error. The `release` pipeline does more work, exposing
it to a higher chance of failures such as transient network errors
etc., also resulting in the repository tag being out-of-date. In
both cases developers must pay close attention as failures in these
pipelines are often less noticable than code not merging with a
gate-job failure.
*Promoting*
As discussed above, the :zuul:role:`promote-container-image` role is
designed to be used in a `promote` pipeline. It re-tags a previously
uploaded image by copying the temporary change-id based tags made
during upload to the final production tags supplied by
:zuul:rolevar:`build-container-image.container_images.tags`. It is
intended to run very quickly and with no dependencies, so it can run
directly on the Zuul executor.
Once this role completes, the temporary upload tags are no longer
required. The role removes the change-id tags from the repository in
the registry, and removes any similar change-ids tags. This keeps the
repository tidy in the case that gated changes fail to merge after
uploading their staged images.
*Dependencies*
Use the :zuul:role:`ensure-skopeo` role as well as the
:zuul:role:`ensure-docker`, or :zuul:role:`ensure-podman` roles before