diff --git a/roles/build-container-image/common.rst b/roles/build-container-image/common.rst index cbcfbdd9a..52583594e 100644 --- a/roles/build-container-image/common.rst +++ b/roles/build-container-image/common.rst @@ -6,33 +6,91 @@ context: * :zuul:role:`upload-container-image`: Upload the images to a registry. * :zuul:role:`promote-container-image`: Promote previously uploaded images. +All roles accept the same input data, principally a list of +dictionaries representing the images to build. YAML anchors_ can be +used to supply the same data to all three roles. + +*Building* + The :zuul:role:`build-container-image` role is designed to be used in `check` and `gate` pipelines and simply builds the images. It can be used to verify that the build functions, or it can be followed by the use of subsequent roles to upload the images to a registry. +*Uploading* + The :zuul:role:`upload-container-image` role uploads the images to a -registry. It can be used in one of two modes: by default it will -upload with a single tag corresponding to the change ID. In this -mode, the role role is designed to be used in a job in a `gate` -pipeline so that the build produced by the gate is staged and can -later be promoted to production if the change is successful. The -other mode allows for use of this job in a `release` pipeline to -directly upload a release build with the final set of tags. +registry. It can be used in one of two modes: -The :zuul:role:`promote-container-image` role is designed to be used -in a `promote` pipeline. It requires no nodes and runs very quickly -on the Zuul executor. It simply re-tags a previously uploaded image -for a change with whatever tags are supplied by -:zuul:rolevar:`build-container-image.container_images.tags`. It also -removes the change ID tag from the repository in the registry, and -removes any similar change ID tags. This keeps the repository tidy in -the case that gated changes fail to merge after uploading their staged -images. +1. The default mode is as part of a two-step `promote` pipeline. This + mode is designed to minimize the time the published registry tag is + out of sync with the changes Zuul has merged to the underlying code + repository. -They all accept the same input data, principally a list of -dictionaries representing the images to build. YAML anchors_ can be -used to supply the same data to all three jobs. + In this mode, the role is intended to run in the `gate` pipeline. + Zuul will build and upload the resulting image with a single tag + prefixed with the change ID (e.g. ``change_12345_``). Thus at + the completion of the `gate` job, all the layers of the new + container are uploaded, but the ```` in the remote repository + will not have been updated. + + Once the gate queue is successfully finished Zuul will merge the + change to the code-repository. At this point, a small window opens + where the ```` is pointing to a container that does not + reflect the state of the code-repository. The merge of the change + will trigger the `promote` pipeline which uses a very quick, + executor-only job to retag ```` to ``change_12345_``. + Since this step does not require any nodes or upload any data, it + generally takes only a few seconds. The remote container pointed + to by ```` will now reflect the underlying code closing the + out-of-sync window. + +2. The other mode allows for use of this job in a `release` pipeline + to directly upload a release build with the final set of tags. + + In this mode, the completion of the `gate` jobs will have merged + the code changes, and the role will now have to build and upload + the resulting image to the remote repository. Once uploaded, the + tags will be updated. + + The alternative `promote` method can be thought of as a + "speculative" upload. There is a possibility the `gate` job + uploads layers and creates a temporary tag, but either the + container upload or another co-gating job fails, causing the `gate` + jobs to fail overall. This causes extra uploads, unsued layers and + unused tags that require cleaning up. Since changes have merged + before the `release` pipeline starts, the upload will simply not + run if the gate jobs fail. This avoids uploading or tagging + anything that will not be used. The trade-off is a higher latency + between merging code and publishing final tags. + + Transient network failures can cause upload errors in both cases. + Although the `promote` job may fail, leaving the tag incorrectly + unmodified, the `promote` job's relatively simplicity minimises + potential error. The `release` pipeline does more work, exposing + it to a higher chance of failures such as transient network errors + etc., also resulting in the repository tag being out-of-date. In + both cases developers must pay close attention as failures in these + pipelines are often less noticable than code not merging with a + gate-job failure. + +*Promoting* + +As discussed above, the :zuul:role:`promote-container-image` role is +designed to be used in a `promote` pipeline. It re-tags a previously +uploaded image by copying the temporary change-id based tags made +during upload to the final production tags supplied by +:zuul:rolevar:`build-container-image.container_images.tags`. It is +intended to run very quickly and with no dependencies, so it can run +directly on the Zuul executor. + +Once this role completes, the temporary upload tags are no longer +required. The role removes the change-id tags from the repository in +the registry, and removes any similar change-ids tags. This keeps the +repository tidy in the case that gated changes fail to merge after +uploading their staged images. + +*Dependencies* Use the :zuul:role:`ensure-skopeo` role as well as the :zuul:role:`ensure-docker`, or :zuul:role:`ensure-podman` roles before