zuul/zuul-registry
Code Issues Proposed changes
A speculative container registry
69 Commits 1 Branch 4 Tags 1.1 MiB
Python 97.4%
Dockerfile 1.3%
Shell 1.3%
4213b96d3a
Go to file
Clark Boylan 4213b96d3a Perform atomic upload updates v2 
The way the registry was previously written if two concurrent uploads
of the same blob were happening one would fail to grab the lock and then
return early. The uploading client would then immediately HEAD the blob
and if it did so quickly enough would get a short size or 404. To avoid
this we need the upload to continue until all concurrent uploads are
complete.

To make this happen we treat upload chunks separately per upload so that
separate uploads cannot overwrite the chunks once they are moved to the
blob directory. We end up moving the chunks to the blob directory in
upload specific locations to facilitate this. Once that is done we can
atomically update the actual blob data from the chunks. In the
filesystem driver we concatenate the chunks into the blob then
atomically rename the result into its final blob/data location. This
ensures that we only ever return valid HEAD info for a blob, and that it
is only requested by the client once it exists.

This should be safe because the objects are hashsum addresses which
means their contents should be identical. If we end up moving one copy
into place then another atomically they will always have the same data
and size.

These logs from an OpenDev test job seem to capture this in action:

  # First upload is completing and grabs the lock
  2022-02-25 21:28:14,514 INFO registry.api: [u: 935f8eddbb9a4dab8dd8cc45ce7f9384] Upload final chunk _local opendevorg/gerrit digest sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343
  2022-02-25 21:28:14,576 DEBUG registry.storage: [u: 935f8eddbb9a4dab8dd8cc45ce7f9384] Locked digest sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343

  # Second upload attempts to complete but ends early without the lock
  2022-02-25 21:28:15,517 INFO registry.api: [u: e817d8fd6c464f80bf405581e580cbab] Upload final chunk _local opendevorg/gerrit digest sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343
  2022-02-25 21:28:15,578 WARNING registry.storage: [u: e817d8fd6c464f80bf405581e580cbab] Failed to obtain lock(1) on digest sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343
  2022-02-25 21:28:15,588 INFO registry.api: [u: e817d8fd6c464f80bf405581e580cbab] Upload complete _local opendevorg/gerrit digest sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343
  2022-02-25 21:28:15,589 INFO cherrypy.access.140551593545056: ::ffff:172.17.0.1 - - [25/Feb/2022:21:28:15] "PUT /v2/opendevorg/gerrit/blobs/uploads/e817d8fd6c464f80bf405581e580cbab?digest=sha256%3A0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343 HTTP/1.1" 201 - "" "docker/20.10.12 go/go1.16.12 git-commit/459d0df kernel/5.4.0-100-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.12 \(linux\))"

  # Second upload completion triggers the HEAD requests that is either a
  # 404 or short read. This causes the second upload client to error.
  2022-02-25 21:28:15,605 INFO registry.api: Head blob _local opendevorg/gerrit sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343 not found
  2022-02-25 21:28:15,607 INFO cherrypy.access.140551593545056: ::ffff:172.17.0.1 - - [25/Feb/2022:21:28:15] "HEAD /v2/opendevorg/gerrit/blobs/sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343 HTTP/1.1" 404 735 "" "docker/20.10.12 go/go1.16.12 git-commit/459d0df kernel/5.4.0-100-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.12 \(linux\))"

  # Now first upload has completed and the HEAD request by the first
  # upload client is successful
  2022-02-25 21:28:18,898 INFO registry.api: [u: 935f8eddbb9a4dab8dd8cc45ce7f9384] Upload complete _local opendevorg/gerrit digest sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343
  2022-02-25 21:28:18,898 INFO cherrypy.access.140551593545056: ::ffff:172.17.0.1 - - [25/Feb/2022:21:28:18] "PUT /v2/opendevorg/gerrit/blobs/uploads/935f8eddbb9a4dab8dd8cc45ce7f9384?digest=sha256%3A0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343 HTTP/1.1" 201 - "" "docker/20.10.12 go/go1.16.12 git-commit/459d0df kernel/5.4.0-100-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.12 \(linux\))"
  2022-02-25 21:28:18,915 INFO registry.api: Head blob _local opendevorg/gerrit sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343 size 54917164
  2022-02-25 21:28:18,916 INFO cherrypy.access.140551593545056: ::ffff:172.17.0.1 - - [25/Feb/2022:21:28:18] "HEAD /v2/opendevorg/gerrit/blobs/sha256:0c6b8ff8c37e92eb1ca65ed8917e818927d5bf318b6f18896049b5d9afc28343 HTTP/1.1" 200 54917164 "" "docker/20.10.12 go/go1.16.12 git-commit/459d0df kernel/5.4.0-100-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.12 \(linux\))"

Change-Id: Ibdf1ca554756af61247d705b2ea3cf85c39c2b83
2022-02-28 14:16:03 -08:00
playbooks/functional-test Add a restricted mode (read authentication required) 2021-07-06 17:46:13 -07:00
tests config: add environment variable substitution 2020-04-13 18:13:26 +00:00
tools Do not allow incorrect sized layers 2021-09-14 05:54:50 +00:00
zuul_registry Perform atomic upload updates v2 2022-02-28 14:16:03 -08:00
.gitignore Add README, license, and Zuul config 2019-09-20 09:53:42 -07:00
.gitreview Added .gitreview 2019-09-19 16:08:44 +00:00
.stestr.conf Add tox configuration and fixe flake8 errors 2019-10-02 18:05:12 +00:00
.zuul.yaml Update the registry docker image to bullseye 2021-12-09 15:29:31 -08:00
bindep.txt Fix container image build 2019-10-02 13:50:42 -04:00
COPYING Add README, license, and Zuul config 2019-09-20 09:53:42 -07:00
Dockerfile Update the registry docker image to bullseye 2021-12-09 15:29:31 -08:00
README.rst Add README, license, and Zuul config 2019-09-20 09:53:42 -07:00
requirements.txt Update pyjwt version to >=2.0.0,<3.0.0 2021-02-01 10:46:09 -08:00
setup.cfg Fix container image build 2019-10-02 13:50:42 -04:00
setup.py Initial implementation 2019-10-01 08:09:55 -07:00
test-requirements.txt Add tox configuration and fixe flake8 errors 2019-10-02 18:05:12 +00:00
tox.ini Add tox configuration and fixe flake8 errors 2019-10-02 18:05:12 +00:00

README.rst

Zuul Registry

This is a container image registry for use with the Zuul project gating system.

The defining feature of this registry is support for shadowing images: it allows you to upload a local version of an image to use instead of an upstream version. If you pull an image from this registry, it will provide the local version if it exists, or the upstream if it does not.

This makes it suitable for use in a Zuul-driven speculative image pipeline.

The latest documentation for Zuul is published at: https://zuul-ci.org/docs/

Getting Help

There are two Zuul-related mailing lists:

zuul-announce

A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.

zuul-discuss

General discussion about Zuul, including questions about how to use it, and future development.

You will also find Zuul developers in the #zuul channel on Freenode IRC.

Contributing

To browse the latest code, see: https://opendev.org/zuul/zuul-registry To clone the latest code, use git clone https://opendev.org/zuul/zuul-registry

Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/zuul-registry

Suspected security vulnerabilities are most appreciated if first reported privately following any of the supported mechanisms described at https://zuul-ci.org/docs/zuul/user/vulnerabilities.html

Code reviews are handled by gerrit at https://review.opendev.org

After creating a Gerrit account, use git review to submit patches. Example:

# Do your commits
$ git review
# Enter your username if prompted

Join #zuul on Freenode to discuss development or usage.

License

Zuul-registry is free software licensed under the General Public License, version 3.0.

Python Version Support

Zuul requires Python 3. It does not support Python 2.