diff --git a/doc/source/index.rst b/doc/source/index.rst index e0a8a6a..e2bfaaa 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -35,6 +35,8 @@ permits. :maxdepth: 1 specs/ansible_puppet_apply + specs/apps-site + specs/centralize-release-tagging specs/code-search specs/doc-publishing specs/infra-cloud diff --git a/specs/centralize-release-tagging.rst b/specs/centralize-release-tagging.rst new file mode 100644 index 0000000..1bb35c3 --- /dev/null +++ b/specs/centralize-release-tagging.rst @@ -0,0 +1,322 @@ +:: + + Copyright 2015, Hewlett-Packard Development Company, L.P. + + This work is licensed under a Creative Commons Attribution 3.0 + Unported License. + http://creativecommons.org/licenses/by/3.0/legalcode + +.. + This template should be in ReSTructured text. Please do not delete + any of the sections in this template. If you have nothing to say + for a whole section, just write: "None". For help with syntax, see + http://sphinx-doc.org/rest.html To test out your formatting, see + http://www.tele3.cz/jbar/rest/rest.html + +============================ + Centralize Release Tagging +============================ + +Include the URL of your StoryBoard story: + +https://storyboard.openstack.org/#!/story/2000292 + +We would like to manage release tags through reviews, just as we do +with other changes. Unfortunately, gerrit does not yet support +reviewing tags directly. This spec describes an alternate solution +using a separate git repository, with some manual tools for short-term +use and some ideas about automation for later. + +Problem Description +=================== + +Until now we have encouraged project teams to prepare their own +library releases as new versions of projects were needed. We've +started running into a couple of problems with that, with releases +not coming often enough, or at a bad time in the release cycle, or +with version numbering not being applied consistently, or without +proper announcements. To address these issues, the release management +team is proposing to create a small team of library release managers +to handle the process around all library releases (clients, +non-application projects, middleware, Oslo libraries, etc.). This +will give us a chance to collaborate and review the version numbers +for new releases, as well as stay on top of "stale" libraries with +fixes or features that sit unreleased for a period of time. It will +also be the first step to creating an automated review process for +release tags. + +Proposed Change +=============== + +Create a new git repository with a name like +``openstack/releases``. + +For each set of deliverables, we will use one YAML file in +``openstack/releases`` per release series to hold all of the metadata +for all releases of that deliverable. For each release, we need to +track: + +* the launchpad project name (such as ``oslo.config``) +* the series (Kilo, Liberty, etc.) +* for each repository + * the name (such as ``openstack/oslo.config``) + * the hash of the commit to be tagged +* the version number to use +* highlights for the release notes email (optional) + +We will track this metadata for the history of all releases of the +deliverable, so we can use the same data to render a set of release +history documentation. + +The release tools need the series name to update launchpad, and they +also need the branch name to check the history of tags visible from +that branch to do some validation. Since git tags apply to a commit, +and are not branch-aware, we can use the series name in the filename +and have the tagging script either assume some defaults (if it does +not find a stable branch of that name use master) or recognize which +series is currently on master. + +The file will be named based on the deliverable to be tagged, so +releases for ``liberty`` from the ``openstack/oslo.config`` repository +will have a file in ``openstack/releases`` called +``liberty/oslo.config.yaml``. Releases of the same deliverable from +the ``stable/kilo`` branch will be described by +``kilo/oslo.config.yaml``. + +For example, one version of +``liberty/oslo.config.yaml`` might contain:: + + --- + launchpad: oslo.config + releases: + - version: 1.12.0 + projects: + - repo: openstack/oslo.config + hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39 + +and then for the subsequent release it would be updated to contain:: + + --- + launchpad: oslo.config + releases: + - version: 1.12.1 + projects: + - repo: openstack/oslo.config + hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023 + highlights: > + This release includes the change to stop importing + from the 'oslo' namespace package. + +For deliverables with multiple repositories, the list of projects +would contain all of them. For example, the Neutron deliverable might +be described by ``liberty/neutron.yaml`` containing: + +:: + + --- + launchpad: neutron + releases: + - version: 7.0.0 + projects: + - repo: openstack/neutron + hash: somethingunique + - repo: openstack/neutron-fwaas + hash: somethingunique + - repo: openstack/neutron-lbaas + hash: somethingunique + - repo: openstack/neutron-vpnaas + hash: somethingunique + +For Phase I, we won't have much true automation using these files, and +someone from the release team will need to run a tool to read the file +and apply the appropriate tags. That tool should be a straightforward +modification to the existing ``release_postversion.sh`` script. + +For future phases we can investigate having enough signed data in the +YAML file to let an automated job apply the tag when the request is +approved. + +Alternatives +------------ + +Allow Project Owners to Continue Tagging Releases +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We will continue to have issues with incorrect semver use and poorly +timed releases. + +Have Project Owners Request Releases via Email/IRC +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Rather than going through the bureaucracy of managing the requests via +git we could just use email and IRC as we have been doing. However, +that would not bring us closer to automating releases after the +requests are reviewed, which is our ultimate goal. + +Update Gerrit to Support Reviewing Tags +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Apparently the Gerrit project team is interested in the feature, but +it isn't a high priority. We could consider this a Phase III for the +project if someone from our community becomes available to work on +it. On the other hand, we would need to find another way to track +releases after-the-fact, and tags in one repository do not handle +multi-repo deliverables such as neutron. + +Use Branches +~~~~~~~~~~~~ + +Earlier drafts of this proposal suggested using different branches of +the ``openstack/releases`` repository to manage releases from +different branches of the upstream projects. That forces us to create +all the same branches in the new repository that are needed in any +repository for which releases are being managed. Since not all of them +will use the same branching structure, this is not optimal. + +One File Per Repository +~~~~~~~~~~~~~~~~~~~~~~~ + +Use a file named based on the git repository to be tagged, so +releases from the ``master`` branch of the ``openstack/oslo.config`` +repository would have a file in ``openstack/releases`` called +``openstack/oslo.config/master/releases.yaml``. Releases for the same +repository from the ``stable/kilo`` branch will be described by +``openstack/oslo.config/stable/kilo/releases.yaml`` + +For example, one version of +``openstack/oslo.config/master/releases.yaml`` might contain:: + + -- + series: liberty + hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39 + version: 1.12.0 + +and then for the subsequent release it would be updated to contain:: + + -- + series: liberty + hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023 + version: 1.12.1 + highlights: > + This release includes the change to stop importing + from the 'oslo' namespace package. + +Multi-repo deliverables such as Neutron could use separate files, +submitted together. + +This scheme does not allow us to easily product web pages showing the +release histories. + +Single File With All Branches +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Rather than maintaining a separate file for each branch, we could use +a single file and list all branches in it. This makes it a little more +complicated to detect new changes, though, and has the same problem as +appending all releases to a single file -- the tool that applies the +tags needs to check all of them, and the list will only grow over +time. + +Branch After Project in the Path +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We could use file names like ``oslo.config/kilo.yaml`` instead of +``kilo/oslo.config.yaml``. That would place all of the files from the +same deliverable in a directory together. However, it is more likely +that we will focus on the contents of a series rather than historical +releases of an individual project. + +Record Launchpad Names in the Governance Repository +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There is a separate list of projects in the governance repository, and +we could list some of the data about projects that doesn't change +there. That would require the tool download the relevant files, +though, and would not help us with scripting releases for projects not +under TC governance. + +Implementation +============== + +Assignee(s) +----------- + +Primary assignee: + doug-hellmann + +I'm comfortable setting up a new repository and building in-tree +tools. I may need help with some of the validation job work. + +Gerrit Topic +------------ + +Use Gerrit topic "centralize-release-tagging" for all patches related to this spec. + +.. code-block:: bash + + git-review -t centralize-release-tagging + +Work Items +---------- + +1. Create the ``openstack/releases`` repository, with a README and + template YAML file. +2. Create a new tool (or update an existing script) in + ``openstack-infra/release-tools`` to read the YAML files from + ``openstack/releases`` and run the interactive release script we + use now. +3. Create a basic validation tool to read the YAML files and provide a + check job. We can't do a lot to validate the requested tag, beyond + noticing that it already exists, but we can make sure all of the + needed parts are there and can be parsed properly, and we can run a + report showing the unreleased changes, what pbr thinks the version + should be, and whether the version means a major jump in the series + to help the reviewer (these are all things I do by hand right now). +4. Make ``release_postversion.sh`` smarter about figuring out the + branch for validating the proposed version number. + +Repositories +------------ + +``openstack/releases`` will hold the release request files. + +Servers +------- + +None + +DNS Entries +----------- + +None + +Documentation +------------- + +We will document the process in the README in ``openstack/releases`` +to start, and then in the Project Driver's Guide portion of +infra-manual. + +Security +-------- + +During Phase I releases will still be tagged by people with +established trust rings. For future phases where the tagging is +handled by a post-merge job we will want to do some validation of +signed data in the request file. + +Testing +------- + +We have fairly robust release tools now, but we will want to test some +of the new tools for working with the YAML files. + +Dependencies +============ + +- https://review.openstack.org/189856 -- Creating a library-release + team with Gerrit ACLs to push tags to repositories containing + libraries. + +- http://lists.openstack.org/pipermail/openstack-dev/2015-June/066346.html -- + Mailing list thread initiating the discussion.