CentOS 8 migration spec
Adds a specification covering the miration from CentOS 7 to 8. Change-Id: Iaff5fc10eec3aa3f5068d78b121a6bc0f08cd61d Related: blueprint centos-rhel-8
This commit is contained in:
Mark Goddard
parent
da105c20ee
commit
73bdec833f
301
specs/centos8-migration.rst
Normal file
301
specs/centos8-migration.rst
Normal file
@ -0,0 +1,301 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
..
|
||||
This template should be in ReSTructured text. The filename in the git
|
||||
repository should match the launchpad URL, for example a URL of
|
||||
https://blueprints.launchpad.net/kolla/+spec/awesome-thing should be named
|
||||
awesome-thing.rst . Please do not delete any of the sections in this
|
||||
template. If you have nothing to say for a whole section, just write: None
|
||||
For help with syntax, see http://www.sphinx-doc.org/en/stable/rest.html
|
||||
To test out your formatting, see http://www.tele3.cz/jbar/rest/rest.html
|
||||
|
||||
==================
|
||||
CentOS 8 Migration
|
||||
==================
|
||||
|
||||
https://blueprints.launchpad.net/kolla/+spec/centos-rhel-8
|
||||
https://blueprints.launchpad.net/kolla-ansible/+spec/centos-rhel-8
|
||||
|
||||
CentOS 8 is here, and RDO support for CentOS 7 is going away.
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
This spec is largely focussed on the issue of migrating running systems from
|
||||
CentOS 7 to CentOS 8. We will not in general look at the details of porting
|
||||
Kolla to CentOS 8 here.
|
||||
|
||||
Kolla images
|
||||
------------
|
||||
|
||||
Train is the last release of OpenStack to support Python 2. Since CentOS 7
|
||||
provides only limited support for Python 3, RDO plans to support only CentOS 8
|
||||
for the Ussuri cycle.
|
||||
|
||||
While RDO Train initially only provided Python 2 based RPMs for CentOS 7, it is
|
||||
expected that Python 3 based Train RPMs will be made available for CentOS 8. We
|
||||
should therefore support building Train images with either CentOS 7 or CentOS 8
|
||||
as a base. Both sets of images should be published to Dockerhub. The image
|
||||
naming scheme will need to take account of the OS version, which it currently
|
||||
does not.
|
||||
|
||||
Kolla Ansible
|
||||
-------------
|
||||
|
||||
There is no supported upgrade path between CentOS 7 and 8 hosts - a full
|
||||
reinstall is necessary. This provides us with one release (Train) that supports
|
||||
both CentOS 7 and 8, with which we could perform a rolling reinstallation of
|
||||
hosts. The cloud should continue to operate in this mixed 7/8 mode during the
|
||||
migration. The following diagram shows an example with 6 hosts h1 to h6 being
|
||||
migrated in batches of two::
|
||||
|
||||
h1 h2 h3 h4 h5 h6
|
||||
-------------------
|
||||
t0 | c7 c7 c7 c7 c7 c7
|
||||
t1 | c8 c8 c7 c7 c7 c7
|
||||
t2 | c8 c8 c8 c8 c7 c7
|
||||
t3 | c8 c8 c8 c8 c8 c8
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
Kolla images - Ussuri
|
||||
---------------------
|
||||
|
||||
Support for building CentOS 8 based container images will be added to Kolla
|
||||
during the Ussuri cycle. Initially, this will exist in parallel with CentOS 7
|
||||
support, and be controlled via the ``distro-package-manager`` configuration
|
||||
option, which takes its default value from the base image tag (``7.*`` for
|
||||
``yum``, ``8.*`` for ``dnf``).
|
||||
|
||||
New build and publishing CI jobs will be added for CentOS 8. We will use an
|
||||
alternative Docker image tag to tag CentOS 8 images on Dockerhub:
|
||||
``master-centos8``. An example image name is
|
||||
``kolla/centos-binary-base:master-centos8``.
|
||||
|
||||
Once this work is complete, support for CentOS 7 will be removed. CentOS 8
|
||||
images will then be tagged as ``master`` (or ``ussuri`` after branching). This
|
||||
two-phase approach allows for a clean backport to Train. A downside is that
|
||||
``master-centos8`` tags will continue to exist in Dockerhub after the
|
||||
transition, which could be confusing for users.
|
||||
|
||||
Kolla images - Train
|
||||
--------------------
|
||||
|
||||
The CentOS 8 support developed for Ussuri will be backported to the
|
||||
stable/train branch. Since these changes are significant, we should advertise
|
||||
this clearly to users who might not be expecting it on a stable branch.
|
||||
Train images will be built and published for CentOS 7 and 8, with CentOS 8
|
||||
images using a tag of ``train-centos8``.
|
||||
|
||||
Kolla Ansible - Ussuri
|
||||
----------------------
|
||||
|
||||
Support for deploying CentOS 8 based container images will be added to Kolla
|
||||
Ansible during the Ussuri cycle. Initially, this will exist in parallel with
|
||||
CentOS 7 support, and be controlled via Ansible host facts about the host OS
|
||||
distribution. We will deploy CentOS 7 containers on CentOS 7 hosts, and CentOS
|
||||
8 containers on CentOS 8 hosts to avoid kernel and userspace incompatibility
|
||||
issues. This also aligns with the `migration plan
|
||||
<http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011133.html>`__
|
||||
proposed by Tripleo.
|
||||
|
||||
During the migration, we may have a mix of CentOS 7 hosts running CentOS 7
|
||||
containers, and CentOS 8 hosts running CentOS 8 containers.
|
||||
|
||||
The image tag used will need to depend on the host OS. We have a hierarchy of
|
||||
image tag variables, for example: the ``nova-compute`` container uses a tag
|
||||
defined by ``nova_compute_tag``, which defaults to ``nova_tag``, which defaults
|
||||
to ``openstack_release``.
|
||||
|
||||
To preserve the usage of ``openstack_release``, a new intermediate variable
|
||||
will be introduced. In this new scheme, the ``nova-compute`` container uses a
|
||||
tag defined by ``nova_compute_tag``, which defaults to ``nova_tag``, which
|
||||
defaults to ``openstack_tag``, which defaults to ``openstack_release ~
|
||||
'-centos8'`` on CentOS 8, or ``openstack_release`` otherwise. Providing this
|
||||
suffix as a variable would help users who are using more fine-grained image
|
||||
tags (e.g. ``nova_compute_tag``).
|
||||
|
||||
Once this work is complete, support for CentOS 7 will be removed. The
|
||||
intermediate variable ``openstack_tag`` will remain, but will always default to
|
||||
``openstack_release``.
|
||||
|
||||
Kolla Ansible - Train
|
||||
---------------------
|
||||
|
||||
The CentOS 8 support developed for Ussuri will be backported to the
|
||||
stable/train branch. Since these changes are significant, we should advertise
|
||||
this clearly to users who might not be expecting it on a stable branch.
|
||||
|
||||
Kolla Ansible - Rolling Migration
|
||||
---------------------------------
|
||||
|
||||
In order to keep the cloud functional during the migration, the changes will be
|
||||
rolled through in batches. The size and scope of the batches are to be
|
||||
determined by the operator, but we should provide some guidance on selecting
|
||||
batches safely to avoid losing quorum of clustered services or reducing service
|
||||
availability below some minimum threshold. Provisioning of the new OS is out of
|
||||
the scope of this discussion, but could be handled by Kayobe or another tool.
|
||||
|
||||
We should treat each batch as a removal of a set of hosts followed by an
|
||||
addition of new set of hosts. This is a good opportunity for us to tidy up
|
||||
these procedures and fill in gaps where necessary.
|
||||
|
||||
To remove a batch of hosts:
|
||||
|
||||
#. Migrate running OpenStack resources from the hosts to other hosts which are
|
||||
not in the batch. This includes compute instances (VMs), storage (Swift
|
||||
disks), network services (DHCP & L3 agents etc.). Ideally this would be
|
||||
automated, but should at least be discussed in documentation
|
||||
#. Cleanly disable (if necessary) and shutdown running OpenStack services
|
||||
#. Shutdown hosts
|
||||
|
||||
The hosts should then be provisioned with CentOS 8 - as mentioned this is out
|
||||
of scope. Then to add a batch of hosts:
|
||||
|
||||
#. Provision CentOS 8 to hosts (out of scope)
|
||||
#. Bootstrap hosts
|
||||
#. Deploy services
|
||||
#. Validate new hosts
|
||||
|
||||
We may wish to adopt existing data on these hosts, e.g. for Swift a full sync
|
||||
from empty for each node would take a long time. For systems where Docker
|
||||
volumes are stored on a separate disk or partition, it may be possible to
|
||||
preserve these. There may be some corner cases where this does not work well,
|
||||
and we should check for these and advise in documentation.
|
||||
|
||||
For the majority of these tasks it should be possible to use the ``--limit``
|
||||
argument to speed up the process, but this should be investigated and verified.
|
||||
|
||||
Stable upgrades
|
||||
---------------
|
||||
|
||||
Where possible, we should aim to use the same versions of packages on CentOS 7
|
||||
and 8 for Train. Upgrades should wait until Ussuri. In some cases this may not
|
||||
be possible, and we should make it clear in release notes where services will
|
||||
be upgraded in Train.
|
||||
|
||||
Given that a rolled migration may take a significant amount of time to
|
||||
complete, we should consider that services may be partially upgraded. In cases
|
||||
where this may cause a problem, it should be called out in the documentation,
|
||||
with any available mitigation (e.g. disable during migration).
|
||||
|
||||
Feature removal
|
||||
---------------
|
||||
|
||||
Due to differences between CentOS 7 and 8, it may not be possible or practical
|
||||
to support all existing features. We should bear in mind that this will apply
|
||||
to the Train release, and continue to support these features on CentOS 7.
|
||||
|
||||
Ceph support
|
||||
^^^^^^^^^^^^
|
||||
|
||||
Migrating a Ceph cluster deployed by Kolla Ansible to CentOS 8 would represent
|
||||
a significant challenge. Ceph deployment support has been deprecated since the
|
||||
Stein release, and this is a good point to remove that support.
|
||||
|
||||
SCSI target daemon
|
||||
^^^^^^^^^^^^^^^^^^
|
||||
|
||||
For CentOS 7, the SCSI target daemon (``tgtd``) is provided by EPEL. For CentOS
|
||||
8, this is no longer available, probably due to a preference for the kernel
|
||||
SCSI target, `LIO <http://linux-iscsi.org/wiki/LIO>`__. Due to the requirement
|
||||
to reinstall, this migration provides a good opportunity to stop supporting
|
||||
``tgtd`` for CentOS. This will affect some Cinder volume drivers, including the
|
||||
LVM backend. ``tgtd`` will still be supported on Debian/Ubuntu. Note that no
|
||||
changes are required for the iSCSI initiator.
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
|
||||
There should be no security impact due to this change.
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
There should be no performance impact due to this change.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
CentOS 7 to 8 upgrade
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
There are various documented procedures for upgrading from CentOS 7 to 8
|
||||
without a reinstall. These typically come with various disclaimers, and are not
|
||||
recommended for production environments.
|
||||
|
||||
Image naming
|
||||
^^^^^^^^^^^^
|
||||
|
||||
Rather than using a different tag to mark CentOS 8 images, it would be possible
|
||||
to use a base distro of ``centos8``. e.g. ``kolla/centos8-binary-base:master``.
|
||||
The reason for not doing this is that there is a reasonable amount of logic
|
||||
both in Kolla and Kolla Ansible that uses the ``base_distro`` which would need
|
||||
to be updated.
|
||||
|
||||
Don't publish master-centos8 tag
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
During the transition period, we could manage with always building images for
|
||||
CentOS 8 on master, thus avoiding the need for a temporary ``master-centos8``
|
||||
tag.
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignees:
|
||||
Marcin Juszkiewicz (hrw)
|
||||
Mark Goddard (mgoddard)
|
||||
Michal Nasiadka (mnasiadka)
|
||||
Radoslaw Piliszek (yoctozepto)
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
Kolla
|
||||
^^^^^
|
||||
|
||||
* Support building CentOS 8 images in Ussuri
|
||||
* Build and publish both CentOS 7 and 8 images in CI
|
||||
* Backport CentOS 8 support to Train
|
||||
* Remove CentOS 7 image support in Ussuri
|
||||
* Update documentation (image tags etc.)
|
||||
|
||||
Kolla Ansible
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
* Support deploying CentOS 8 images in Ussuri
|
||||
* Test (separately) deployment to both CentOS 7 and 8 in CI
|
||||
* Backport CentOS 8 support to Train
|
||||
* Test migration from CentOS 7 to 8 on Train in CI
|
||||
* Test upgrade from Train to Ussuri on CentOS 8
|
||||
* Remove CentOS 7 support in Ussuri
|
||||
* Update documentation (migration)
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
This will be tested by parallel build and deploy jobs for CentOS 7 and 8, in
|
||||
addition to a job testing migration from CentOS 7 to 8. Significant manual
|
||||
testing will be required to explore edge cases associated with the migration.
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
* Building CentOS 8 images
|
||||
* Deploying to CentOS 8
|
||||
* Image tags for CentOS 8
|
||||
* Migration procedure covered in detail
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
* `CentOS 8 release notes <https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.1905>`__
|
||||
Loading…
Reference in New Issue
Block a user