starlingx/docs
Code Issues Proposed changes

Upgrades edits (r6,r7,dsR6,dsR7)

Browse Source 
Copy edits for typos, markup and other technical issues.
Fix case typo
Fix merge conflicts.
Resolve FixMe issues.
Minor copy edit updates.

Signed-off-by: Ron Stone <ronald.stone@windriver.com>
Change-Id: I7d710d3faa957c1cb66cc18c9a248a567e7c580f
This commit is contained in:
Ron Stone 2022-08-18 14:14:14 -04:00
parent 822a273fbd
commit dd8efe5f85
37 changed files with 668 additions and 661 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
doc/source/updates/kubernetes/aborting-simplex-system-upgrades.rst
View File

@ -9,11 +9,11 @@ Abort Simplex System Upgrades
You can abort a Simplex System upgrade before or after upgrading controller-0. You can abort a Simplex System upgrade before or after upgrading controller-0.
The upgrade abort procedure can only be applied before the The upgrade abort procedure can only be applied before the
:command:`upgrade-complete` command is issued. Once this command is issued the :command:`upgrade-complete` command is issued. Once this command is issued the
upgrade can not be aborted. If the return to the previous release is required, upgrade can not be aborted. If you must return to the previous release, then
then restore the system using the backup data taken prior to the upgrade. restore the system using the backup data taken prior to the upgrade.
Before starting, verify the upgrade data under `/opt/platform-backup`. This data Before starting, verify the upgrade data under ``/opt/platform-backup``. This
must be present to perform the abort process. data must be present to perform the abort process.
.. _aborting-simplex-system-upgrades-section-N10025-N1001B-N10001: .. _aborting-simplex-system-upgrades-section-N10025-N1001B-N10001:
@ -31,20 +31,20 @@ Before upgrading controller-0
.. code-block:: none .. code-block:: none
$ system upgrade-abort ~(keystone_admin)$ system upgrade-abort
The upgrade state is set to aborting. Once this is executed, there is no The upgrade state is set to ``aborting``. Once this is executed, it cannot
canceling; the upgrade must be completely aborted. be cancelled; the upgrade must be completely aborted.
#. Complete the upgrade. #. Complete the upgrade.
.. code-block:: none .. code-block:: none
$ system upgrade-complete ~(keystone_admin)$ system upgrade-complete
At this time any upgrade data generated as part of the upgrade-start At this time any upgrade data generated as part of the upgrade-start
command will be deleted. This includes the upgrade data in command will be deleted. This includes the upgrade data in
/opt/platform-backup. ``/opt/platform-backup``.
.. _aborting-simplex-system-upgrades-section-N10063-N1001B-N10001: .. _aborting-simplex-system-upgrades-section-N10063-N1001B-N10001:
@ -52,7 +52,7 @@ Before upgrading controller-0
After upgrading controller-0 After upgrading controller-0
---------------------------- ----------------------------
After controller-0 has been upgraded it is possible to roll back the software After controller-0 has been upgraded, it is possible to roll back the software
upgrade. This involves performing a system restore with the previous release. upgrade. This involves performing a system restore with the previous release.
.. _aborting-simplex-system-upgrades-ol-jmw-kcp-xdb: .. _aborting-simplex-system-upgrades-ol-jmw-kcp-xdb:
@ -61,41 +61,42 @@ upgrade. This involves performing a system restore with the previous release.
USB. USB.
#. Verify and configure IP connectivity. External connectivity is required to #. Verify and configure IP connectivity. External connectivity is required to
run the Ansible restore playbook. The |prod-long| boot image will DHCP out all run the Ansible restore playbook. The |prod-long| boot image will |DHCP| out
interfaces so the server may have obtained an IP address and have external IP all interfaces so the server may have obtained an IP address and have
connectivity if a DHCP server is present in your environment. Verify this using external IP connectivity if a |DHCP| server is present in your environment.
the :command:`ip addr` command. Otherwise, manually configure an IP address and default IP Verify this using the :command:`ip addr` command. Otherwise, manually
route. configure an IP address and default IP route.
#. Restore the system data. The restore is preserved in /opt/platform-backup. #. Restore the system data. The restore is preserved in ``/opt/platform-backup``.
The system will be restored to the state when the :command:`upgrade-start` The system will be restored to the state when the :command:`upgrade-start`
command was issued. Follow the process in :ref:`Run Restore Playbook Locally on the command was issued. Follow the process in :ref:`Run Restore Playbook Locally
Controller <running-restore-playbook-locally-on-the-controller>`. on the Controller <running-restore-playbook-locally-on-the-controller>`.
Specify the upgrade data filename as `backup_filename` and the `initial_backup_dir` Specify the upgrade data filename as `backup_filename` and the
as `/opt/platform-backup`. `initial_backup_dir` as ``/opt/platform-backup``.
The user images will also need to be restored as described in the Postrequisites section. The user images will also need to be restored as described in the
Postrequisites section.
#. Unlock controller-0 #. Unlock controller-0
.. code-block:: none .. code-block:: none
$ system host-unlock controller-0 ~(keystone_admin)$ system host-unlock controller-0
#. Abort the upgrade with the :command:`upgrade-abort` command. #. Abort the upgrade with the :command:`upgrade-abort` command.
.. code-block:: none .. code-block:: none
$ system upgrade-abort ~(keystone_admin)$ system upgrade-abort
The upgrade state is set to aborting. Once this is executed, there is no The upgrade state is set to ``aborting``. Once this is executed, it cannot
canceling; the upgrade must be completely aborted. be cancelled; the upgrade must be completely aborted.
#. Complete the upgrade. #. Complete the upgrade.
.. code-block:: none .. code-block:: none
$ system upgrade-complete ~(keystone_admin)$ system upgrade-complete

16
doc/source/updates/kubernetes/configure-firmware-update-orchestration.rst
View File

@ -7,12 +7,12 @@ Configure Firmware Update Orchestration
======================================= =======================================
You can configure *Firmware Update Orchestration Strategy* using the You can configure *Firmware Update Orchestration Strategy* using the
**sw-manager** |CLI|. :command:`sw-manager` |CLI|.
.. note:: .. note::
Management-affecting alarms cannot be ignored using relaxed alarm rules Management-affecting alarms cannot be ignored using relaxed alarm rules
during an orchestrated firmware update operation. For a list of during an orchestrated firmware update operation. For a list of
management-affecting alarms, see |prod| Fault Management: management-affecting alarms, see |fault-doc|:
:ref:`Alarm Messages <100-series-alarm-messages>`. To display :ref:`Alarm Messages <100-series-alarm-messages>`. To display
management-affecting active alarms, use the following command: management-affecting active alarms, use the following command:
@ -37,9 +37,9 @@ ignored even when the default strict restrictions are selected:
.. _noc1590162360081-ul-ls2-pxs-tlb: .. _noc1590162360081-ul-ls2-pxs-tlb:
- Hosts that need to be updated must be in the **unlocked-enabled** state. - Hosts that need to be updated must be in the ``unlocked-enabled`` state.
- The firmware update image must be in the **applied** state. For more - The firmware update image must be in the ``applied`` state. For more
information, see :ref:`Managing Software Updates <managing-software-updates>`. information, see :ref:`Managing Software Updates <managing-software-updates>`.
.. rubric:: |proc| .. rubric:: |proc|
@ -69,7 +69,7 @@ ignored even when the default strict restrictions are selected:
state: building state: building
inprogress: true inprogress: true
#. Optional: Display the strategy in summary, if required. The firmware update #. |Optional| Display the strategy in summary, if required. The firmware update
strategy :command:`show` command displays the strategy in a summary. strategy :command:`show` command displays the strategy in a summary.
.. code-block:: none .. code-block:: none
@ -87,7 +87,7 @@ ignored even when the default strict restrictions are selected:
state: ready-to-apply state: ready-to-apply
build-result: success build-result: success
The strategy steps and stages are displayed using the **--details** option. The strategy steps and stages are displayed using the ``--details`` option.
#. Apply the strategy. #. Apply the strategy.
@ -96,7 +96,7 @@ ignored even when the default strict restrictions are selected:
all the hosts in the strategy is complete. all the hosts in the strategy is complete.
- Use the **-stage-id** option to specify a specific stage to apply; one - Use the ``-stage-id`` option to specify a specific stage to apply; one
at a time. at a time.
.. note:: .. note::
@ -138,7 +138,7 @@ ignored even when the default strict restrictions are selected:
state: applying state: applying
inprogress: true inprogress: true
#. Optional: Abort the strategy, if required. This is only used to stop, and #. |optional| Abort the strategy, if required. This is only used to stop, and
abort the entire strategy. abort the entire strategy.
The firmware update strategy :command:`abort` command can be used to abort The firmware update strategy :command:`abort` command can be used to abort

15
doc/source/updates/kubernetes/configuring-kubernetes-update-orchestration.rst
View File

@ -12,7 +12,7 @@ You can configure *Kubernetes Version Upgrade Orchestration Strategy* using the
.. note:: .. note::
You require administrator privileges to use :command:`sw-manager`. You must You require administrator privileges to use :command:`sw-manager`. You must
log in to the active controller as **user sysadmin** and source the script log in to the active controller as **user sysadmin** and source the script
by using the command, source /etc/platform/openrc to obtain administrator by using the command, source ``/etc/platform/openrc`` to obtain administrator
privileges. Do not use :command:`sudo`. privileges. Do not use :command:`sudo`.
.. note:: .. note::
@ -75,9 +75,10 @@ For example:
- Hosts that need to be upgraded must be in the ``unlocked-enabled`` state. - Hosts that need to be upgraded must be in the ``unlocked-enabled`` state.
- If you are using NetApp Trident, ensure that your NetApp version is - If you are using NetApp Trident, ensure that your NetApp version is
compatible with Trident 22.01 before upgrading Kubernetes to version |kube-ver| compatible with Trident 22.01 before upgrading Kubernetes to version
and after updating |prod| to version |prod-ver|. For more information, |kube-ver| and after updating |prod| to version |prod-ver|. For more
see :ref:`Upgrade the NetApp Trident Software <upgrade-the-netapp-trident-software-c5ec64d213d3>`. information, see :ref:`Upgrade the NetApp Trident Software
<upgrade-the-netapp-trident-software-c5ec64d213d3>`.
.. only:: partner .. only:: partner
@ -104,7 +105,7 @@ For example:
#. Confirm that the system is healthy. #. Confirm that the system is healthy.
Check the current system health status, resolve any alarms and other issues Check the current system health status, resolve any alarms and other issues
reported by the :command:`system health-query-kube-upgrade` command then reported by the :command:`system health-query-kube-upgrade` command, then
recheck the system health status to confirm that all **System Health** recheck the system health status to confirm that all **System Health**
fields are set to **OK**. fields are set to **OK**.
@ -272,7 +273,7 @@ For example:
defaults to strict defaults to strict
#. Optional: Display the strategy in summary, if required. The Kubernetes #. |optional| Display the strategy in summary, if required. The Kubernetes
upgrade strategy :command:`show` command displays the strategy in a summary. upgrade strategy :command:`show` command displays the strategy in a summary.
.. code-block:: none .. code-block:: none
@ -350,7 +351,7 @@ For example:
``downloading-images``, ``downloaded-images``, ``upgrading-first-master``, ``downloading-images``, ``downloaded-images``, ``upgrading-first-master``,
``upgraded-first-master``, etc. ``upgraded-first-master``, etc.
#. Optional: Abort the strategy, if required. This is only used to stop, and #. |optional| Abort the strategy, if required. This is only used to stop, and
abort the entire strategy. abort the entire strategy.
The Kubernetes version upgrade strategy :command:`abort` command can be The Kubernetes version upgrade strategy :command:`abort` command can be

12
doc/source/updates/kubernetes/configuring-update-orchestration.rst
View File

@ -156,14 +156,14 @@ status before creating a update strategy.
- Worker hosts with no hosted application pods are updated before - Worker hosts with no hosted application pods are updated before
worker hosts with hosted application pods. worker hosts with hosted application pods.
- The final step in each stage is "system-stabilize," which waits for - The final step in each stage is ``system-stabilize``, which waits
a period of time \(up to several minutes\) and ensures that the for a period of time \(up to several minutes\) and ensures that the
system is free of alarms. This ensures that the update orchestrator system is free of alarms. This ensures that the update orchestrator
does not continue to update more hosts if the update application does not continue to update more hosts if the update application has
has caused an issue resulting in an alarm. caused an issue resulting in an alarm.
#. Click the **Apply Strategy** button to apply the update- strategy. You can #. Click the **Apply Strategy** button to apply the update strategy. You can
optionally apply a single stage at a time by clicking the **Apply Stage** optionally apply a single stage at a time by clicking the **Apply Stage**
button. button.
@ -181,7 +181,7 @@ status before creating a update strategy.
attempt to unlock any hosts that were locked. attempt to unlock any hosts that were locked.
.. note:: .. note::
If a update-strategy is aborted after hosts were locked, but before If a update strategy is aborted after hosts were locked, but before
they were updated, the hosts will not be unlocked, as this would result they were updated, the hosts will not be unlocked, as this would result
in the updates being installed. You must either install the updates on in the updates being installed. You must either install the updates on
the hosts or remove the updates before unlocking the hosts. the hosts or remove the updates before unlocking the hosts.

6
doc/source/updates/kubernetes/firmware-update-operations-requiring-manual-migration.rst
View File

@ -15,13 +15,13 @@ Do the following to manage the instance re-location manually:
.. _rbp1590431075472-ul-mgr-kvs-tlb: .. _rbp1590431075472-ul-mgr-kvs-tlb:
- Manually firmware update at least one openstack-compute worker host. This - Manually firmware-update at least one openstack-compute worker host. This
assumes that at least one openstack-compute worker host does not have any assumes that at least one openstack-compute worker host does not have any
instances, or has instances that can be migrated. For more information on instances, or has instances that can be migrated. For more information on
manually updating a host, see the :ref:`Display Worker Host Information manually updating a host, see the :ref:`Display Worker Host Information
<displaying-worker-host-information>`. <displaying-worker-host-information>`.
- If the migration is prevented by limitations in the VNF or virtual - If the migration is prevented by limitations in the |VNF| or virtual
application, perform the following: application, perform the following:
@ -35,7 +35,7 @@ Do the following to manage the instance re-location manually:
- Terminate the old instances. - Terminate the old instances.
- If the migration is prevented by the size of the instances local disks: - If the migration is prevented by the size of the instances' local disks:
- For each openstack-compute worker host that has instances that cannot - For each openstack-compute worker host that has instances that cannot
be migrated, manually move the instances using the CLI. For more be migrated, manually move the instances using the CLI. For more

44
doc/source/updates/kubernetes/firmware-update-orchestration-using-the-cli.rst
View File

@ -7,23 +7,24 @@ Firmware Update Orchestration Using the CLI
=========================================== ===========================================
You can configure the *Firmware Update Orchestration Strategy* using the You can configure the *Firmware Update Orchestration Strategy* using the
**sw-manager** |CLI|. :command:`sw-manager`` |CLI| commands.
--------------- ---------------
About this task About this task
--------------- ---------------
.. note:: .. note::
You require administrator privileges to use **sw-manager**. You must log in
to the active controller as **user sysadmin** and source the script by You require administrator privileges to use :command:`sw-manager` commands.
using the command, source /etc/platform/openrc to obtain administrator You must log in to the active controller as **user sysadmin** and source the
privileges. Do not use sudo. script by using the command, source ``/etc/platform/openrc`` to obtain
administrator privileges. Do not use sudo.
.. note:: .. note::
Management-affecting alarms cannot be ignored at the indicated severity Management-affecting alarms cannot be ignored at the indicated severity
level or higher by using relaxed alarm rules during an orchestrated level or higher by using relaxed alarm rules during an orchestrated
firmware update operation. For a list of management-affecting alarms, see firmware update operation. For a list of management-affecting alarms, see
|prod| Fault Management: :ref:`Alarm Messages |fault-doc|: :ref:`Alarm Messages
<100-series-alarm-messages>`. To display management-affecting active <100-series-alarm-messages>`. To display management-affecting active
alarms, use the following command: alarms, use the following command:
@ -72,9 +73,8 @@ be created with override worker apply type concurrency with a max host
parallelism, instance action, and alarm restrictions. parallelism, instance action, and alarm restrictions.
``--controller-apply-type`` and ``--storage-apply-type`` ``--controller-apply-type`` and ``--storage-apply-type``
These options cannot be changed from ``ignore`` because firmware update is
These options cannot be changed from '**ignore**' because firmware update only supported for worker hosts.
is only supported for worker hosts.
.. note:: .. note::
Firmware update is currently only supported for hosts with worker Firmware update is currently only supported for hosts with worker
@ -82,7 +82,6 @@ is only supported for worker hosts.
rejected. rejected.
``--worker-apply-type`` ``--worker-apply-type``
This option specifies the host concurrency of the firmware update strategy: This option specifies the host concurrency of the firmware update strategy:
- ``serial`` \(default\): worker hosts will be patched one at a time - ``serial`` \(default\): worker hosts will be patched one at a time
@ -96,18 +95,17 @@ This option specifies the host concurrency of the firmware update strategy:
- ``ignore``: worker hosts will not be updated; strategy create will fail - ``ignore``: worker hosts will not be updated; strategy create will fail
Worker hosts with no instances are updated before worker hosts with instances. Worker hosts with no instances are updated before worker hosts with
instances.
``--max-parallel-worker-hosts`` ``--max-parallel-worker-hosts``
This option applies to the parallel worker apply type selection to specify This option applies to the parallel worker apply type selection to specify
the maximum worker hosts to update in parallel \(minimum: 2, maximum: 10\). the maximum worker hosts to update in parallel \(minimum: 2, maximum: 10\).
``-–instance-action`` ``-–instance-action``
This option only has significance when the |prefix|-openstack application is This option only has significance when the |prefix|-openstack application is
loaded and there are instances running on worker hosts. It specifies how loaded and there are instances running on worker hosts. It specifies how the
the strategy deals with worker host instances over the strategy execution. strategy deals with worker host instances over the strategy execution.
- ``stop-start`` (default) - ``stop-start`` (default)
@ -128,7 +126,6 @@ the strategy deals with worker host instances over the strategy execution.
to reboot patching only\). to reboot patching only\).
``--alarm-restrictions`` ``--alarm-restrictions``
This option sets how the how the firmware update orchestration behaves when This option sets how the how the firmware update orchestration behaves when
alarms are present. alarms are present.
@ -203,11 +200,10 @@ of the strategy. A complete view of the strategy can be shown using the
Firmware update orchestration strategy apply Firmware update orchestration strategy apply
-------------------------------------------- --------------------------------------------
The ``apply`` strategy subcommand with no options executes the firmware The ``apply`` strategy subcommand with no options executes the firmware update
update strategy from current state to the end. The apply strategy operation can strategy from current state to the end. The apply strategy operation can be
be called with the ``stage-id`` option to execute the next stage of the called with the ``stage-id`` option to execute the next stage of the strategy.
strategy. The ``stage-id`` option cannot be used to execute the strategy out of The ``stage-id`` option cannot be used to execute the strategy out of order.
order.
.. code-block:: none .. code-block:: none
@ -226,9 +222,9 @@ Firmware update orchestration strategy abort
The ``abort`` strategy subcommand with no options sets the strategy to abort The ``abort`` strategy subcommand with no options sets the strategy to abort
after the current applying stage is complete. The abort strategy operation can after the current applying stage is complete. The abort strategy operation can
be called with the ``stage-id`` option to specify that the strategy abort be called with the ``stage-id`` option to specify that the strategy abort before
before executing the next stage of the strategy. The ``stage-id`` option cannot executing the next stage of the strategy. The ``stage-id`` option cannot be used
be used to execute the strategy out of order. to execute the strategy out of order.
.. code-block:: none .. code-block:: none

60
doc/source/updates/kubernetes/handle-firmware-update-orchestration-failures.rst
View File

@ -6,9 +6,8 @@
Handle Firmware Update Orchestration Failures Handle Firmware Update Orchestration Failures
============================================= =============================================
The creation or application of a strategy could fail for any of the listed The creation or application of a strategy could fail for any of the reasons
reasons described in this section. Follow the suggested actions in each case to listed below. Follow the suggested actions in each case to resolve the issue.
resolve the issue.
------------------------- -------------------------
Strategy creation failure Strategy creation failure
@ -20,31 +19,31 @@ Strategy creation failure
- **Action**: - **Action**:
- verify that the **--worker-apply-type** was not set to '**ignore**' - Verify that the ``--worker-apply-type`` was not set to ``ignore``.
- check recent logs added to /var/log/nfv-vim.log - Check recent logs added to ``/var/log/nfv-vim.log``.
- **Reason**: alarms from platform are present - **Reason**: alarms from platform are present
- **Action**: - **Action**:
- query for management affecting alarms and take actions to clear them - Query for management affecting alarms and take actions to clear them.
.. code-block:: none .. code-block:: none
~(keystone_admin)$ fm alarm-list --mgmt_affecting ~(keystone_admin)$ fm alarm-list --mgmt_affecting
- if there are no management affecting alarms present take actions to - If there are no management affecting alarms present take actions to
clear other reported alarms or try creating the strategy with the clear other reported alarms or try creating the strategy with the
'**relaxed**' alarms restrictions option **--alarm-restrictions '**relaxed**' alarms restrictions option ``--alarm-restrictions
relaxed** relaxed``.
- **Reason**: no firmware update required - **Reason**: no firmware update required
- **Action**: - **Action**:
- verify that the firmware device image has been applied for the - Verify that the firmware device image has been applied for the
worker hosts that require updating worker hosts that require updating.
.. note:: .. note::
If the strategy create failed. After resolving the strategy If the strategy create failed. After resolving the strategy
@ -57,52 +56,53 @@ Strategy apply failure
.. _jkf1590184623714-ul-rdf-4pq-5lb: .. _jkf1590184623714-ul-rdf-4pq-5lb:
- **Reason**: alarms from platform are present - **Reason**: Alarms from platform are present.
- **Action**: suggests that an alarm has been raised since the creation - **Action**: This suggests that an alarm has been raised since the
of the strategy. Address the cause of the new alarm, delete the strategy creation of the strategy. Address the cause of the new alarm, delete the
and try creating and applying a new strategy strategy and try creating and applying a new strategy.
- **Reason**: unable to migrate instances - **Reason**: Unable to migrate instances.
- **Action**: See :ref:`Firmware Update Operations Requiring Manual - **Action**: See :ref:`Firmware Update Operations Requiring Manual
Migration <firmware-update-operations-requiring-manual-migration>` for Migration <firmware-update-operations-requiring-manual-migration>` for
steps to resolve migration issues. steps to resolve migration issues.
- **Reason**: firmware update failed. Suggests that the firmware update for - **Reason**: Firmware update failed. Suggests that the firmware update for
the specified host has failed the specified host has failed
- **Action**: For more information, see |prod| Node Management: - **Action**: For more information, see |prod| Node Management:
:ref:`Display Worker Host Information <displaying-worker-host-information>` :ref:`Display Worker Host Information <displaying-worker-host-information>`
- **Reason**: lock host failed - **Reason**: Lock host failed.
- **Action**: - **Action**:
- investigate the /var/log/sysinv.log, and /var/log/nfv-vim.log files - Investigate the ``/var/log/sysinv.log``, and
``/var/log/nfv-vim.log`` files.
- address the underlying issue - Address the underlying issue.
- manually lock and unlock the host - Manually lock and unlock the host.
- try recreating and re-applying the firmware update strategy to - Try recreating and re-applying the firmware update strategy to
automatically finish the update process automatically finish the update process.
- **Reason**: unlock host failed - **Reason**: Unlock host failed.
- **Action**: - **Action**:
- investigate /var/log/mtcAgent.log file for cause logs files - Investigate the ``/var/log/mtcAgent.log`` file for cause logs files
- address the underlying issue - Address the underlying issue.
- manually lock and unlock the host to recover - Manually lock and unlock the host to recover.
- try recreating and re-applying the firmware update strategy to - Try recreating and re-applying the firmware update strategy to
automatically finish the update process automatically finish the update process.
.. note:: .. note::
If the strategy :command:`apply` fails, you must resolve the If the strategy :command:`apply` fails, you must resolve the
strategy:command:`apply` failure, and delete the failed strategy before strategy:command:`apply` failure and delete the failed strategy before
trying to create and apply another strategy. trying to create and apply another strategy.

22
doc/source/updates/kubernetes/handling-kubernetes-update-orchestration-failures.rst
View File

@ -18,7 +18,7 @@ Strategy creation failure
.. _jkf1590184623714-ul-fvs-vnq-5lb: .. _jkf1590184623714-ul-fvs-vnq-5lb:
- **Reason**: build failed with no reason. - **Reason**: Build failed with no reason.
- **Action**: - **Action**:
@ -27,7 +27,7 @@ Strategy creation failure
- Check recent logs added to /var/log/nfv-vim.log. - Check recent logs added to /var/log/nfv-vim.log.
- **Reason**: alarms from platform are present. - **Reason**: Alarms from platform are present.
- **Action**: - **Action**:
@ -43,7 +43,7 @@ Strategy creation failure
the ``relaxed`` alarms restrictions option ``--alarm-restrictions the ``relaxed`` alarms restrictions option ``--alarm-restrictions
relaxed``. relaxed``.
- **Reason**: no Kubernetes version upgrade required. - **Reason**: No Kubernetes version upgrade required.
- **Action**: - **Action**:
@ -65,14 +65,14 @@ Strategy Apply Failure
.. _jkf1590184623714-ul-rdf-4pq-5lb: .. _jkf1590184623714-ul-rdf-4pq-5lb:
- **Reason**: alarms from platform are present. - **Reason**: Alarms from platform are present.
- **Action**: suggests that an alarm has been raised since the creation - **Action**: This suggests that an alarm has been raised since the
of the strategy. Address the cause of the new alarm, delete the creation of the strategy. Address the cause of the new alarm, delete the
strategy and try creating and applying a new strategy. strategy and try creating and applying a new strategy.
- **Reason**: unable to migrate instances. - **Reason**: Unable to migrate instances.
- **Action**: See :ref:`Kubernetes Version Upgrade Operations Requiring - **Action**: See :ref:`Kubernetes Version Upgrade Operations Requiring
Manual Migration Manual Migration
@ -91,11 +91,11 @@ Strategy Apply Failure
.. include:: /_includes/handling-kubernetes-update-orchestration-failures.rest .. include:: /_includes/handling-kubernetes-update-orchestration-failures.rest
- **Reason**: lock host failed. - **Reason**: Lock host failed.
- **Action**: - **Action**:
- Investigate the /var/log/sysinv.log, and /var/log/nfv-vim.log - Investigate the ``/var/log/sysinv.log``, and ``/var/log/nfv-vim.log``
files. files.
- Address the underlying issue. - Address the underlying issue.
@ -106,11 +106,11 @@ Strategy Apply Failure
strategy to automatically finish the upgrade process. strategy to automatically finish the upgrade process.
- **Reason**: unlock host failed. - **Reason**: Unlock host failed.
- **Action**: - **Action**:
- Investigate /var/log/mtcAgent.log file for cause logs files. - Investigate ``/var/log/mtcAgent.log`` file for cause logs files.
- Address the underlying issue. - Address the underlying issue.

12
doc/source/updates/kubernetes/identifying-the-software-version-and-update-level-using-horizon.rst
View File

@ -11,10 +11,10 @@ interface. The system type is also shown.
.. rubric:: |proc| .. rubric:: |proc|
#. In the |prod| Horizon, open the System Configuration page. #. In the |prod| Horizon, open the **System Configuration** page.
The System Configuration page is available from **Admin** \> **Platform** The **System Configuration** page is available from **Admin** \>
\> **System Configuration** in the left-hand pane. **Platform** \> **System Configuration** in the left-hand pane.
#. Select the **Systems** tab to view the software version. #. Select the **Systems** tab to view the software version.
@ -24,10 +24,10 @@ interface. The system type is also shown.
shown in the **System Type** field. The mode \(**simplex**, **duplex**, or shown in the **System Type** field. The mode \(**simplex**, **duplex**, or
**standard**\) is shown in the **System Mode** field. **standard**\) is shown in the **System Mode** field.
#. In the |prod| Horizon interface, open the Software Management page. #. In the |prod| Horizon interface, open the **Software Management** page.
The Software Management page is available from **Admin** \> **Platform** \> The **Software Management** page is available from **Admin** \> **Platform**
**Software Management** in the left-hand pane. \> **Software Management** in the left-hand pane.
#. Select the **Patches** tab to view update information. #. Select the **Patches** tab to view update information.

2
doc/source/updates/kubernetes/identifying-the-software-version-and-update-level-using-the-cli.rst
View File

@ -47,7 +47,7 @@ For more about working with software updates, see :ref:`Manage Software Updates
+----------------------+----------------------------------------------------+ +----------------------+----------------------------------------------------+
.. note:: .. note::
The **system\_mode** field is shown only for a |prod| Simplex or Duplex The **system_mode** field is shown only for a |prod| Simplex or Duplex
system. system.
- To list applied software updates from the CLI, use the :command:`sw-patch - To list applied software updates from the CLI, use the :command:`sw-patch

17
doc/source/updates/kubernetes/in-service-versus-reboot-required-software-updates.rst
View File

@ -9,16 +9,15 @@ In-Service Versus Reboot-Required Software Updates
In-Service \(Reboot-not-Required\) and a Reboot-Required software updates are In-Service \(Reboot-not-Required\) and a Reboot-Required software updates are
available depending on the nature of the update to be performed. available depending on the nature of the update to be performed.
In-Service software updates provides a mechanism to issue updates that do not In-Service software updates provide a mechanism to issue updates that do not
require a reboot, allowing the update to be installed on in-service nodes and require a reboot, allowing the update to be installed on in-service nodes and
restarting affected processes as needed. restarting affected processes as needed.
Depending on the area of software being updated and the type of software Depending on the area of software being updated and the type of software change,
change, installation of the update may or may not require the |prod| hosts to installation of the update may or may not require the |prod| hosts to be
be rebooted. For example, a software update to the kernel would require the rebooted. For example, a software update to the kernel would require the host to
host to be rebooted in order to apply the update. Software updates are be rebooted in order to apply the update. Software updates are classified as
classified as reboot-required or reboot-not-required \(also referred to as reboot-required or reboot-not-required \(also referred to as
in-service\) type updates to indicate this. For reboot-required updates, the in-service\) type updates to indicate this. For reboot-required updates, the
hosted application pods are automatically relocated to an alternate host as hosted application pods are automatically relocated to an alternate host as part
part of the update procedure, prior to applying the update and rebooting the of the update procedure, prior to applying the update and rebooting the host.
host.

30
doc/source/updates/kubernetes/installing-in-service-software-update-using-horizon.rst
View File

@ -18,13 +18,13 @@ unlocked as part of applying the update.
#. In |prod| Horizon, open the Software Management page. #. In |prod| Horizon, open the Software Management page.
The Software Management page is available from **Admin** \> **Platform** \> The **Software Management** page is available from **Admin** \> **Platform**
**Software Management** in the left-hand pane. \> **Software Management** in the left-hand pane.
#. Select the Patches tab to see the current update status. #. Select the **Patches** tab to see the current update status.
The Patches page shows the current status of all updates uploaded to the The **Patches** tab shows the current status of all updates uploaded to the
system. If there are no updates, an empty Patch Table is displayed. system. If there are no updates, an empty **Patch Table** is displayed.
#. Upload the update \(patch\) file to the update storage area. #. Upload the update \(patch\) file to the update storage area.
@ -34,7 +34,7 @@ unlocked as part of applying the update.
The update file is transferred to the Active Controller and is copied to The update file is transferred to the Active Controller and is copied to
the update storage area, but it has yet to be applied to the cluster. This the update storage area, but it has yet to be applied to the cluster. This
is reflected in the Patches page. is reflected in the **Patches** tab.
#. Apply the update. #. Apply the update.
@ -43,29 +43,29 @@ unlocked as part of applying the update.
click the **Apply Patches** button at the top. You can use this selection click the **Apply Patches** button at the top. You can use this selection
process to apply all updates, or a selected subset, in a single operation. process to apply all updates, or a selected subset, in a single operation.
The Patches page is updated to report the update to be in the The **Patches** tab is updated to report the update to be in the
*Partial-Apply* state. *Partial-Apply* state.
#. Install the update on **controller-0**. #. Install the update on controller-0.
#. Select the **Hosts** tab. #. Select the **Hosts** tab.
The **Hosts** tab on the Host Inventory page reflects the new status of The **Hosts** tab on the **Host Inventory** page reflects the new status
the hosts with respect to the new update state. In this example, the of the hosts with respect to the new update state. In this example, the
update only applies to controller software, as can be seen by the update only applies to controller software, as can be seen by the
worker host's status field being empty, indicating that it is 'patch worker host's status field being empty, indicating that it is 'patch
current'. current'.
.. image:: figures/ekn1453233538504.png .. image:: figures/ekn1453233538504.png
#. Next, select the Install Patches option from the **Edit Host** button #. Select the Install Patches option from the **Edit Host** button
associated with **controller-0** to install the update. associated with controller-0 to install the update.
A confirmation window is presented giving you a last opportunity to A confirmation window is presented giving you a last opportunity to
cancel the operation before proceeding. cancel the operation before proceeding.
#. Repeat the steps 6 a,b, above with **controller-1** to install the update #. Repeat the steps 6 a,b, above with controller-1 to install the update
on **controller-1**. on controller-1.
#. Repeat the steps 6 a,b above for the worker and/or storage hosts \(if #. Repeat the steps 6 a,b above for the worker and/or storage hosts \(if
present\). present\).
@ -74,7 +74,7 @@ unlocked as part of applying the update.
#. Verify the state of the update. #. Verify the state of the update.
Visit the Patches page again. The update is now in the *Applied* state. Visit the **Patches** tab again. The update is now in the *Applied* state.
.. rubric:: |result| .. rubric:: |result|

2
doc/source/updates/kubernetes/installing-in-service-software-updates-using-the-cli.rst
View File

@ -39,7 +39,7 @@ unlocked as part of applying the update.
controller-0 192.168.204.3 Yes No nn.nn idle controller-0 192.168.204.3 Yes No nn.nn idle
controller-1 192.168.204.4 Yes No nn.nn idle controller-1 192.168.204.4 Yes No nn.nn idle
#. Ensure the original update files have been deleted from the root drive. #. Ensure that the original update files have been deleted from the root drive.
After they are uploaded to the storage area, the original files are no After they are uploaded to the storage area, the original files are no
longer required. You must use the command-line interface to delete them, in longer required. You must use the command-line interface to delete them, in

38
doc/source/updates/kubernetes/installing-reboot-required-software-updates-using-horizon.rst
View File

@ -32,15 +32,15 @@ update. The main steps of the procedure are:
#. Log in to the Horizon Web interface interface as the **admin** user. #. Log in to the Horizon Web interface interface as the **admin** user.
#. In Horizon, open the Software Management page. #. In Horizon, open the **Software Management** page.
The Software Management page is available from **Admin** \> **Platform** \> The **Software Management** page is available from **Admin** \> **Platform**
**Software Management** in the left-hand pane. \> **Software Management** in the left-hand pane.
#. Select the Patches tab to see the current status. #. Select the **Patches** tab to see the current status.
The Patches page shows the current status of all updates uploaded to the The **Patches** tab shows the current status of all updates uploaded to the
system. If there are no updates, an empty Patch Table is displayed. system. If there are no updates, an empty **Patch Table** is displayed.
#. Upload the update \(patch\) file to the update storage area. #. Upload the update \(patch\) file to the update storage area.
@ -50,7 +50,7 @@ update. The main steps of the procedure are:
The update file is transferred to the Active Controller and is copied to The update file is transferred to the Active Controller and is copied to
the storage area, but it has yet to be applied to the cluster. This is the storage area, but it has yet to be applied to the cluster. This is
reflected in the Patches page. reflected on the **Patches** tab.
#. Apply the update. #. Apply the update.
@ -62,14 +62,14 @@ update. The main steps of the procedure are:
The Patches page is updated to report the update to be in the The Patches page is updated to report the update to be in the
*Partial-Apply* state. *Partial-Apply* state.
#. Install the update on **controller-0**. #. Install the update on controller-0.
.. _installing-reboot-required-software-updates-using-horizon-step-N10107-N10028-N1001C-N10001: .. _installing-reboot-required-software-updates-using-horizon-step-N10107-N10028-N1001C-N10001:
#. Select the **Hosts** tab. #. Select the **Hosts** tab.
The **Hosts** tab on the Host Inventory page reflects the new status of The **Hosts** tab on the **Host Inventory** page reflects the new status
the hosts with respect to the new update state. As shown below, both of the hosts with respect to the new update state. As shown below, both
controllers are now reported as not 'patch current' and requiring controllers are now reported as not 'patch current' and requiring
reboot. reboot.
@ -83,10 +83,10 @@ update. The main steps of the procedure are:
Access to Horizon may be lost briefly during the active controller Access to Horizon may be lost briefly during the active controller
transition. You may have to log in again. transition. You may have to log in again.
#. Select the Lock Host option from the **Edit Host** button associated #. Select the **Lock Host** option from the **Edit Host** button associated
with **controller-0**. with **controller-0**.
#. Select the Install Patches option from the **Edit Host** button #. Select the **Install Patches** option from the **Edit Host** button
associated with **controller-0** to install the update. associated with **controller-0** to install the update.
A confirmation window is presented giving you a last opportunity to A confirmation window is presented giving you a last opportunity to
@ -94,12 +94,12 @@ update. The main steps of the procedure are:
Wait for the update install to complete. Wait for the update install to complete.
#. Select the Unlock Host option from the **Edit Host** button associated #. Select the **Unlock Host** option from the **Edit Host** button
with controller-0. associated with controller-0.
#. Repeat steps :ref:`6 #. Repeat steps :ref:`6
<installing-reboot-required-software-updates-using-horizon-step-N10107-N10028-N1001C-N10001>` <installing-reboot-required-software-updates-using-horizon-step-N10107-N10028-N1001C-N10001>`
a to e, with **controller-1** to install the update on **controller-1**. a to e, with **controller-1** to install the update on controller-1.
.. note:: .. note::
For |prod| Simplex systems, this step does not apply. For |prod| Simplex systems, this step does not apply.
@ -113,14 +113,14 @@ update. The main steps of the procedure are:
#. Verify the state of the update. #. Verify the state of the update.
Visit the Patches page. The update is now in the Applied state. Visit the **Patches** page. The update is now in the *Applied* state.
.. rubric:: |result| .. rubric:: |result|
The update is applied now, and all affected hosts have been updated. The update is now applied, and all affected hosts have been updated.
Updates can be removed using the **Remove Patches** button from the Patches Updates can be removed using the **Remove Patches** button from the **Patches**
page. The workflow is similar to the one presented in this section, with the tab. The workflow is similar to the one presented in this section, with the
exception that updates are being removed from each host instead of being exception that updates are being removed from each host instead of being
applied. applied.

71
doc/source/updates/kubernetes/installing-reboot-required-software-updates-using-the-cli.rst
View File

@ -14,7 +14,7 @@ You can install reboot-required software updates using the CLI.
.. _installing-reboot-required-software-updates-using-the-cli-steps-v1q-vlv-vw: .. _installing-reboot-required-software-updates-using-the-cli-steps-v1q-vlv-vw:
#. Log in as user **sysadmin** to the active controller and source the script #. Log in as user **sysadmin** to the active controller and source the script
/etc/platform/openrc to obtain administrative privileges. ``/etc/platform/openrc`` to obtain administrative privileges.
#. Verify that the updates are available using the :command:`sw-patch query` #. Verify that the updates are available using the :command:`sw-patch query`
command. command.
@ -49,10 +49,10 @@ You can install reboot-required software updates using the CLI.
.. parsed-literal:: .. parsed-literal::
~(keystone_admin)]$ sudo sw-patch apply |pn|-nn.nn_PATCH_0001 ~(keystone_admin)]$ sudo sw-patch apply |pn|-<nn>.<nn>_PATCH_0001
|pn|-nn.nn_PATCH_0001 is now in the repo |pn|-<nn>.<nn>_PATCH_0001 is now in the repo
where nn.nn in the update filename is the |prod-long| release number. where <nn>.<nn> in the update filename is the |prod-long| release number.
The update is now in the Partial-Apply state, ready for installation from The update is now in the Partial-Apply state, ready for installation from
the software updates repository on the impacted hosts. the software updates repository on the impacted hosts.
@ -101,7 +101,7 @@ You can install reboot-required software updates using the CLI.
host. host.
The **Patch Current** field of the :command:`query-hosts` command will The **Patch Current** field of the :command:`query-hosts` command will
briefly report “Pending” after you apply or remove an update, until briefly report *Pending* after you apply or remove an update, until
that host has checked against the repository to see if it is impacted that host has checked against the repository to see if it is impacted
by the patching operation. by the patching operation.
@ -124,7 +124,8 @@ You can install reboot-required software updates using the CLI.
**install-failed** **install-failed**
The operation failed, either due to an update error or something The operation failed, either due to an update error or something
killed the process. Check the patching.log on the node in question. killed the process. Check the ``patching.log`` on the node in
question.
**install-rejected** **install-rejected**
The node is unlocked, therefore the request to install has been The node is unlocked, therefore the request to install has been
@ -168,8 +169,8 @@ You can install reboot-required software updates using the CLI.
~(keystone_admin)]$ sudo sw-patch host-install <controller-0> ~(keystone_admin)]$ sudo sw-patch host-install <controller-0>
.. note:: .. note::
You can use the :command:`sudo sw-patch host-install-async` You can use the :command:`sudo sw-patch host-install-async <hostname>`
<hostname> command if you are launching multiple installs in command if you are launching multiple installs in
parallel. parallel.
#. Unlock the host. #. Unlock the host.
@ -181,7 +182,7 @@ You can install reboot-required software updates using the CLI.
Unlocking the host forces a reset of the host followed by a reboot. Unlocking the host forces a reset of the host followed by a reboot.
This ensures that the host is restarted in a known state. This ensures that the host is restarted in a known state.
All updates are now installed on **controller-0**. Querying the current All updates are now installed on controller-0. Querying the current
update status displays the following information: update status displays the following information:
.. code-block:: none .. code-block:: none
@ -199,14 +200,14 @@ You can install reboot-required software updates using the CLI.
storage-0 192.168.204.37 Yes No nn.nn idle storage-0 192.168.204.37 Yes No nn.nn idle
storage-1 192.168.204.90 Yes No nn.nn idle storage-1 192.168.204.90 Yes No nn.nn idle
#. Install all pending updates on **controller-1**. #. Install all pending updates on controller-1.
.. note:: .. note::
For |prod| Simplex systems, this step does not apply. For |prod| Simplex systems, this step does not apply.
Repeat the previous step targeting **controller-1**. Repeat the previous step targeting controller-1.
All updates are now installed on **controller-1** as well. Querying the All updates are now installed on controller-1 as well. Querying the
current updating status displays the following information: current updating status displays the following information:
.. code-block:: none .. code-block:: none
@ -227,12 +228,12 @@ You can install reboot-required software updates using the CLI.
#. Install any pending updates for the worker or storage hosts. #. Install any pending updates for the worker or storage hosts.
.. note:: .. note::
For |prod| Simplex or Duplex systems, this step does not apply. This step does not apply for |prod| Simplex or Duplex systems.
All hosted application pods currently running on a worker host are All hosted application pods currently running on a worker host are
re-located to another host. re-located to another host.
If the **Patch Current** status for a worker or storage host is **No**, If the **Patch Current** status for a worker or storage host is *No*,
apply the pending updates using the following commands: apply the pending updates using the following commands:
.. code-block:: none .. code-block:: none
@ -247,32 +248,36 @@ You can install reboot-required software updates using the CLI.
~(keystone_admin)]$ system host-unlock <hostname> ~(keystone_admin)]$ system host-unlock <hostname>
where <hostname> is the name of the host \(for example, **worker-0**\). where <hostname> is the name of the host \(for example, ``worker-0``\).
.. note:: .. note::
Update installations can be triggered in parallel. Update installations can be triggered in parallel.
The :command:`sw-patch host-install-async` command \(**install The :command:`sw-patch host-install-async` command \( cooresponding to
patches** on the Horizon Web interface\) can be run on all locked **install patches** on the Horizon Web interface\) can be run on all
nodes, without waiting for one node to complete the install before locked nodes, without waiting for one node to complete the install
triggering the install on the next. If you can lock the nodes at the before triggering the install on the next. If you can lock the nodes at
same time, without impacting hosted application services, you can the same time, without impacting hosted application services, you can
update them at the same time. update them at the same time.
Likewise, you can install an update to the standby controller and a Likewise, you can install an update to the standby controller and a
worker node at the same time. The only restrictions are those of the worker node at the same time. The only restrictions are those of the
lock: You cannot lock both controllers, and you cannot lock a worker lock:
node if you do not have enough free resources to relocate the hosted
applications from it. Also, in a Ceph configuration \(with storage * You cannot lock both controllers.
nodes\), you cannot lock more than one of
controller-0/controller-1/storage-0 at the same time, as these nodes * You cannot lock a worker node if you do not have enough free resources
are running Ceph monitors and you must have at least two in service at to relocate the hosted applications from it.
all times.
Also, in a Ceph configuration \(with storage nodes\), you cannot lock
more than one of controller-0/controller-1/storage-0 at the same time,
as these nodes are running Ceph monitors and you must have at least two
in service at all times.
#. Confirm that all updates are installed and the |prod| is up-to-date. #. Confirm that all updates are installed and the |prod| is up-to-date.
Use the :command:`sw-patch query` command to verify that all updates are Use the :command:`sw-patch query` command to verify that all updates are
**Applied**. *Applied*.
.. parsed-literal:: .. parsed-literal::
@ -280,13 +285,13 @@ You can install reboot-required software updates using the CLI.
Patch ID Patch State Patch ID Patch State
========================= =========== ========================= ===========
|pn|-nn.nn_PATCH_0001 Applied |pn|-<nn>.<nn>_PATCH_0001 Applied
where *nn.nn* in the update filename is the |prod| release number. where <nn>.<nn> in the update filename is the |prod| release number.
If the **Patch State** for any update is still shown as **Available** or If the **Patch State** for any update is still shown as *Available* or
**Partial-Apply**, use the **sw-patch query-hosts** command to identify *Partial-Apply*, use the :command:`sw-patch query-hosts`` command to identify
which hosts are not **Patch Current**, and then apply updates to them as which hosts are not *Patch Current*, and then apply updates to them as
described in the preceding steps. described in the preceding steps.

16
doc/source/updates/kubernetes/installing-software-updates-before-initial-commissioning.rst
View File

@ -12,18 +12,18 @@ This section describes installing software updates before you can commission
.. rubric:: |context| .. rubric:: |context|
This procedure assumes that the software updates to install are available on a This procedure assumes that the software updates to install are available on a
USB flash drive, or from a server reachable by **controller-0**. USB flash drive, or from a server reachable by controller-0.
.. rubric:: |prereq| .. rubric:: |prereq|
When initially installing the |prod-long| software, it is required that you When initially installing the |prod-long| software, it is required that you
install the latest available updates on **controller-0** before running Ansible install the latest available updates on controller-0 before running Ansible
Bootstrap Playbook, and before installing the software on other hosts. This Bootstrap Playbook, and before installing the software on other hosts. This
ensures that: ensures that:
.. _installing-software-updates-before-initial-commissioning-ul-gsq-1ht-vp: .. _installing-software-updates-before-initial-commissioning-ul-gsq-1ht-vp:
- The software on **controller-0**, and all other hosts, is up to date when - The software on controller-0, and all other hosts, is up to date when
the cluster comes alive. the cluster comes alive.
- You reduce installation time by avoiding updating the system right after an - You reduce installation time by avoiding updating the system right after an
@ -31,12 +31,12 @@ ensures that:
.. rubric:: |proc| .. rubric:: |proc|
#. Install software on **controller-0**. #. Install software on controller-0.
Use the |prod-long| bootable ISO image to initialize **controller-0**. Use the |prod-long| bootable ISO image to initialize controller-0.
This step takes you to the point where you use the console port to log in This step takes you to the point where you use the console port to log in
to **controller-0** as user **sysadmin**. to controller-0 as user **sysadmin**.
#. Populate the storage area. #. Populate the storage area.
@ -68,9 +68,9 @@ ensures that:
Patch installation is complete. Patch installation is complete.
Please reboot before continuing with configuration. Please reboot before continuing with configuration.
This command installs all applied updates on **controller-0**. This command installs all applied updates on controller-0.
#. Reboot **controller-0**. #. Reboot controller-0.
You must reboot the controller to ensure that it is running with the You must reboot the controller to ensure that it is running with the
software fully updated. software fully updated.

36
doc/source/updates/kubernetes/managing-software-updates.rst
View File

@ -7,9 +7,9 @@ Manage Software Updates
======================= =======================
Updates \(also known as patches\) to the system software become available as Updates \(also known as patches\) to the system software become available as
needed to address issues associated with a current |prod-long| software needed to address issues associated with a current |prod-long| software release.
release. Software updates must be uploaded to the active controller and applied Software updates must be uploaded to the active controller and applied to all
to all required hosts in the cluster. required hosts in the cluster.
.. note:: .. note::
Updating |prod-dc| is distinct from updating other |prod| configurations. Updating |prod-dc| is distinct from updating other |prod| configurations.
@ -21,8 +21,8 @@ to all required hosts in the cluster.
The following elements form part of the software update environment: The following elements form part of the software update environment:
**Reboot-Required Software Updates** **Reboot-Required Software Updates**
Reboot-required updates are typically major updates that require hosts to Reboot-required updates are typically major updates that require hosts to be
be locked during the update process and rebooted to complete the process. locked during the update process and rebooted to complete the process.
.. note:: .. note::
When a |prod| host is locked and rebooted for updates, the hosted When a |prod| host is locked and rebooted for updates, the hosted
@ -30,26 +30,26 @@ The following elements form part of the software update environment:
minimize the impact to the hosted application service. minimize the impact to the hosted application service.
**In-Service Software Updates** **In-Service Software Updates**
In-service \(reboot-not-required\), software updates are updates that do In-service \(reboot-not-required\), software updates are updates that do not
not require the locking and rebooting of hosts. The required |prod| require the locking and rebooting of hosts. The required |prod| software is
software is updated and any required |prod| processes are re-started. updated and any required |prod| processes are re-started. Hosted
Hosted applications pods and services are completely unaffected. applications pods and services are completely unaffected.
**Software Update Commands** **Software Update Commands**
The :command:`sw-patch` command is available on both active controllers. It The :command:`sw-patch` command is available on both active controllers. It
must be run as root using :command:`sudo`. It provides the user interface must be run as root using :command:`sudo`. It provides the user interface to
to process the updates, including querying the state of an update, listing process the updates, including querying the state of an update, listing
affected hosts, and applying, installing, and removing updates. affected hosts, and applying, installing, and removing updates.
**Software Update Storage Area** **Software Update Storage Area**
A central storage area maintained by the update controller. Software A central storage area maintained by the update controller. Software updates
updates are initially uploaded to the storage area and remains there until are initially uploaded to the storage area and remains there until they are
they are deleted. deleted.
**Software Update Repository** **Software Update Repository**
A central repository of software updates associated with any updates A central repository of software updates associated with any updates applied
applied to the system. This repository is used by all hosts in the cluster to the system. This repository is used by all hosts in the cluster to
to identify the software updates and rollbacks required on each host. identify the software updates and rollbacks required on each host.
**Software Update Logs** **Software Update Logs**
The following logs are used to record software update activity: The following logs are used to record software update activity:
@ -102,7 +102,7 @@ upload the software update directly from your workstation using a file browser
window provided by the software update upload facility. window provided by the software update upload facility.
A special case occurs during the initial provisioning of a cluster when you A special case occurs during the initial provisioning of a cluster when you
want to update **controller-0** before the system software is configured. This want to update controller-0 before the system software is configured. This
can only be done from the command line interface. See :ref:`Install Software can only be done from the command line interface. See :ref:`Install Software
Updates Before Initial Commissioning Updates Before Initial Commissioning
<installing-software-updates-before-initial-commissioning>` for details. <installing-software-updates-before-initial-commissioning>` for details.

32
doc/source/updates/kubernetes/manual-kubernetes-components-upgrade.rst
View File

@ -6,8 +6,8 @@
Manual Kubernetes Version Upgrade Manual Kubernetes Version Upgrade
================================= =================================
You can upgrade the Kubernetes version on a running system from one You can upgrade the Kubernetes version on a running system from one supported
supported version to another. version to another.
.. rubric:: |context| .. rubric:: |context|
@ -102,26 +102,26 @@ and upgrade various systems.
**State** **State**
Can be one of: Can be one of:
**active** *active*
The version is running everywhere. The version is running everywhere.
**partial** *partial*
The version is running somewhere. The version is running somewhere.
**available** *available*
The version can be upgraded to. The version can be upgraded to.
**unavailable** *unavailable*
The version is not available for upgrading. Either it is a The version is not available for upgrading. Either it is a downgrade
downgrade or it requires an intermediate upgrade first. Kubernetes or it requires an intermediate upgrade first. Kubernetes can be only
can be only upgraded one version at a time. upgraded one version at a time.
#. Confirm that the system is healthy. #. Confirm that the system is healthy.
Check the current system health status, resolve any alarms and other issues Check the current system health status, resolve any alarms and other issues
reported by the :command:`system health-query-kube-upgrade` command then reported by the :command:`system health-query-kube-upgrade` command then
recheck the system health status to confirm that all **System Health** recheck the system health status to confirm that all **System Health**
fields are set to **OK**. fields are set to *OK*.
.. code-block:: none .. code-block:: none
@ -156,8 +156,8 @@ and upgrade various systems.
| state | upgrade-started | | state | upgrade-started |
+-------------------+-------------------+ +-------------------+-------------------+
The upgrade process checks the applied/available updates, the upgrade path, The upgrade process checks the *applied*/*available* updates, the upgrade
the health of the system, the installed applications compatibility and path, the health of the system, the installed applications compatibility and
validates the system is ready for an upgrade. validates the system is ready for an upgrade.
.. warning:: .. warning::
@ -218,7 +218,7 @@ and upgrade various systems.
| updated_at | 2020-02-20T16:18:11.459736+00:00 | | updated_at | 2020-02-20T16:18:11.459736+00:00 |
+--------------+--------------------------------------+ +--------------+--------------------------------------+
The state **upgraded-networking** will be entered when the networking The state *upgraded-networking* will be entered when the networking
upgrade has completed. upgrade has completed.
#. Upgrade the control plane on the first controller. #. Upgrade the control plane on the first controller.
@ -241,7 +241,7 @@ and upgrade various systems.
You can upgrade either controller first. You can upgrade either controller first.
The state **upgraded-first-master** will be entered when the first control The state *upgraded-first-master* will be entered when the first control
plane upgrade has completed. plane upgrade has completed.
#. Upgrade the control plane on the second controller. #. Upgrade the control plane on the second controller.
@ -261,7 +261,7 @@ and upgrade various systems.
| target_version | v1.19.13 | | target_version | v1.19.13 |
+-----------------------+-------------------------+ +-----------------------+-------------------------+
The state **upgraded-second-master** will be entered when the upgrade has The state *upgraded-second-master* will be entered when the upgrade has
completed. completed.
#. Show the Kubernetes upgrade status for all hosts. #. Show the Kubernetes upgrade status for all hosts.
@ -298,7 +298,7 @@ and upgrade various systems.
~(keystone_admin)]$ system host-lock controller-1 ~(keystone_admin)]$ system host-lock controller-1
.. note:: .. warning::
For All-In-One Simplex systems, the controller must **not** be For All-In-One Simplex systems, the controller must **not** be
locked. locked.

6
doc/source/updates/kubernetes/manual-upgrade-overview.rst
View File

@ -15,7 +15,7 @@ Standard, |prod-dc|, and subcloud deployments.
.. xbooklink For information on updating |prod-dc|, see |distcloud-doc|: :ref:`Upgrade .. xbooklink For information on updating |prod-dc|, see |distcloud-doc|: :ref:`Upgrade
Management <upgrade-management-overview>`. Management <upgrade-management-overview>`.
An upgrade can be performed manually or by the Upgrade Orchestrator which An upgrade can be performed manually or using the Upgrade Orchestrator, which
automates a rolling install of an update across all of the |prod-long| hosts. automates a rolling install of an update across all of the |prod-long| hosts.
This section describes the manual upgrade procedures. This section describes the manual upgrade procedures.
@ -28,8 +28,8 @@ met:
- The system is patch current. - The system is patch current.
- There are no management-affecting alarms and the "system - There are no management-affecting alarms and the :command:`system
health-query-upgrade" check passes. health-query-upgrade` check passes.
- The new software load has been imported. - The new software load has been imported.

43
doc/source/updates/kubernetes/orchestration-upgrade-overview.rst
View File

@ -21,11 +21,12 @@ manual steps for operator oversight.
:ref:`Upgrade Management <upgrade-management-overview>`. :ref:`Upgrade Management <upgrade-management-overview>`.
.. note:: .. note::
The upgrade orchestration CLI is :command:`sw-manager`.To use upgrade
orchestration commands, you need administrator privileges. You must log in The upgrade orchestration commands are prefixed with :command:`sw-manager`.
to the active controller as user **sysadmin** and source the To use upgrade orchestration commands, you need administrator privileges.
/etc/platform/openrc script to obtain administrator privileges. Do not use You must log in to the active controller as user **sysadmin** and source the
**sudo**. ``/etc/platform/openrc`` script to obtain administrator privileges. Do not use
:command:`sudo`.
.. code-block:: none .. code-block:: none
@ -71,9 +72,9 @@ conditions:
orchestrated while another orchestration is in progress. orchestrated while another orchestration is in progress.
- Sufficient free capacity or unused worker resources must be available - Sufficient free capacity or unused worker resources must be available
across the cluster. A rough calculation is: Required spare capacity \( %\) across the cluster. A rough calculation is:
= \(Number of hosts to upgrade in parallel divided by the total number of
hosts\) times 100. ``Required spare capacity ( %) = (<Number-of-hosts-to-upgrade-in-parallel> / <total-number-of-hosts>) * 100``
.. _orchestration-upgrade-overview-section-N10081-N10026-N10001: .. _orchestration-upgrade-overview-section-N10081-N10026-N10001:
@ -81,16 +82,16 @@ conditions:
The Upgrade Orchestration Process The Upgrade Orchestration Process
--------------------------------- ---------------------------------
Upgrade orchestration can be initiated after the manual upgrade and stability Upgrade orchestration can be initiated after the initial controller host has
of the initial controller host. Upgrade orchestration automatically iterates been manual upgraded and returned to a stability state. Upgrade orchestration
through the remaining hosts, installing the new software load on each one: automatically iterates through the remaining hosts, installing the new software
first the other controller host, then the storage hosts, and finally the worker load on each one: first the other controller host, then the storage hosts, and
hosts. During worker host upgrades, pods are moved to alternate worker hosts finally the worker hosts. During worker host upgrades, pods are automatically
automatically. moved to alternate worker hosts.
The user first creates an upgrade orchestration strategy, or plan, for the You first create an upgrade orchestration strategy, or plan, for the automated
automated upgrade procedure. This customizes the upgrade orchestration, using upgrade procedure. This customizes the upgrade orchestration, using parameters
parameters to specify: to specify:
.. _orchestration-upgrade-overview-ul-eyw-fyr-31b: .. _orchestration-upgrade-overview-ul-eyw-fyr-31b:
@ -103,9 +104,9 @@ creates a number of stages for the overall upgrade strategy. Each stage
generally consists of moving pods, locking hosts, installing upgrades, and generally consists of moving pods, locking hosts, installing upgrades, and
unlocking hosts for a subset of the hosts on the system. unlocking hosts for a subset of the hosts on the system.
After creating the upgrade orchestration strategy, the user can either apply After creating the upgrade orchestration strategy, the you can either apply the
the entire strategy automatically, or apply individual stages to control and entire strategy automatically, or apply individual stages to control and monitor
monitor its progress manually. their progress manually.
Update and upgrade orchestration are mutually exclusive; they perform Update and upgrade orchestration are mutually exclusive; they perform
conflicting operations. Only a single strategy \(sw-patch or sw-upgrade\) is conflicting operations. Only a single strategy \(sw-patch or sw-upgrade\) is
@ -115,7 +116,7 @@ strategy before going back to the upgrade.
Some stages of the upgrade could take a significant amount of time \(hours\). Some stages of the upgrade could take a significant amount of time \(hours\).
For example, after upgrading a storage host, re-syncing the OSD data could take For example, after upgrading a storage host, re-syncing the OSD data could take
30m per TB \(assuming 500MB/s sync rate, which is about half of a 10G 30 minutes per TB \(assuming 500MB/s sync rate, which is about half of a 10G
infrastructure link\). infrastructure link\).
.. _orchestration-upgrade-overview-section-N10101-N10026-N10001: .. _orchestration-upgrade-overview-section-N10101-N10026-N10001:

6
doc/source/updates/kubernetes/overview-of-firmware-update-orchestration.rst
View File

@ -10,7 +10,7 @@ Firmware update orchestration allows the firmware on the hosts of an entire
|prod-long| system to be updated with a single operation. |prod-long| system to be updated with a single operation.
You can configure and run firmware update orchestration using the |CLI|, or the You can configure and run firmware update orchestration using the |CLI|, or the
stx-nfv VIM REST API. ``stx-nfv`` VIM REST API.
.. note:: .. note::
Firmware update is currently not supported on the Horizon Web interface. Firmware update is currently not supported on the Horizon Web interface.
@ -28,7 +28,7 @@ following conditions:
.. note:: .. note::
When configuring firmware update orchestration, you have the option to When configuring firmware update orchestration, you have the option to
ignore alarms that are not management-affecting severity. For more ignore alarms that are not of management-affecting severity. For more
information, see :ref:`Kubernetes Version Upgrade Cloud Orchestration information, see :ref:`Kubernetes Version Upgrade Cloud Orchestration
<configuring-kubernetes-update-orchestration>`. <configuring-kubernetes-update-orchestration>`.
@ -36,7 +36,7 @@ following conditions:
requires firmware update. The *Firmware Update Orchestration Strategy* requires firmware update. The *Firmware Update Orchestration Strategy*
creation step will fail if there are no qualified hosts detected. creation step will fail if there are no qualified hosts detected.
- Firmware update is a reboot required operation. Therefore, in systems that - Firmware update is a reboot-required operation. Therefore, in systems that
have the |prefix|-openstack application applied with running instances, if have the |prefix|-openstack application applied with running instances, if
the migrate option is selected there must be spare openstack-compute \ the migrate option is selected there must be spare openstack-compute \
(worker\) capacity to move instances off the openstack-compute \ (worker\) capacity to move instances off the openstack-compute \

40
doc/source/updates/kubernetes/performing-an-orchestrated-upgrade-using-the-cli.rst
View File

@ -13,8 +13,8 @@ The upgrade orchestration CLI is :command:`sw-manager`.
.. note:: .. note::
To use upgrade orchestration commands, you need administrator privileges. To use upgrade orchestration commands, you need administrator privileges.
You must log in to the active controller as user **sysadmin** and source the You must log in to the active controller as user **sysadmin** and source the
/etc/platform/openrc script to obtain administrator privileges. Do not use ``/etc/platform/openrc`` script to obtain administrator privileges. Do not use
**sudo**. :command:`sudo`.
The upgrade strategy options are shown in the following output: The upgrade strategy options are shown in the following output:
@ -34,9 +34,9 @@ The upgrade strategy options are shown in the following output:
abort Abort a strategy abort Abort a strategy
show Show a strategy show Show a strategy
You can perform a partially orchestrated upgrade using the CLI. Upgrade and You can perform a partially orchestrated upgrade using the |CLI|. Upgrade
stability of the initial controller node must be done manually before using orchestration of other |prod| nodes can be initiated after the initial
upgrade orchestration to orchestrate the remaining nodes of the |prod|. controller host has been manually upgraded and returned to a stability state.
.. note:: .. note::
Management-affecting alarms cannot be ignored at the indicated severity Management-affecting alarms cannot be ignored at the indicated severity
@ -65,9 +65,11 @@ See :ref:`Upgrading All-in-One Duplex / Standard
upgrade the initial controller node before doing the upgrade orchestration upgrade the initial controller node before doing the upgrade orchestration
described below to upgrade the remaining nodes of the |prod|. described below to upgrade the remaining nodes of the |prod|.
- The subclouds must use the Redfish platform management service if it is an All-in-one Simplex subcloud. - The subclouds must use the Redfish platform management service if it is an
All-in-one Simplex subcloud.
- Duplex \(AIODX/Standard\) upgrades are supported, and they do not require remote install via Redfish. - Duplex \(AIODX/Standard\) upgrades are supported, and they do not require
remote install via Redfish.
.. rubric:: |proc| .. rubric:: |proc|
@ -95,20 +97,20 @@ described below to upgrade the remaining nodes of the |prod|.
- storage-apply-type: - storage-apply-type:
- serial \(default\): storage hosts will be upgraded one at a time - ``serial`` \(default\): storage hosts will be upgraded one at a time
- parallel: storage hosts will be upgraded in parallel, ensuring that - ``parallel``: storage hosts will be upgraded in parallel, ensuring that
only one storage node in each replication group is patched at a only one storage node in each replication group is patched at a
time. time.
- ignore: storage hosts will not be upgraded - ``ignore``: storage hosts will not be upgraded
- worker-apply-type: - worker-apply-type:
**serial** \(default\) ``serial`` \(default\)
Worker hosts will be upgraded one at a time. Worker hosts will be upgraded one at a time.
**ignore** ``ignore``
Worker hosts will not be upgraded. Worker hosts will not be upgraded.
- Alarm Restrictions - Alarm Restrictions
@ -177,8 +179,8 @@ described below to upgrade the remaining nodes of the |prod|.
relocated before it is upgraded. relocated before it is upgraded.
#. Run :command:`sw-manager upgrade-strategy show` command, to display the #. Run :command:`sw-manager upgrade-strategy show` command, to display the
current-phase-completion displaying the field goes from 0% to 100% in current-phase-completion percentage progress indicator in various
various increments. Once at 100%, it returns: increments. Once at 100%, it returns:
.. code-block:: none .. code-block:: none
@ -196,7 +198,7 @@ described below to upgrade the remaining nodes of the |prod|.
build-result: success build-result: success
build-reason: build-reason:
#. Apply the upgrade-strategy. You can optionally apply a single stage at a #. Apply the upgrade strategy. You can optionally apply a single stage at a
time. time.
.. code-block:: none .. code-block:: none
@ -214,7 +216,7 @@ described below to upgrade the remaining nodes of the |prod|.
state: applying state: applying
inprogress: true inprogress: true
While an upgrade-strategy is being applied, it can be aborted. This results While an upgrade strategy is being applied, it can be aborted. This results
in: in:
- The current step will be allowed to complete. - The current step will be allowed to complete.
@ -222,9 +224,9 @@ described below to upgrade the remaining nodes of the |prod|.
- If necessary an abort phase will be created and applied, which will - If necessary an abort phase will be created and applied, which will
attempt to unlock any hosts that were locked. attempt to unlock any hosts that were locked.
After an upgrade-strategy has been applied \(or aborted\) it must be After an upgrade strategy has been applied \(or aborted\) it must be
deleted before another upgrade-strategy can be created. If an deleted before another upgrade strategy can be created. If an
upgrade-strategy application fails, you must address the issue that caused upgrade strategy application fails, you must address the issue that caused
the failure, then delete/re-create the strategy before attempting to apply the failure, then delete/re-create the strategy before attempting to apply
it again. it again.

41
doc/source/updates/kubernetes/performing-an-orchestrated-upgrade.rst
View File

@ -6,9 +6,10 @@
Perform an Orchestrated Upgrade Perform an Orchestrated Upgrade
=============================== ===============================
You can perform a partially-Orchestrated Upgrade of a |prod| system using the CLI and Horizon You can perform a partially orchestrated Upgrade of a |prod| system using the
Web interface. Upgrade and stability of the initial controller node must be done manually CLI and Horizon Web interface. Upgrade and stability of the initial controller
before using upgrade orchestration to orchestrate the remaining nodes of the |prod|. node must be done manually before using upgrade orchestration to orchestrate the
remaining nodes of the |prod|.
.. rubric:: |context| .. rubric:: |context|
@ -50,31 +51,31 @@ described below to upgrade the remaining nodes of the |prod| system.
#. Click the **Create Strategy** button. #. Click the **Create Strategy** button.
The Create Strategy dialog appears. The **Create Strategy** dialog appears.
#. Create an upgrade strategy by specifying settings for the parameters in the #. Create an upgrade strategy by specifying settings for the parameters in the
Create Strategy dialog box. **Create Strategy** dialog box.
Create an upgrade strategy, specifying the following parameters: Create an upgrade strategy, specifying the following parameters:
- storage-apply-type: - storage-apply-type:
**serial** \(default\) ``serial`` \(default\)
Storage hosts will be upgraded one at a time. Storage hosts will be upgraded one at a time.
**parallel** ``parallel``
Storage hosts will be upgraded in parallel, ensuring that only one Storage hosts will be upgraded in parallel, ensuring that only one
storage node in each replication group is upgraded at a time. storage node in each replication group is upgraded at a time.
**ignore** ``ignore``
Storage hosts will not be upgraded. Storage hosts will not be upgraded.
- worker-apply-type: - worker-apply-type:
**serial** \(default\): ``serial`` \(default\):
Worker hosts will be upgraded one at a time. Worker hosts will be upgraded one at a time.
**parallel** ``parallel``
Worker hosts will be upgraded in parallel, ensuring that: Worker hosts will be upgraded in parallel, ensuring that:
- At most max-parallel-worker-hosts \(see below\) worker hosts - At most max-parallel-worker-hosts \(see below\) worker hosts
@ -86,10 +87,10 @@ described below to upgrade the remaining nodes of the |prod| system.
- Worker hosts with no application pods are upgraded before - Worker hosts with no application pods are upgraded before
worker hosts with application pods. worker hosts with application pods.
**ignore** ``ignore``
Worker hosts will not be upgraded. Worker hosts will not be upgraded.
**max-parallel-worker-hosts** ``max-parallel-worker-hosts``
Specify the maximum worker hosts to upgrade in parallel \(minimum: Specify the maximum worker hosts to upgrade in parallel \(minimum:
2, maximum: 10\). 2, maximum: 10\).
@ -98,19 +99,19 @@ described below to upgrade the remaining nodes of the |prod| system.
(50), the value shall be at the maximum 2, which represents the (50), the value shall be at the maximum 2, which represents the
minimum value. minimum value.
**alarm-restrictions** ``alarm-restrictions``
This option lets you specify how upgrade orchestration behaves when This option lets you specify how upgrade orchestration behaves when
alarms are present. alarms are present.
You can use the CLI command :command:`fm alarm-list You can use the CLI command :command:`fm alarm-list
--mgmt_affecting` to view the alarms that are management affecting. --mgmt_affecting` to view the alarms that are management affecting.
**Strict** ``Strict``
The default strict option will result in upgrade orchestration The default strict option will result in upgrade orchestration
failing if there are any alarms present in the system \(except failing if there are any alarms present in the system \(except
for a small list of alarms\). for a small list of alarms\).
**Relaxed** ``Relaxed``
This option allows orchestration to proceed if alarms are This option allows orchestration to proceed if alarms are
present, as long as none of these alarms are management present, as long as none of these alarms are management
affecting. affecting.
@ -157,10 +158,10 @@ described below to upgrade the remaining nodes of the |prod| system.
NOT updated, but any additional pods on each worker host will be NOT updated, but any additional pods on each worker host will be
relocated before it is upgraded. relocated before it is upgraded.
#. Apply the upgrade-strategy. You can optionally apply a single stage at a #. Apply the upgrade strategy. You can optionally apply a single stage at a
time. time.
While an upgrade-strategy is being applied, it can be aborted. This results While an upgrade strategy is being applied, it can be aborted. This results
in: in:
- The current step will be allowed to complete. - The current step will be allowed to complete.
@ -168,9 +169,9 @@ described below to upgrade the remaining nodes of the |prod| system.
- If necessary an abort phase will be created and applied, which will - If necessary an abort phase will be created and applied, which will
attempt to unlock any hosts that were locked. attempt to unlock any hosts that were locked.
After an upgrade-strategy has been applied \(or aborted\) it must be After an upgrade strategy has been applied \(or aborted\) it must be
deleted before another upgrade-strategy can be created. If an deleted before another upgrade strategy can be created. If an
upgrade-strategy application fails, you must address the issue that caused upgrade strategy application fails, you must address the issue that caused
the failure, then delete/re-create the strategy before attempting to apply the failure, then delete/re-create the strategy before attempting to apply
it again. it again.

6
doc/source/updates/kubernetes/populating-the-storage-area.rst
View File

@ -18,9 +18,9 @@ before they can be applied.
.. parsed-literal:: .. parsed-literal::
$ sudo sw-patch upload /home/sysadmin/patches/|pn|-CONTROLLER_<nn.nn>_PATCH_0001.patch $ sudo sw-patch upload /home/sysadmin/patches/|pn|-CONTROLLER_<nn.nn>_PATCH_0001.patch
Cloud_Platform__CONTROLLER_nn.nn_PATCH_0001 is now available Cloud_Platform__CONTROLLER_<nn.nn>_PATCH_0001 is now available
where *nn.nn* in the update file name is the |prod| release number. where <nn.nn> in the update file name is the |prod| release number.
This example uploads a single update to the storage area. You can specify This example uploads a single update to the storage area. You can specify
multiple update files on the same command separating their names with multiple update files on the same command separating their names with
@ -42,7 +42,7 @@ before they can be applied.
$ sudo sw-patch query $ sudo sw-patch query
The update state is *Available* now, indicating that it is included in the The update state displays *Available*, indicating that it is included in the
storage area. Further details about the updates can be retrieved as storage area. Further details about the updates can be retrieved as
follows: follows:

18
doc/source/updates/kubernetes/reclaiming-disk-space.rst
View File

@ -20,10 +20,10 @@ version of an update has been committed to the system.
The :command:`query-dependencies` command will show a list of updates that The :command:`query-dependencies` command will show a list of updates that
are required by the specified update \(including itself\). The are required by the specified update \(including itself\). The
**--recursive** option will crawl through those dependencies to return a ``--recursive`` option will crawl through those dependencies to return a
list of all the updates in the specified update's dependency tree. This list of all the updates in the specified update's dependency tree. This
query is used by the “commit” command in calculating the set of updates to query is used by the :command:`commit` command in calculating the set of
be committed.For example, updates to be committed. For example,
.. parsed-literal:: .. parsed-literal::
@ -48,12 +48,12 @@ version of an update has been committed to the system.
updates to be committed. The commit set is calculated by querying the updates to be committed. The commit set is calculated by querying the
dependencies of each specified update. dependencies of each specified update.
The **--all** option, without the **--release** option, commits all updates The ``--all`` option, without the ``--release`` option, commits all updates
of the currently running release. When two releases are on the system use of the currently running release. When two releases are on the system use
the **--release** option to specify a particular release's updates if the ``--release`` option to specify a particular release's updates if
committing all updates for the non-running release. The **--dry-run** committing all updates for the non-running release. The ``--dry-run``
option shows the list of updates to be committed and how much disk space option shows the list of updates to be committed and how much disk space
will be freed up. This information is also shown without the **--dry-run** will be freed up. This information is also shown without the ``--dry-run``
option, before prompting to continue with the operation. An update can only option, before prompting to continue with the operation. An update can only
be committed once it has been fully applied to the system, and cannot be be committed once it has been fully applied to the system, and cannot be
removed after. removed after.
@ -61,7 +61,7 @@ version of an update has been committed to the system.
Following are examples that show the command usage. Following are examples that show the command usage.
The following command lists the status of all updates that are in an The following command lists the status of all updates that are in an
APPLIED state. *Applied* state.
.. code-block:: none .. code-block:: none
@ -84,7 +84,7 @@ version of an update has been committed to the system.
Would you like to continue? [y/N]: y Would you like to continue? [y/N]: y
The patches have been committed. The patches have been committed.
The following command shows the updates now in the COMMITTED state. The following command shows the updates now in the *Committed* state.
.. parsed-literal:: .. parsed-literal::

20
doc/source/updates/kubernetes/removing-reboot-required-software-updates.rst
View File

@ -23,20 +23,20 @@ following state transitions:
Use the command :command:`sw-patch remove` to trigger this transition. Use the command :command:`sw-patch remove` to trigger this transition.
**Partial-Remove to Available** **Partial-Remove to Available**
Use the command :command:`sudo sw-patch host-install-async` <hostname> Use the command :command:`sudo sw-patch host-install-async <hostname>`
repeatedly targeting each one of the applicable hosts in the cluster. The repeatedly targeting each one of the applicable hosts in the cluster. The
transition to the *Available* state is complete when the update is removed transition to the *Available* state is complete when the update is removed
from all target hosts. The update remains in the update storage area as if from all target hosts. The update remains in the update storage area as if
it had just been uploaded. it had just been uploaded.
.. note:: .. note::
The command :command:`sudo sw-patch host-install-async` <hostname> both The command :command:`sudo sw-patch host-install-async <hostname>` both
installs and removes updates as necessary. installs and removes updates as necessary.
The following example describes removing an update that applies only to the The following example describes removing an update that applies only to the
controllers. Removing updates can be done using the Horizon Web interface, controllers. Update removal can be done using the Horizon Web interface as
also, as discussed in :ref:`Install Reboot-Required Software Updates Using discussed in :ref:`Install Reboot-Required Software Updates Using Horizon
Horizon <installing-reboot-required-software-updates-using-horizon>`. <installing-reboot-required-software-updates-using-horizon>`.
.. rubric:: |proc| .. rubric:: |proc|
@ -52,7 +52,7 @@ Horizon <installing-reboot-required-software-updates-using-horizon>`.
|pn|-|pvr|-PATCH_0001 Applied |pn|-|pvr|-PATCH_0001 Applied
In this example the update is listed in the *Applied* state, but it could In this example the update is listed in the *Applied* state, but it could
be in the *Partial-Apply* state as well. alo be in the *Partial-Apply* state.
#. Remove the update. #. Remove the update.
@ -62,7 +62,7 @@ Horizon <installing-reboot-required-software-updates-using-horizon>`.
|pn|-|pvr|-PATCH_0001 has been removed from the repo |pn|-|pvr|-PATCH_0001 has been removed from the repo
The update is now in the *Partial-Remove* state, ready to be removed from The update is now in the *Partial-Remove* state, ready to be removed from
the impacted hosts where it was already installed. the impacted hosts where it was currently installed.
#. Query the updating status of all hosts in the cluster. #. Query the updating status of all hosts in the cluster.
@ -83,7 +83,7 @@ Horizon <installing-reboot-required-software-updates-using-horizon>`.
In this example, the controllers have updates ready to be removed, and In this example, the controllers have updates ready to be removed, and
therefore must be rebooted. therefore must be rebooted.
#. Remove all pending-for-removal updates from **controller-0**. #. Remove all pending-for-removal updates from controller-0.
#. Swact controller services away from controller-0. #. Swact controller services away from controller-0.
@ -93,7 +93,7 @@ Horizon <installing-reboot-required-software-updates-using-horizon>`.
.. code-block:: none .. code-block:: none
~(keystone_admin)]$ sudo sw-patch host-install-async <controller-0> ~(keystone_admin)]$ sudo sw-patch host-install-async controller-0
#. Unlock controller-0. #. Unlock controller-0.
@ -109,7 +109,7 @@ Horizon <installing-reboot-required-software-updates-using-horizon>`.
.. code-block:: none .. code-block:: none
~(keystone_admin)]$ sudo sw-patch host-install-async <controller-1> ~(keystone_admin)]$ sudo sw-patch host-install-async controller-1
.. rubric:: |result| .. rubric:: |result|

26
doc/source/updates/kubernetes/rolling-back-a-software-upgrade-after-the-second-controller-upgrade.rst
View File

@ -11,7 +11,7 @@ upgrade, however, the rollback will impact the hosting of applications.
The upgrade abort procedure can only be applied before the The upgrade abort procedure can only be applied before the
:command:`upgrade-complete` command is issued. Once this command is issued :command:`upgrade-complete` command is issued. Once this command is issued
the upgrade can not be aborted. If the return to the previous release is required, the upgrade cannot be aborted. If you must revert to the previous release,
then restore the system using the backup data taken prior to the upgrade. then restore the system using the backup data taken prior to the upgrade.
In some scenarios additional actions will be required to complete the upgrade In some scenarios additional actions will be required to complete the upgrade
@ -23,7 +23,7 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system upgrade-abort ~(keystone_admin)]$ system upgrade-abort
Once this is done there is no going back; the upgrade must be completely Once this is done there is no going back; the upgrade must be completely
aborted. aborted.
@ -41,13 +41,13 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system host-swact controller-0 ~(keystone_admin)]$ system host-swact controller-0
#. Lock controller-0. #. Lock controller-0.
.. code-block:: none .. code-block:: none
$ system host-lock controller-0 ~(keystone_admin)]$ system host-lock controller-0
#. Wipe the disk and power down all storage \(if applicable\) and worker hosts. #. Wipe the disk and power down all storage \(if applicable\) and worker hosts.
@ -66,13 +66,13 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system host-lock <hostID> ~(keystone_admin)]$ system host-lock <hostID>
#. Downgrade controller-0. #. Downgrade controller-0.
.. code-block:: none .. code-block:: none
$ system host-downgrade controller-0 ~(keystone_admin)]$ system host-downgrade controller-0
The host is re-installed with the previous release load. The host is re-installed with the previous release load.
@ -80,7 +80,7 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system host-unlock controller-0 ~(keystone_admin)]$ system host-unlock controller-0
.. note:: .. note::
Wait for controller-0 to become unlocked-enabled. Wait for the Wait for controller-0 to become unlocked-enabled. Wait for the
@ -90,7 +90,7 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system host-swact controller-1 ~(keystone_admin)]$ system host-swact controller-1
Swacting back to controller-0 will switch back to using the previous Swacting back to controller-0 will switch back to using the previous
release databases, which were frozen at the time of the swact to release databases, which were frozen at the time of the swact to
@ -100,11 +100,11 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system host-lock controller-1 ~(keystone_admin)]$ system host-lock controller-1
.. code-block:: none .. code-block:: none
$ system host-downgrade controller-1 ~(keystone_admin)]$ system host-downgrade controller-1
The host is re-installed with the previous release load. The host is re-installed with the previous release load.
@ -112,7 +112,7 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system host-unlock controller-1 ~(keystone_admin)]$ system host-unlock controller-1
#. Power up and unlock the storage hosts one at a time \(if using a Ceph #. Power up and unlock the storage hosts one at a time \(if using a Ceph
@ -134,7 +134,7 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system upgrade-complete ~(keystone_admin)]$ system upgrade-complete
This cleans up the upgrade release, configuration, databases, and so forth. This cleans up the upgrade release, configuration, databases, and so forth.
@ -142,4 +142,4 @@ abort. It may be necessary to restore the system from a backup.
.. code-block:: none .. code-block:: none
$ system load-delete ~(keystone_admin)]$ system load-delete

14
doc/source/updates/kubernetes/rolling-back-a-software-upgrade-before-the-second-controller-upgrade.rst
View File

@ -18,7 +18,7 @@ has upgraded successfully.
.. code-block:: none .. code-block:: none
$ system upgrade-abort ~(keystone_admin)]$ system upgrade-abort
The upgrade state is set to aborting. Once this is executed, there is no The upgrade state is set to aborting. Once this is executed, there is no
canceling; the upgrade must be completely aborted. canceling; the upgrade must be completely aborted.
@ -36,7 +36,7 @@ has upgraded successfully.
.. code-block:: none .. code-block:: none
$ system host-swact controller-1 ~(keystone_admin)]$ system host-swact controller-1
If controller-1 was active with the new upgrade release, swacting back to If controller-1 was active with the new upgrade release, swacting back to
controller-0 will switch back to using the previous release databases, controller-0 will switch back to using the previous release databases,
@ -47,8 +47,8 @@ has upgraded successfully.
.. code-block:: none .. code-block:: none
$ system host-lock controller-1 ~(keystone_admin)]$ system host-lock controller-1
$ system host-downgrade controller-1 ~(keystone_admin)]$ system host-downgrade controller-1
The host is re-installed with the previous release load. The host is re-installed with the previous release load.
@ -63,16 +63,16 @@ has upgraded successfully.
.. code-block:: none .. code-block:: none
$ system host-unlock controller-1 ~(keystone_admin)]$ system host-unlock controller-1
#. Complete the upgrade. #. Complete the upgrade.
.. code-block:: none .. code-block:: none
$ system upgrade-complete ~(keystone_admin)]$ system upgrade-complete
#. Delete the newer upgrade release that has been aborted. #. Delete the newer upgrade release that has been aborted.
.. code-block:: none .. code-block:: none
$ system load-delete <loadID> ~(keystone_admin)]$ system load-delete <loadID>

10
doc/source/updates/kubernetes/software-updates-and-upgrades-software-updates.rst
View File

@ -25,7 +25,7 @@ following items:
- feature enhancements - feature enhancements
Software updates can be installed manually or by the Update Orchestrator which Software updates can be installed manually or by the Update Orchestrator, which
automates a rolling install of an update across all of the |prod-long| hosts. automates a rolling install of an update across all of the |prod-long| hosts.
For more information on manual updates, see :ref:`Manage Software Updates For more information on manual updates, see :ref:`Manage Software Updates
<managing-software-updates>`. For more information on upgrade orchestration, <managing-software-updates>`. For more information on upgrade orchestration,
@ -40,7 +40,7 @@ see :ref:`Orchestrated Software Update <update-orchestration-overview>`.
.. xbooklink For more information, see, |distcloud-doc|: :ref:`Update Management for .. xbooklink For more information, see, |distcloud-doc|: :ref:`Update Management for
Distributed Cloud <update-management-for-distributed-cloud>`. Distributed Cloud <update-management-for-distributed-cloud>`.
The |prod| handles multiple updates being applied and removed at once. Software |prod| handles multiple updates being applied and removed at once. Software
updates can modify and update any area of |prod| software, including the kernel updates can modify and update any area of |prod| software, including the kernel
itself. For information on populating, installing and removing software itself. For information on populating, installing and removing software
updates, see :ref:`Manage Software Updates <managing-software-updates>`. updates, see :ref:`Manage Software Updates <managing-software-updates>`.
@ -73,9 +73,9 @@ the |prod| software:
#. **Application Software Updates** #. **Application Software Updates**
These software updates apply to software being managed through the These software updates apply to software being managed through the
StarlingX Application Package Manager, that is, ':command:`system StarlingX Application Package Manager, that is, :command:`system
application-upload/apply/remove/delete`'. |prod| delivers some software application-upload/apply/remove/delete`. |prod| delivers some software
through this mechanism, for example, **platform-integ-apps**. through this mechanism, for example, ``platform-integ-apps``.
For software updates for these applications, download the updated For software updates for these applications, download the updated
application tarball, containing the updated FluxCD manifest, and updated application tarball, containing the updated FluxCD manifest, and updated

53
doc/source/updates/kubernetes/software-upgrades.rst
View File

@ -18,7 +18,7 @@ hosts are upgraded one at time while continuing to provide its hosting services
to its hosted applications. An upgrade can be performed manually or using to its hosted applications. An upgrade can be performed manually or using
Upgrade Orchestration, which automates much of the upgrade procedure, leaving a Upgrade Orchestration, which automates much of the upgrade procedure, leaving a
few manual steps to prevent operator oversight. For more information on manual few manual steps to prevent operator oversight. For more information on manual
upgrades, see :ref:`Manual PLatform Components Upgrade upgrades, see :ref:`Manual Platform Components Upgrade
<manual-upgrade-overview>`. For more information on upgrade orchestration, see <manual-upgrade-overview>`. For more information on upgrade orchestration, see
:ref:`Orchestrated Platform Component Upgrade :ref:`Orchestrated Platform Component Upgrade
<orchestration-upgrade-overview>`. <orchestration-upgrade-overview>`.
@ -26,7 +26,7 @@ upgrades, see :ref:`Manual PLatform Components Upgrade
.. warning:: .. warning::
Do NOT use information in the |updates-doc| guide for |prod-dc| Do NOT use information in the |updates-doc| guide for |prod-dc|
orchestrated software upgrades. If information in this document is used for orchestrated software upgrades. If information in this document is used for
a |prod-dc| orchestrated upgrade, the upgrade will fail resulting a |prod-dc| orchestrated upgrade, the upgrade will fail, resulting
in an outage. The |prod-dc| Upgrade Orchestrator automates a in an outage. The |prod-dc| Upgrade Orchestrator automates a
recursive rolling upgrade of all subclouds and all hosts within the recursive rolling upgrade of all subclouds and all hosts within the
subclouds. subclouds.
@ -40,40 +40,41 @@ Before starting the upgrades process:
.. _software-upgrades-ul-ant-vgq-gmb: .. _software-upgrades-ul-ant-vgq-gmb:
- the system must be “patch current” - The system must be 'patch current'.
- there must be no management-affecting alarms present on the system - There must be no management-affecting alarms present on the system.
- ensure any certificates managed by cert manager will not be renewed during - Ensure that any certificates managed by cert manager will not be renewed
the upgrade process during the upgrade process.
- the new software load must be imported, and - The new software load must be imported.
- a valid license file for the new software release must be installed - A valid license file for the new software release must be installed.
The upgrade process starts by upgrading the controllers. The standby controller The upgrade process starts by upgrading the controllers. The standby controller
is upgraded first and involves loading the standby controller with the new is upgraded first and involves loading the standby controller with the new
release of software and migrating all the controller services' databases for release of software and migrating all the controller services' databases for the
the new release of software. Activity is switched to the upgraded controller, new release of software. Activity is switched to the upgraded controller,
running in a 'compatibility' mode where all inter-node messages are using running in a 'compatibility' mode where all inter-node messages are using
message formats from the old release of software. Before upgrading the second message formats from the old release of software. Prior to upgrading the second
controller, is the "point-of-no-return for an in-service abort" of the upgrades controller, you reach a "point-of-no-return for an in-service abort" of the
process. The second controller is loaded with the new release of software and upgrades process. The second controller is loaded with the new release of
becomes the new Standby controller. For more information on manual upgrades, software and becomes the new Standby controller. For more information on manual
see :ref:`Manual Platform Components Upgrade <manual-upgrade-overview>` . upgrades, see :ref:`Manual Platform Components Upgrade
<manual-upgrade-overview>` .
If present, storage nodes are locked, upgraded and unlocked one at a time in If present, storage nodes are locked, upgraded and unlocked one at a time in
order to respect the redundancy model of |prod| storage nodes. Storage nodes order to respect the redundancy model of |prod| storage nodes. Storage nodes
can be upgraded in parallel if using upgrade orchestration. can be upgraded in parallel if using upgrade orchestration.
Upgrade of worker nodes is the next step in the process. When locking a worker Worker nodes are then upgraded. Worker nodes are tainted when locked, such that
node the node is tainted, such that Kubernetes shuts down any pods on this Kubernetes shuts down any pods on this worker node and restarts the pods on
worker node and restarts the pods on another worker node. When upgrading the another worker node. When upgrading the worker node, the worker node network
worker node, the worker node network boots/installs the new software from the boots/installs the new software from the active controller. After unlocking the
active controller. After unlocking the worker node, the worker services are worker node, the worker services are running in a 'compatibility' mode where all
running in a 'compatibility' mode where all inter-node messages are using inter-node messages are using message formats from the old release of software.
message formats from the old release of software. Note that the worker nodes Note that the worker nodes can only be upgraded in parallel if using upgrade
can only be upgraded in parallel if using upgrade orchestration. orchestration.
The final step of the upgrade process is to activate and complete the upgrade. The final step of the upgrade process is to activate and complete the upgrade.
This involves disabling 'compatibility' modes on all hosts and clearing the This involves disabling 'compatibility' modes on all hosts and clearing the
@ -97,9 +98,9 @@ resolved. Issues specific to a storage or worker host can be addressed by
temporarily downgrading the host, addressing the issues and then upgrading the temporarily downgrading the host, addressing the issues and then upgrading the
host again, or in some cases by replacing the node. host again, or in some cases by replacing the node.
In extremely rare cases, it may be necessary to abort an upgrade. This is a In extremely rare cases, it may be necessary to abort an upgrade. This is a last
last resort and should only be done if there is no other way to address the resort and should only be done if there is no other way to address the issue
issue within the context of the upgrade. There are two cases for doing such an within the context of the upgrade. There are two scenarios for doing such an
abort: abort:
.. _software-upgrades-ul-dqp-brt-cx: .. _software-upgrades-ul-dqp-brt-cx:

22
doc/source/updates/kubernetes/the-firmware-update-orchestration-process.rst
View File

@ -32,13 +32,14 @@ instances and since the firmware update is a reboot required operation for a
host, the strategy offers **stop/start** or **migrate** options for managing host, the strategy offers **stop/start** or **migrate** options for managing
instances over the **lock/unlock** \(reboot\) steps in the update process. instances over the **lock/unlock** \(reboot\) steps in the update process.
You must use the **sw-manager** |CLI| tool to **create**, and then **apply** the You must use the :command:`sw-manager` |CLI| commands to create, and then apply
update strategy. A created strategy can be monitored with the **show** command. the update strategy. A created strategy can be monitored with the
For more information, see :ref:`Firmware Update Orchestration Using the CLI command:`sw-manager show` command. For more information, see :ref:`Firmware
Update Orchestration Using the CLI
<firmware-update-orchestration-using-the-cli>`. <firmware-update-orchestration-using-the-cli>`.
Firmware update orchestration automatically iterates through all Firmware update orchestration automatically iterates through all
**unlocked-enabled** hosts on the system looking for hosts with the worker *unlocked-enabled* hosts on the system looking for hosts with the worker
function that need firmware update and then proceeds to update them on the function that need firmware update and then proceeds to update them on the
strategy :command:`apply` action. strategy :command:`apply` action.
@ -52,13 +53,13 @@ After creating the *Firmware Update Orchestration Strategy*, you can either
apply the entire strategy automatically, or manually apply individual stages to apply the entire strategy automatically, or manually apply individual stages to
control and monitor the firmware update progress one stage at a time. control and monitor the firmware update progress one stage at a time.
When the firmware update strategy is **applied**, if the system is All-in-one, When the firmware update strategy is applied, if the system is All-in-one,
the controllers are updated first, one after the other with a swact in between, the controllers are updated first, one after the other with a swact in between,
followed by the remaining worker hosts according to the selected worker apply followed by the remaining worker hosts according to the selected worker apply
concurrency \(**serial** or **parallel**\) method. concurrency \(**serial** or **parallel**\) method.
The strategy creation default is to update the worker hosts serially unless the The strategy creation default is to update the worker hosts serially unless the
**parallel** worker apply type option is specified which configures the ``parallel`` worker apply type option is specified which configures the
firmware update process for worker hosts to be in parallel \(up to a maximum firmware update process for worker hosts to be in parallel \(up to a maximum
parallel number\) to reduce the overall firmware update installation time. parallel number\) to reduce the overall firmware update installation time.
@ -73,7 +74,8 @@ steps involved in a firmware update for a single or group of hosts include:
#. Alarm Query – is an update pre-check. #. Alarm Query – is an update pre-check.
#. Firmware update – non-service affecting update that can take over 45 minutes. #. Firmware update – is a non-service affecting update that can take over 45
minutes.
#. Lock Host. #. Lock Host.
@ -89,11 +91,11 @@ Strategy* considers any configured server groups and host aggregates when
creating the stages to reduce the impact to running instances. The *Firmware creating the stages to reduce the impact to running instances. The *Firmware
Update Orchestration Strategy* automatically manages the instances during the Update Orchestration Strategy* automatically manages the instances during the
strategy application process. The instance management options include strategy application process. The instance management options include
**start-stop** or **migrate**. ``start-stop`` or ``migrate``.
.. _htb1590431033292-ul-vcp-dvs-tlb: .. _htb1590431033292-ul-vcp-dvs-tlb:
- **start-stop**: where instances are stopped following the actual firmware - ``start-stop``: where instances are stopped following the actual firmware
update but before the lock operation and then automatically started again update but before the lock operation and then automatically started again
after the unlock completes. This is typically used for instances that do after the unlock completes. This is typically used for instances that do
not support migration or for cases where migration takes too long. To not support migration or for cases where migration takes too long. To
@ -101,6 +103,6 @@ strategy application process. The instance management options include
instance, the instance\(s\) should be protected and grouped into an instance, the instance\(s\) should be protected and grouped into an
anti-affinity server group\(s\) with its standby instance. anti-affinity server group\(s\) with its standby instance.
- **migrate**: where instances are moved off a host following the firmware - ``migrate``: where instances are moved off a host following the firmware
update but before the host is locked. Instances with **Live Migration** update but before the host is locked. Instances with **Live Migration**
support are **Live Migrated**. Otherwise, they are **Cold Migrated**. support are **Live Migrated**. Otherwise, they are **Cold Migrated**.

77
doc/source/updates/kubernetes/the-kubernetes-update-orchestration-process.rst
View File

@ -35,35 +35,33 @@ operation for a host, the strategy offers **stop/start** or **migrate** options
for managing instances over the **lock/unlock** \(reboot\) steps in the upgrade for managing instances over the **lock/unlock** \(reboot\) steps in the upgrade
process. process.
You must use the **sw-manager** CLI tool to **create**, and then **apply** the You must use the :command:`sw-manager`` CLI tool to create, and then apply the
upgrade strategy. A created strategy can be monitored with the **show** upgrade strategy. A created strategy can be monitored with the **show** command.
command.
Kubernetes version upgrade orchestration automatically iterates through all Kubernetes version upgrade orchestration automatically iterates through all
**unlocked-enabled** hosts on the system looking for hosts with the worker *unlocked-enabled* hosts on the system looking for hosts with the worker
function that need Kubernetes version upgrade and then proceeds to upgrade them function that need Kubernetes version upgrades and then proceeds to upgrade them
on the strategy :command:`apply` action. on the strategy :command:`apply` action.
.. note:: .. note::
Controllers (including |AIO| controllers) are upgraded before worker only Controllers (including |AIO| controllers) are upgraded before worker-only
hosts. Storage hosts do not run Kubernetes so Kubernetes is not upgraded hosts. Since storage hosts do not run Kubernetes, no upgrade is performed,
on them, although they still may be patched. although they may still be patched.
After creating the *Kubernetes Version Upgrade Orchestration Strategy*, you can After creating the *Kubernetes Version Upgrade Orchestration Strategy*, you can
either apply the entire strategy automatically, or manually apply individual either apply the entire strategy automatically, or manually apply individual
stages to control and monitor the Kubernetes version upgrade progress one stage stages to control and monitor the Kubernetes version upgrade progress one stage
at a time. at a time.
When the Kubernetes version upgrade strategy is **applied**, if the system is When the Kubernetes version upgrade strategy is applied, if the system is
All-in-one, the controllers are upgraded first, one after the other with a All-in-one, the controllers are upgraded first, one after the other with a
swact in between, followed by the remaining worker hosts according to the swact in between, followed by the remaining worker hosts according to the
selected worker apply concurrency \(**serial** or **parallel**\) method. selected worker apply concurrency \(**serial** or **parallel**\) method.
The strategy creation default is to upgrade the worker hosts serially unless By default, strategies upgrade the worker hosts serially unless the **parallel**
the **parallel** worker apply type option is specified which configures the worker apply type option is specified, which configures the Kubernetes version
Kubernetes version upgrade process for worker hosts to be in parallel \(up to a upgrade process for worker hosts to be in parallel \(up to a maximum parallel
maximum parallel number\) to reduce the overall Kubernetes version upgrade number\). This reduces the overall Kubernetes version upgrade installation time.
installation time.
The upgrade takes place in two phases. The first phase upgrades the patches The upgrade takes place in two phases. The first phase upgrades the patches
(controllers, storage and then workers), and the second phase upgrades (controllers, storage and then workers), and the second phase upgrades
@ -113,25 +111,25 @@ Upgrade Operations Requiring Manual Migration
On systems with |prefix|-openstack application, the *Kubernetes Version Upgrade On systems with |prefix|-openstack application, the *Kubernetes Version Upgrade
Orchestration Strategy* considers any configured server groups and host Orchestration Strategy* considers any configured server groups and host
aggregates when creating the stages to reduce the impact to running instances. aggregates when creating the stages in order to reduce the impact to running
The *Kubernetes Version Upgrade Orchestration Strategy* automatically manages instances. The *Kubernetes Version Upgrade Orchestration Strategy* automatically
the instances during the strategy application process. The instance management manages the instances during the strategy application process. The instance
options include **start-stop** or **migrate**. management options include **start-stop** or **migrate**.
.. _htb1590431033292-ul-vcp-dvs-tlb: .. _htb1590431033292-ul-vcp-dvs-tlb:
- **start-stop**: where instances are stopped following the actual Kubernetes - **start-stop**: Instances are stopped following the actual Kubernetes
upgrade but before the lock operation and then automatically started again upgrade but before the lock operation and then automatically started again
after the unlock completes. This is typically used for instances that do after the unlock completes. This is typically used for instances that do not
not support migration or for cases where migration takes too long. To support migration or for cases where migration takes too long. To ensure
ensure this does not impact the high-level service being provided by the this does not impact the high-level service being provided by the instance,
instance, the instance\(s\) should be protected and grouped into an the instance\(s\) should be protected and grouped into an anti-affinity
anti-affinity server group\(s\) with its standby instance. server group\(s\) with its standby instance.
- **migrate**: where instances are moved off a host following the Kubernetes - **migrate**: Instances are moved off a host following the Kubernetes upgrade
upgrade but before the host is locked. Instances with **Live Migration** but before the host is locked. Instances with **Live Migration** support are
support are **Live Migrated**. Otherwise, they are **Cold Migrated**. **Live Migrated**. Otherwise, they are **Cold Migrated**.
.. _kubernetes-update-operations-requiring-manual-migration: .. _kubernetes-update-operations-requiring-manual-migration:
@ -149,34 +147,33 @@ Do the following to manage the instance re-location manually:
.. _rbp1590431075472-ul-mgr-kvs-tlb: .. _rbp1590431075472-ul-mgr-kvs-tlb:
- Manually perform Kubernetes version upgrade at least one openstack-compute worker host. This - Manually perform a Kubernetes version upgrade of at least one
assumes that at least one openstack-compute worker host does not have any openstack-compute worker host. This assumes that at least one
instances, or has instances that can be migrated. For more information on openstack-compute worker host does not have any instances, or has instances
manually updating a host, see :ref:`Manual Kubernetes Version Upgrade that can be migrated. For more information on manually updating a host, see
:ref:`Manual Kubernetes Version Upgrade
<manual-kubernetes-components-upgrade>`. <manual-kubernetes-components-upgrade>`.
- If the migration is prevented by limitations in the VNF or virtual - If the migration is prevented by limitations in the VNF or virtual
application, perform the following: application, perform the following:
- Create new instances on an already upgraded openstack-compute worker #. Create new instances on an already upgraded openstack-compute worker
host. host.
- Manually migrate the data from the old instances to the new instances. #. Manually migrate the data from the old instances to the new instances.
.. note:: .. note::
This is specific to your environment and depends on the virtual This is specific to your environment and depends on the virtual
application running in the instance. application running in the instance.
- Terminate the old instances. #. Terminate the old instances.
- If the migration is prevented by the size of the instances local disks: - If the migration is prevented by the size of the instances local disks, then
for each openstack-compute worker host that has instances that cannot
- For each openstack-compute worker host that has instances that cannot
be migrated, manually move the instances using the CLI. be migrated, manually move the instances using the CLI.
Once all openstack-compute worker hosts containing instances that cannot be Once all openstack-compute worker hosts containing instances that cannot be
migrated have been Kubernetes version upgraded, Kubernetes version upgrade migrated have been Kubernetes-version upgraded, Kubernetes version upgrade
orchestration can then be used to upgrade the remaining worker hosts. orchestration can be used to upgrade the remaining worker hosts.

4
doc/source/updates/kubernetes/update-orchestration-cli.rst
View File

@ -16,8 +16,8 @@ interface dialog, described in :ref:`Configuring Update Orchestration
.. note:: .. note::
To use update orchestration commands, you need administrator privileges. To use update orchestration commands, you need administrator privileges.
You must log in to the active controller as user **sysadmin** and source You must log in to the active controller as user **sysadmin** and source
the /etc/platform/openrc script to obtain administrator privileges. Do not the ``/etc/platform/openrc`` script to obtain administrator privileges. Do not
use **sudo**. use :command:`sudo`.
.. note:: .. note::
Management-affecting alarms cannot be ignored at the indicated severity Management-affecting alarms cannot be ignored at the indicated severity

8
doc/source/updates/kubernetes/update-orchestration-overview.rst
View File

@ -14,7 +14,7 @@ operation.
:depth: 1 :depth: 1
You can configure and run update orchestration using the CLI, the Horizon Web You can configure and run update orchestration using the CLI, the Horizon Web
interface, or the stx-nfv REST API. interface, or the ``stx-nfv`` REST API.
.. note:: .. note::
Updating of |prod-dc| is distinct from updating of other |prod| Updating of |prod-dc| is distinct from updating of other |prod|
@ -74,9 +74,9 @@ the same time. Update orchestration only locks and unlocks \(that is, reboots\)
a host to install an update if at least one reboot-required update has been a host to install an update if at least one reboot-required update has been
applied. applied.
The user first creates an update orchestration strategy, or plan, for the You first create an update orchestration strategy, or plan, for the automated
automated updating procedure. This customizes the update orchestration, using updating procedure. This customizes the update orchestration, using parameters
parameters to specify: to specify:
.. _update-orchestration-overview-ul-eyw-fyr-31b: .. _update-orchestration-overview-ul-eyw-fyr-31b:

14
doc/source/updates/kubernetes/update-status-and-lifecycle.rst
View File

@ -29,17 +29,17 @@ To keep track of software update installation, you can use the
~(keystone_admin)]$ sudo sw-patch query ~(keystone_admin)]$ sudo sw-patch query
Patch ID Patch State Patch ID Patch State
=========== ============ =========== ============
|pvr|-nn.nn_PATCH_0001 Applied |pvr|-<nn>.<nn>_PATCH_0001 Applied
where *nn.nn* in the update filename is the |prod| release number. where <nn>.<nn> in the update filename is the |prod| release number.
This shows the **Patch State** for each of the updates in the storage area: This shows the 'Patch State' for each of the updates in the storage area:
**Available** ``Available``
An update in the *Available* state has been added to the storage area, but An update in the *Available* state has been added to the storage area, but
is not currently in the repository or installed on the hosts. is not currently in the repository or installed on the hosts.
**Partial-Apply** ``Partial-Apply``
An update in the *Partial-Apply* state has been added to the software An update in the *Partial-Apply* state has been added to the software
updates repository using the :command:`sw-patch apply` command, but has not updates repository using the :command:`sw-patch apply` command, but has not
been installed on all hosts that require it. It may have been installed on been installed on all hosts that require it. It may have been installed on
@ -51,12 +51,12 @@ This shows the **Patch State** for each of the updates in the storage area:
node X, you cannot just install the non-reboot-required update to the node X, you cannot just install the non-reboot-required update to the
unlocked node X. unlocked node X.
**Applied** ``Applied``
An update in the *Applied* state has been installed on all hosts that An update in the *Applied* state has been installed on all hosts that
require it. require it.
You can use the :command:`sw-patch query-hosts` command to see which hosts are You can use the :command:`sw-patch query-hosts` command to see which hosts are
fully updated \(**Patch Current**\). This also shows which hosts require fully updated \(Patch Current\). This also shows which hosts require
reboot, either because they are not fully updated, or because they are fully reboot, either because they are not fully updated, or because they are fully
updated but not yet rebooted. updated but not yet rebooted.

140
doc/source/updates/kubernetes/upgrading-all-in-one-duplex-or-standard.rst
View File

@ -16,11 +16,11 @@ of |prod| software.
- Perform a full backup to allow recovery. - Perform a full backup to allow recovery.
.. note:: .. note::
Back up files in the /home/sysadmin and /root directories prior Back up files in the ``/home/sysadmin`` and ``/root`` directories prior
to doing an upgrade. Home directories are not preserved during backup or to doing an upgrade. Home directories are not preserved during backup or
restore operations, blade replacement, or upgrades. restore operations, blade replacement, or upgrades.
- The system must be "patch current". All updates available for the current - The system must be 'patch current'. All updates available for the current
release running on the system must be applied, and all patches must be release running on the system must be applied, and all patches must be
committed. To find and download applicable updates, visit the |dnload-loc|. committed. To find and download applicable updates, visit the |dnload-loc|.
@ -29,9 +29,9 @@ of |prod| software.
.. note:: .. note::
Make sure that the ``/home/sysadmin`` directory has enough space Make sure that the ``/home/sysadmin`` directory has enough space
(at least 2GB of free space), otherwise the upgrade may fail once it (at least 2GB of free space), otherwise the upgrade may fail.
starts. If more space is needed, it is recommended to delete the If more space is needed, it is recommended to delete the
``.iso bootimage`` previously imported after the `load-import` command. ``.iso bootimage`` previously imported after the :command:`load-import` command.
- Transfer the new release software license file to controller-0, \(or onto a - Transfer the new release software license file to controller-0, \(or onto a
USB stick\). USB stick\).
@ -41,8 +41,8 @@ of |prod| software.
- Unlock all hosts. - Unlock all hosts.
- All nodes must be unlocked. The upgrade cannot be started when there - All nodes must be unlocked as the health check prevents the upgrade
are locked nodes \(the health check prevents it\). cannot if there are locked nodes.
.. note:: .. note::
The upgrade procedure includes steps to resolve system health issues. The upgrade procedure includes steps to resolve system health issues.
@ -51,8 +51,7 @@ of |prod| software.
#. Ensure that controller-0 is the active controller. #. Ensure that controller-0 is the active controller.
#. Install the license file for the release you are upgrading to, for example, #. Install the license file for the release you are upgrading.
nn.nn.
.. code-block:: none .. code-block:: none
@ -66,13 +65,12 @@ of |prod| software.
#. Import the new release. #. Import the new release.
#. Run the :command:`load-import` command on controller-0 to import
#. Run the :command:`load-import` command on **controller-0** to import
the new release. the new release.
First, source /etc/platform/openrc. Also, you must specify an exact Source ``/etc/platform/openrc``. Also, you must specify an exact
path to the \*.iso bootimage file and to the \*.sig bootimage signature path to the ``*.iso`` bootimage file and to the ``*.sig`` bootimage
file. signature file.
.. code-block:: none .. code-block:: none
@ -89,7 +87,7 @@ of |prod| software.
| required_patches | | | required_patches | |
+--------------------+-----------+ +--------------------+-----------+
The :command:`load-import` must be done on **controller-0** and accepts The :command:`load-import` must be done on controller-0 and accepts
relative paths. relative paths.
.. note:: .. note::
@ -112,7 +110,7 @@ of |prod| software.
The system must be 'patch current'. All software updates related to your The system must be 'patch current'. All software updates related to your
current |prod| software release must be uploaded, applied, and installed. current |prod| software release must be uploaded, applied, and installed.
All software updates to the new |prod| release, only need to be uploaded All software updates to the new |prod| release only need to be uploaded
and applied. The install of these software updates will occur automatically and applied. The install of these software updates will occur automatically
during the software upgrade procedure as the hosts are reset to load the during the software upgrade procedure as the hosts are reset to load the
new release of software. new release of software.
@ -127,7 +125,7 @@ of |prod| software.
Check the current system health status, resolve any alarms and other issues Check the current system health status, resolve any alarms and other issues
reported by the :command:`system health-query-upgrade` command, then reported by the :command:`system health-query-upgrade` command, then
recheck the system health status to confirm that all **System Health** recheck the system health status to confirm that all **System Health**
fields are set to **OK**. For example: fields are set to *OK*. For example:
.. code-block:: none .. code-block:: none
@ -148,10 +146,9 @@ of |prod| software.
All kubernetes applications are in a valid state: [OK] All kubernetes applications are in a valid state: [OK]
Active controller is controller-0: [OK] Active controller is controller-0: [OK]
By default, the upgrade process cannot be run and is not recommended to be By default, the upgrade process cannot be run with active alarms present.
run with active alarms present. Use the command :command:`system upgrade-start --force` Use the command :command:`system upgrade-start --force` to force the upgrade
to force the upgrade process to start and ignore non-management-affecting process to start and ignore non-management-affecting alarms.
alarms.
.. note:: .. note::
It is strongly recommended that you clear your system of any and all It is strongly recommended that you clear your system of any and all
@ -177,17 +174,18 @@ of |prod| software.
| to_release | nn.nn | | to_release | nn.nn |
+--------------+--------------------------------------+ +--------------+--------------------------------------+
This will make a copy of the upgrade data onto a DRBD file system to be
This will make a copy of the upgrade data onto a |DRBD| file system to be
used in the upgrade. Configuration changes are not allowed after this point used in the upgrade. Configuration changes are not allowed after this point
until the swact to controller-1 is completed. until the swact to controller-1 is completed.
The following upgrade state applies once this command is executed: The following upgrade state applies once this command is executed:
- started: - ``started``:
- State entered after :command:`system upgrade-start` completes. - State entered after :command:`system upgrade-start` completes.
- Release nn.nn system data \(for example, postgres databases\) has - Release <nn>.<nn> system data \(for example, postgres databases\) has
been exported to be used in the upgrade. been exported to be used in the upgrade.
- Configuration changes must not be made after this point, until the - Configuration changes must not be made after this point, until the
@ -200,13 +198,14 @@ of |prod| software.
upgrade. upgrade.
.. note:: .. note::
Use the command :command:`system upgrade-start --force` to force the Use the command :command:`system upgrade-start --force` to force the
upgrade process to start and ignore non-management-affecting alarms. upgrade process to start and ignore non-management-affecting alarms.
This should ONLY be done if you feel these alarms will not be an issue This should **ONLY** be done if you ascertain that these alarms will
over the upgrades process. interfere with the upgrades process.
On systems with Ceph storage, it also checks that the Ceph cluster is On systems with Ceph storage, the process also checks that the Ceph cluster
healthy. is healthy.
#. Upgrade controller-1. #. Upgrade controller-1.
@ -231,7 +230,7 @@ of |prod| software.
The following data migration states apply when this command is The following data migration states apply when this command is
executed: executed:
- data-migration: - ``data-migration``:
- State entered when :command:`system host-upgrade controller-1` - State entered when :command:`system host-upgrade controller-1`
is executed. is executed.
@ -245,21 +244,21 @@ of |prod| software.
You can view the upgrade progress on controller-1 using the You can view the upgrade progress on controller-1 using the
serial console. serial console.
- data-migration-complete or upgrading-controllers: - ``data-migration-complete or upgrading-controllers``:
- State entered when controller-1 upgrade is complete. - State entered when controller-1 upgrade is complete.
- System data has been successfully migrated from release nn.nn - System data has been successfully migrated from release <nn>.<nn>
to release nn.nn. to the newer Version.
- data-migration-failed: - ``data-migration-failed``:
- State entered if data migration on controller-1 fails. - State entered if data migration on controller-1 fails.
- Upgrade must be aborted. - Upgrade must be aborted.
.. note:: .. note::
Review the /var/log/sysinv.log on the active controller for Review the ``/var/log/sysinv.log`` on the active controller for
more details on data migration failure. more details on data migration failure.
#. Check the upgrade state. #. Check the upgrade state.
@ -277,7 +276,7 @@ of |prod| software.
+--------------+--------------------------------------+ +--------------+--------------------------------------+
If the :command:`upgrade-show` status indicates If the :command:`upgrade-show` status indicates
'data-migration-failed', then there is an issue with the data *data-migration-failed*, then there is an issue with the data
migration. Check the issue before proceeding to the next step. migration. Check the issue before proceeding to the next step.
#. Unlock controller-1. #. Unlock controller-1.
@ -286,20 +285,22 @@ of |prod| software.
~(keystone_admin)]$ system host-unlock controller-1 ~(keystone_admin)]$ system host-unlock controller-1
Wait for controller-1 to become **unlocked-enabled**. Wait for the DRBD Wait for controller-1 to enter the state *unlocked-enabled*. Wait for
sync **400.001** Services-related alarm is raised and then cleared. the |DRBD| sync **400.001** Services-related alarm to be raised and then
cleared.
The following states apply when this command is executed. The following states apply when this command is executed.
- upgrading-controllers: - ``upgrading-controllers``:
- State entered when controller-1 has been unlocked and is - State entered when controller-1 has been unlocked and is
running release nn.nn software. running release nn.nn software.
If it transitions to **unlocked-disabled-failed**, check the issue If the controller transitions to **unlocked-disabled-failed**, check the
before proceeding to the next step. The alarms may indicate a issue before proceeding to the next step. The alarms may indicate a
configuration error. Check the result of the configuration logs on configuration error. Check the result of the configuration logs on
controller-1, \(for example, Error logs in controller1:/var/log/puppet\). controller-1, \(for example, Error logs in
controller1:``/var/log/puppet``\).
#. Set controller-1 as the active controller. Swact to controller-1. #. Set controller-1 as the active controller. Swact to controller-1.
@ -307,36 +308,33 @@ of |prod| software.
~(keystone_admin)]$ system host-swact controller-0 ~(keystone_admin)]$ system host-swact controller-0
Wait until all services are enabled / active and the swact is complete Wait until services have become active on the new active controller-1 before
on controller-0 before proceeding to the next step. Use the following proceeding to the next step. The swact is complete when all services on
command below: controller-1 are in the state ``enabled-active``. Use the command ``system
servicegroup-list`` to monitor progress.
.. code-block:: none #. Upgrade controller-0.
~(keystone_admin)]$ system servicegroup-list #. Lock controller-0.
#. Upgrade **controller-0**.
#. Lock **controller-0**.
.. code-block:: none .. code-block:: none
~(keystone_admin)]$ system host-lock controller-0 ~(keystone_admin)]$ system host-lock controller-0
#. Upgrade **controller-0**. #. Upgrade controller-0.
.. code-block:: none .. code-block:: none
~(keystone_admin)]$ system host-upgrade controller-0 ~(keystone_admin)]$ system host-upgrade controller-0
#. Unlock **controller-0**. #. Unlock controller-0.
.. code-block:: none .. code-block:: none
~(keystone_admin)]$ system host-unlock controller-0 ~(keystone_admin)]$ system host-unlock controller-0
Wait until the DRBD sync **400.001** Services-related alarm is raised Wait until the |DRBD| sync **400.001** Services-related alarm is raised
and then cleared before proceeding to the next step. and then cleared before proceeding to the next step.
- upgrading-hosts: - upgrading-hosts:
@ -356,7 +354,7 @@ of |prod| software.
Clear all alarms unrelated to the upgrade process. Clear all alarms unrelated to the upgrade process.
#. If using Ceph storage backend, upgrade the storage nodes one at a time. #. If using Ceph a storage backend, upgrade the storage nodes one at a time.
.. note:: .. note::
Proceed to step 13 if no storage/worker node is present. Proceed to step 13 if no storage/worker node is present.
@ -370,10 +368,10 @@ of |prod| software.
~(keystone_admin)]$ system host-lock storage-0 ~(keystone_admin)]$ system host-lock storage-0
#. Verify that the OSDs are down after the storage node is locked. #. Verify that the |OSDs| are down after the storage node is locked.
In the Horizon interface, navigate to **Admin** \> **Platform** \> In the Horizon interface, navigate to **Admin** \> **Platform** \>
**Storage Overview** to view the status of the OSDs. **Storage Overview** to view the status of the |OSDs|.
#. Upgrade storage-0. #. Upgrade storage-0.
@ -381,7 +379,7 @@ of |prod| software.
~(keystone_admin)]$ system host-upgrade storage-0 ~(keystone_admin)]$ system host-upgrade storage-0
The upgrade is complete when the node comes online, and at that point, The upgrade is complete when the node comes online. At that point
you can safely unlock the node. you can safely unlock the node.
After upgrading a storage node, but before unlocking, there are Ceph After upgrading a storage node, but before unlocking, there are Ceph
@ -408,7 +406,7 @@ of |prod| software.
**800.003**. The alarm is cleared after all storage nodes are **800.003**. The alarm is cleared after all storage nodes are
upgraded. upgraded.
#. Upgrade worker hosts, one at a time, if any. #. Upgrade worker hosts, if any, one at a time.
#. Lock worker-0. #. Lock worker-0.
@ -431,7 +429,7 @@ of |prod| software.
~(keystone_admin)]$ system host-unlock worker-0 ~(keystone_admin)]$ system host-unlock worker-0
Wait for all alarms to clear after the unlock before proceeding to the After the unlock wait for all alarms to clear before proceeding to the
next worker host. next worker host.
#. Repeat the above steps for each worker host. #. Repeat the above steps for each worker host.
@ -442,9 +440,9 @@ of |prod| software.
~(keystone_admin)]$ system host-swact controller-1 ~(keystone_admin)]$ system host-swact controller-1
Wait until services have gone active on the active controller-0 before Wait until services have become available on the active controller-0 before
proceeding to the next step. When all services on controller-0 are proceeding to the next step. When all services on controller-0 are in the
enabled-active, the swact is complete. ``enabled-active`` state, the swact is complete.
#. Activate the upgrade. #. Activate the upgrade.
@ -460,31 +458,31 @@ of |prod| software.
| to_release | nn.nn | | to_release | nn.nn |
+--------------+--------------------------------------+ +--------------+--------------------------------------+
During the running of the :command:`upgrade-activate` command, new When running the :command:`upgrade-activate` command, new
configurations are applied to the controller. 250.001 \(**hostname configurations are applied to the controller. 250.001 \(**hostname
Configuration is out-of-date**\) alarms are raised and are cleared as the Configuration is out-of-date**\) alarms are raised and are cleared as the
configuration is applied. The upgrade state goes from **activating** to configuration is applied. The upgrade state goes from ``activating`` to
**activation-complete** once this is done. ``activation-complete`` once this is done.
The following states apply when this command is executed. The following states apply when this command is executed.
**activation-requested** ``activation-requested``
State entered when :command:`system upgrade-activate` is executed. State entered when :command:`system upgrade-activate` is executed.
**activating** ``activating``
State entered when we have started activating the upgrade by applying State entered when the system has started activating the upgrade by applying
new configurations to the controller and compute hosts. new configurations to the controller and compute hosts.
**activating-hosts** ``activating-hosts``
State entered when applying host-specific configurations. This state is State entered when applying host-specific configurations. This state is
entered only if needed. entered only if needed.
**activation-complete** ``activation-complete``
State entered when new configurations have been applied to all State entered when new configurations have been applied to all
controller and compute hosts. controller and compute hosts.
#. Check the status of the upgrade again to see it has reached #. Check the status of the upgrade again to see it has reached
**activation-complete**. ``activation-complete``.
.. code-block:: none .. code-block:: none

117
doc/source/updates/kubernetes/upgrading-all-in-one-simplex.rst
View File

@ -30,8 +30,8 @@ software.
- The system is patch current. - The system is patch current.
- There should be sufficient free space in /opt/platform-backup. Remove - There should be sufficient free space in ``/opt/platform-backup.``.
any unused files if necessary. Remove any unused files if necessary.
- The new software load has been imported. - The new software load has been imported.
@ -41,10 +41,12 @@ software.
stick\); controller-0 must be active. stick\); controller-0 must be active.
.. note:: .. note::
Make sure that the ``/home/sysadmin`` directory has enough space
(at least 2GB of free space), otherwise the upgrade may fail once it Make sure that the ``/home/sysadmin`` directory has enough space (at
starts. If more space is needed, it is recommended to delete the least 2GB of free space), otherwise the upgrade may fail once it starts.
``.iso bootimage`` previously imported after the `load-import` command. If more space is needed, it is recommended to delete the ``.iso``
bootimage previously imported after the :command:`load-import`
command.
- Transfer the new release software license file to controller-0 \(or onto a - Transfer the new release software license file to controller-0 \(or onto a
USB stick\). USB stick\).
@ -55,7 +57,7 @@ software.
.. note:: .. note::
The upgrade procedure includes steps to resolve system health issues. The upgrade procedure includes steps to resolve system health issues.
End user container images in `registry.local` will be backed up during the End user container images in ``registry.local`` will be backed up during the
upgrade process. This only includes images other than |prod| system and upgrade process. This only includes images other than |prod| system and
application images. These images are limited to 5 GB in total size. If the application images. These images are limited to 5 GB in total size. If the
system contains more than 5 GB of these images, the upgrade start will fail. system contains more than 5 GB of these images, the upgrade start will fail.
@ -71,8 +73,7 @@ For more details, see :ref:`Detailed contents of a system backup
$ source /etc/platform/openrc $ source /etc/platform/openrc
~(keystone_admin)]$ ~(keystone_admin)]$
#. Install the license file for the release you are upgrading to, for example, #. Install the license file for the release you are upgrading to.
nn.nn.
.. code-block:: none .. code-block:: none
@ -86,13 +87,13 @@ For more details, see :ref:`Detailed contents of a system backup
#. Import the new release. #. Import the new release.
#. Run the :command:`load-import` command on **controller-0** to import #. Run the :command:`load-import` command on controller-0 to import
the new release. the new release.
First, source /etc/platform/openrc. First, source ``/etc/platform/openrc``.
You must specify an exact path to the \*.iso bootimage file and to the You must specify an exact path to the ``*.iso`` bootimage file and to the
\*.sig bootimage signature file. ``*.sig`` bootimage signature file.
.. code-block:: none .. code-block:: none
@ -109,7 +110,7 @@ For more details, see :ref:`Detailed contents of a system backup
| required_patches | | | required_patches | |
+--------------------+-----------+ +--------------------+-----------+
The :command:`load-import` must be done on **controller-0** and accepts The :command:`load-import` must be done on controller-0 and accepts
relative paths. relative paths.
.. note:: .. note::
@ -130,9 +131,9 @@ For more details, see :ref:`Detailed contents of a system backup
#. Apply any required software updates. #. Apply any required software updates.
The system must be 'patch current'. All software updates related to your The system must be 'patch current'. All software updates related to your
current |prod| software release must be, uploaded, applied, and installed. current |prod| software release must be uploaded, applied, and installed.
All software updates to the new |prod| release, only need to be uploaded All software updates to the new |prod| release only need to be uploaded
and applied. The install of these software updates will occur automatically and applied. The install of these software updates will occur automatically
during the software upgrade procedure as the hosts are reset to load the during the software upgrade procedure as the hosts are reset to load the
new release of software. new release of software.
@ -147,7 +148,7 @@ For more details, see :ref:`Detailed contents of a system backup
Check the current system health status, resolve any alarms and other issues Check the current system health status, resolve any alarms and other issues
reported by the :command:`system health-query-upgrade` command, then reported by the :command:`system health-query-upgrade` command, then
recheck the system health status to confirm that all **System Health** recheck the system health status to confirm that all **System Health**
fields are set to **OK**. fields are set to *OK*.
.. code-block:: none .. code-block:: none
@ -167,13 +168,13 @@ For more details, see :ref:`Detailed contents of a system backup
All kubernetes applications are in a valid state: [OK] All kubernetes applications are in a valid state: [OK]
Active controller is controller-0: [OK] Active controller is controller-0: [OK]
By default, the upgrade process cannot be run and is not recommended to be By default, the upgrade process cannot be run with Active Alarms present.
run with Active Alarms present. However, management affecting alarms can be However, management affecting alarms can be ignored with the
ignored with the :command:`--force` option with the :command:`system :command:`--force` option with the :command:`system upgrade-start` command
upgrade-start` command to force the upgrade process to start. to force the upgrade process to start.
.. note:: .. note::
It is strongly recommended that you clear your system of any and all It is strongly recommended that you clear your system of all
alarms before doing an upgrade. While the :command:`--force` option is alarms before doing an upgrade. While the :command:`--force` option is
available to run the upgrade, it is a best practice to clear any available to run the upgrade, it is a best practice to clear any
alarms. alarms.
@ -192,40 +193,41 @@ For more details, see :ref:`Detailed contents of a system backup
| to_release | nn.nn | | to_release | nn.nn |
+--------------+--------------------------------------+ +--------------+--------------------------------------+
This will back up the system data and images to /opt/platform-backup. This will back up the system data and images to ``/opt/platform-backup.``.
/opt/platform-backup is preserved when the host is reinstalled. With the ``/opt/platform-backup.`` is preserved when the host is reinstalled. With the
platform backup, the size of /home/sysadmin must be less than 2GB. platform backup, the size of ``/home/sysadmin`` must be less than 2GB.
This process may take several minutes. This process may take several minutes.
When the upgrade state is upgraded to **started** the process is complete. When the upgrade state is upgraded to *started* the process is complete.
Any changes made to the system after this point will be lost when the data Any changes made to the system after this point will be lost when the data
is restored. is restored.
The following upgrade state applies once this command is executed: The following upgrade state applies once this command is executed:
- started: - ``started``:
- State entered after :command:`system upgrade-start` completes. - State entered after :command:`system upgrade-start` completes.
- Release nn.nn system data \(for example, postgres databases\) has - Release <nn>.<nn> system data \(for example, postgres databases\) has
been exported to be used in the upgrade. been exported to be used in the upgrade.
- Configuration changes must not be made after this point, until the - Configuration changes must not be made after this point, until the
upgrade is completed. upgrade is completed.
As part of the upgrade, the upgrade process checks the health of the system The upgrade process checks the health of the system and validates that the
and validates that the system is ready for an upgrade. system is ready for an upgrade.
The upgrade process checks that no alarms are active before starting an The upgrade process checks that no alarms are active before starting an
upgrade. upgrade.
.. note:: .. note::
Use the command :command:`system upgrade-start --force` to force the Use the command :command:`system upgrade-start --force` to force the
upgrades process to start and to ignore management affecting alarms. upgrades process to start and to ignore management affecting alarms.
This should ONLY be done if you feel these alarms will not be an issue This should **ONLY** be done if you have ascertained that these alarms
over the upgrades process. will not interfere with the upgrades process.
#. Check the upgrade state. #. Check the upgrade state.
@ -241,13 +243,13 @@ For more details, see :ref:`Detailed contents of a system backup
| to_release | nn.nn | | to_release | nn.nn |
+--------------+--------------------------------------+ +--------------+--------------------------------------+
Ensure the upgrade state is **started**. It will take several minutes to Ensure the upgrade state is *started*. It will take several minutes to
transition to the started state. transition to the *started* state.
#. \(Optional\) Copy the upgrade data from the system to an alternate safe #. \(Optional\) Copy the upgrade data from the system to an alternate safe
location \(such as a USB drive or remote server\). location \(such as a USB drive or remote server\).
The upgrade data is located under /opt/platform-backup. Example file names The upgrade data is located under ``/opt/platform-backup``. Example file names
are: are:
**lost+found upgrade\_data\_2020-06-23T033950\_61e5fcd7-a38d-40b0-ab83-8be55b87fee2.tgz** **lost+found upgrade\_data\_2020-06-23T033950\_61e5fcd7-a38d-40b0-ab83-8be55b87fee2.tgz**
@ -264,8 +266,8 @@ For more details, see :ref:`Detailed contents of a system backup
#. Upgrade controller-0. #. Upgrade controller-0.
This is the point of no return. All data except /opt/platform-backup/ will This is the point of no return. All data except ``/opt/platform-backup/``
be erased from the system. This will wipe the **rootfs** and reboot the will be erased from the system. This will wipe the ``rootfs`` and reboot the
host. The new release must then be manually installed \(via network or host. The new release must then be manually installed \(via network or
USB\). USB\).
@ -279,11 +281,11 @@ For more details, see :ref:`Detailed contents of a system backup
#. Install the new release of |prod-long| Simplex software via network or USB. #. Install the new release of |prod-long| Simplex software via network or USB.
#. Verify and configure IP connectivity. External connectivity is required to #. Verify and configure IP connectivity. External connectivity is required to
run the Ansible upgrade playbook. The |prod-long| boot image will DHCP out all run the Ansible upgrade playbook. The |prod-long| boot image will |DHCP| out
interfaces so the server may have obtained an IP address and have external IP all interfaces so the server may have obtained an IP address and have
connectivity if a DHCP server is present in your environment. Verify this using external IP connectivity if a |DHCP| server is present in your environment.
the :command:`ip addr` command. Otherwise, manually configure an IP address and default IP Verify this using the :command:`ip addr` command. Otherwise, manually
route. configure an IP address and default IP route.
#. Restore the upgrade data. #. Restore the upgrade data.
@ -298,14 +300,13 @@ For more details, see :ref:`Detailed contents of a system backup
following parameter: following parameter:
``ansible_become_pass`` ``ansible_become_pass``
The ansible playbook will check ``/home/sysadmin/<hostname\>.yml`` for
The ansible playbook will check /home/sysadmin/<hostname\>.yml for these these user configuration override files for hosts. For example, if
user configuration override files for hosts. For example, if running running ansible locally, ``/home/sysadmin/localhost.yml``.
ansible locally, /home/sysadmin/localhost.yml.
By default the playbook will search for the upgrade data file under By default the playbook will search for the upgrade data file under
/opt/platform-backup. If required, use the **upgrade\_data\_file** ``/opt/platform-backup``. If required, use the ``upgrade_data_file``
parameter to specify the path to the **upgrade\_data**. parameter to specify the path to the ``upgrade_data``.
.. note:: .. note::
This playbook does not support replay. This playbook does not support replay.
@ -314,7 +315,7 @@ For more details, see :ref:`Detailed contents of a system backup
This can take more than one hour to complete. This can take more than one hour to complete.
Once the data restoration is complete the upgrade state will be set to Once the data restoration is complete the upgrade state will be set to
**upgrading-hosts**. *upgrading-hosts*.
#. Check the status of the upgrade. #. Check the status of the upgrade.
@ -342,9 +343,9 @@ For more details, see :ref:`Detailed contents of a system backup
During the running of the :command:`upgrade-activate` command, new During the running of the :command:`upgrade-activate` command, new
configurations are applied to the controller. 250.001 \(**hostname configurations are applied to the controller. 250.001 \(**hostname
Configuration is out-of-date**\) alarms are raised and are cleared as the Configuration is out-of-date**\) alarms are raised and then cleared as the
configuration is applied. The upgrade state goes from **activating** to configuration is applied. The upgrade state goes from *activating* to
**activation-complete** once this is done. *activation-complete* once this is done.
.. code-block:: none .. code-block:: none
@ -360,23 +361,25 @@ For more details, see :ref:`Detailed contents of a system backup
The following states apply when this command is executed. The following states apply when this command is executed.
**activation-requested** ``activation-requested``
State entered when :command:`system upgrade-activate` is executed. State entered when :command:`system upgrade-activate` is executed.
**activating** ``activating``
State entered when we have started activating the upgrade by applying State entered when we have started activating the upgrade by applying
new configurations to the controller and compute hosts. new configurations to the controller and compute hosts.
**activating-hosts** ``activating-hosts``
State entered when applying host-specific configurations. This state is State entered when applying host-specific configurations. This state is
entered only if needed. entered only if needed.
**activation-complete** ``activation-complete``
State entered when new configurations have been applied to all State entered when new configurations have been applied to all
controller and compute hosts. controller and compute hosts.
#. Check the status of the upgrade again to see it has reached #. Check the status of the upgrade again to see it has reached
**activation-complete**. ``activation-complete``.
.. code-block:: none .. code-block:: none