dd8efe5f85
Copy edits for typos, markup and other technical issues. Fix case typo Fix merge conflicts. Resolve FixMe issues. Minor copy edit updates. Signed-off-by: Ron Stone <ronald.stone@windriver.com> Change-Id: I7d710d3faa957c1cb66cc18c9a248a567e7c580f
126 lines
3.7 KiB
ReStructuredText
126 lines
3.7 KiB
ReStructuredText
|
|
.. jkf1590184623714
|
|
.. _handling-kubernetes-update-orchestration-failures:
|
|
|
|
========================================================
|
|
Handle Kubernetes Version Upgrade Orchestration Failures
|
|
========================================================
|
|
|
|
The creation or application of a strategy could fail for any of the listed
|
|
reasons described in this section. Follow the suggested actions in each case to
|
|
resolve the issue.
|
|
|
|
.. _jkf1590184623714-section-fhk-nnq-5lb:
|
|
|
|
-------------------------
|
|
Strategy creation failure
|
|
-------------------------
|
|
|
|
.. _jkf1590184623714-ul-fvs-vnq-5lb:
|
|
|
|
- **Reason**: Build failed with no reason.
|
|
|
|
- **Action**:
|
|
|
|
- Verify that the ``--worker-apply-type`` was not set to ``ignore``.
|
|
|
|
- Check recent logs added to /var/log/nfv-vim.log.
|
|
|
|
|
|
- **Reason**: Alarms from platform are present.
|
|
|
|
- **Action**:
|
|
|
|
- Query for management affecting alarms and take actions to clear
|
|
them.
|
|
|
|
.. code-block:: none
|
|
|
|
~(keystone_admin)$ fm alarm-list --mgmt_affecting
|
|
|
|
- If there are no management affecting alarms present, take actions
|
|
to clear other reported alarms or try creating the strategy with
|
|
the ``relaxed`` alarms restrictions option ``--alarm-restrictions
|
|
relaxed``.
|
|
|
|
- **Reason**: No Kubernetes version upgrade required.
|
|
|
|
- **Action**:
|
|
|
|
- Verify that the Kubernetes patches have been uploaded and applied.
|
|
Verify the version of Kubernetes on the hosts by executing "system
|
|
kube-host-upgrade-list.
|
|
|
|
.. note::
|
|
If the strategy create failed, first you must resolve it. You
|
|
must delete the failed strategy before you create another
|
|
strategy.
|
|
|
|
|
|
.. _jkf1590184623714-section-ppt-gpq-5lb:
|
|
|
|
----------------------
|
|
Strategy Apply Failure
|
|
----------------------
|
|
|
|
.. _jkf1590184623714-ul-rdf-4pq-5lb:
|
|
|
|
- **Reason**: Alarms from platform are present.
|
|
|
|
- **Action**: This suggests that an alarm has been raised since the
|
|
creation of the strategy. Address the cause of the new alarm, delete the
|
|
strategy and try creating and applying a new strategy.
|
|
|
|
|
|
- **Reason**: Unable to migrate instances.
|
|
|
|
- **Action**: See :ref:`Kubernetes Version Upgrade Operations Requiring
|
|
Manual Migration
|
|
<kubernetes-update-operations-requiring-manual-migration>` for steps to
|
|
resolve migration issues.
|
|
|
|
- **Reason**: Kubernetes version upgrade failed. Suggests that the Kubernetes
|
|
upgrade for the specified host has failed.
|
|
|
|
.. only:: starlingx
|
|
|
|
- **Action**: Consult the `StarlingX community
|
|
<https://www.starlingx.io/community/>`_.
|
|
|
|
.. only:: partner
|
|
|
|
.. include:: /_includes/handling-kubernetes-update-orchestration-failures.rest
|
|
|
|
- **Reason**: Lock host failed.
|
|
|
|
- **Action**:
|
|
|
|
- Investigate the ``/var/log/sysinv.log``, and ``/var/log/nfv-vim.log``
|
|
files.
|
|
|
|
- Address the underlying issue.
|
|
|
|
- Manually lock and unlock the host.
|
|
|
|
- Try recreating and re-applying the Kubernetes version upgrade
|
|
strategy to automatically finish the upgrade process.
|
|
|
|
|
|
- **Reason**: Unlock host failed.
|
|
|
|
- **Action**:
|
|
|
|
- Investigate ``/var/log/mtcAgent.log`` file for cause logs files.
|
|
|
|
- Address the underlying issue.
|
|
|
|
- Manually lock and unlock the host to recover.
|
|
|
|
- Try recreating and re-applying the Kubernetes version upgrade
|
|
strategy to automatically finish the upgrade process.
|
|
|
|
.. note::
|
|
If the strategy :command:`apply` fails, you must resolve the
|
|
strategy:command:`apply` failure, and delete the failed strategy before
|
|
trying to create and apply another strategy.
|