From 579a08d6fd1f75a6df61e21f8f55b6d87b5200e9 Mon Sep 17 00:00:00 2001 From: Elisamara Aoki Goncalves Date: Wed, 13 Apr 2022 22:35:37 -0300 Subject: [PATCH] Update Docs to include Hybrid Cluster (r6) Add documentation about "hybrid clusters" where there are a mix of Kubernetes and OpenStack nodes in the same cluster. Add documentation about converting worker nodes to compute nodes. Removed new content from r5 as it was not relevant. Fixed typos. New step added in Convert Worker Nodes section. Signed-off-by: Elisamara Aoki Goncalves Change-Id: Ie39bc8b94e2281649a26b13ceb778325ff53422e --- ...ck-node-in-hybrid-cluster-e8b37e8d1b48.rst | 36 ++++++ .../convert-worker-nodes-0007b1532308.rst | 103 ++++++++++++++++++ .../openstack/hybrid-cluster-c7a3134b6f2a.rst | 45 ++++++++ .../index-install-r6-os-adc44604968c.rst | 11 ++ 4 files changed, 195 insertions(+) create mode 100644 doc/source/deploy_install_guides/r6_release/openstack/add-taints-to-openstack-node-in-hybrid-cluster-e8b37e8d1b48.rst create mode 100644 doc/source/deploy_install_guides/r6_release/openstack/convert-worker-nodes-0007b1532308.rst create mode 100644 doc/source/deploy_install_guides/r6_release/openstack/hybrid-cluster-c7a3134b6f2a.rst diff --git a/doc/source/deploy_install_guides/r6_release/openstack/add-taints-to-openstack-node-in-hybrid-cluster-e8b37e8d1b48.rst b/doc/source/deploy_install_guides/r6_release/openstack/add-taints-to-openstack-node-in-hybrid-cluster-e8b37e8d1b48.rst new file mode 100644 index 000000000..5c794f6a7 --- /dev/null +++ b/doc/source/deploy_install_guides/r6_release/openstack/add-taints-to-openstack-node-in-hybrid-cluster-e8b37e8d1b48.rst @@ -0,0 +1,36 @@ +.. _add-taints-to-openstack-node-in-hybrid-cluster-e8b37e8d1b48-r6: + +============================================== +Add Taints to OpenStack Node in Hybrid Cluster +============================================== + +.. rubric:: |context| + +In a hybrid (Kubernetes and OpenStack) cluster scenario, to prevent end-users' +hosted containerized workloads/pods from being scheduled on +``openstack-compute-nodes`` a taint that only openstack and platform pods are +able to tolerate must be added. Thus, any pod that is not platform or +OpenStack specific can be repelled. To achieve this control, the +``openstack-compute-node`` taint must be added to all +``openstack-compute-nodes`` (i.e. worker nodes or |AIO|-Controller nodes with +the ``openstack-compute-node`` label). + +By applying taints, it is possible to separate end users' containerized +workloads/pods from OpenStack in a hybrid (Kubernetes and OpenStack) cluster +scenario. + +.. rubric:: |proc| + +#. Apply taints. + + This step is needed to prevent end-users' hosted containerized + workloads/pods from being scheduled on openstack computes (i.e. worker + nodes or |AIO|-Controller nodes with the ``openstack-compute-node`` label). + + Apply this taint on every ``openstack-compute-node`` (i.e. worker nodes + or |AIO|-Controller nodes with the ``openstack-compute-node`` label): + + .. code-block:: none + + kubectl taint nodes openstack-compute-node:NoSchedule + diff --git a/doc/source/deploy_install_guides/r6_release/openstack/convert-worker-nodes-0007b1532308.rst b/doc/source/deploy_install_guides/r6_release/openstack/convert-worker-nodes-0007b1532308.rst new file mode 100644 index 000000000..ced6d1488 --- /dev/null +++ b/doc/source/deploy_install_guides/r6_release/openstack/convert-worker-nodes-0007b1532308.rst @@ -0,0 +1,103 @@ +.. _convert-worker-nodes-0007b1532308: + +==================== +Convert Worker Nodes +==================== + +.. rubric:: |context| + +In a hybrid (Kubernetes and OpenStack) cluster scenario you may need to convert +worker nodes to/from ``openstack-compute-nodes``. + +.. rubric:: |proc| + +#. Convert a k8s-only worker into a OpenStack compute + + #. Lock the worker host: + + .. code-block:: none + + system host-lock + + #. Add the ``openstack-compute-node`` taint, see :ref:`Add Taints to + OpenStack Node in Hybrid Cluster + `. + + #. Assign OpenStack labels: + + .. code-block:: none + + system host-label-assign --overwrite openstack-compute-node=enabled avs=enabled sriov=enabled + + #. Allocate vswitch huge pages: + + .. code-block:: none + + system host-memory-modify -1G 1 -f vswitch 0 + system host-memory-modify -1G 1 -f vswitch 1 + + #. Change the class of the data network interface: + + .. code-block:: none + + system host-if-modify -c data + + .. note:: + + If data network interface does not exist yet, refer to |prod-os| + documentation on creating it. + + #. Change Kubernetes CPU Manager Policy to allow |VMs| to use application + cores: + + .. code-block:: none + + system host-label-remove kube-cpu-mgr-policy + + #. Unlock the worker host: + + .. code-block:: none + + system host-unlock + +#. Convert a OpenStack compute into a k8s-only worker. + + #. Lock the worker host: + + .. code-block:: none + + system host-lock + + #. Remove OpenStack labels: + + .. code-block:: none + + system host-label-remove openstack-compute-node avs sriov + + .. note:: + + The labels have to be removed, not to have its values changed. + + #. Deallocate vswitch huge pages: + + .. code-block:: none + + system host-memory-modify -1G 0 -f vswitch 0 + system host-memory-modify -1G 0 -f vswitch 1 + + #. Change the class of the data network interface: + + .. code-block:: none + + system host-if-modify -c none + + .. note:: + + This change is needed to avoid raising a permanent alarm for the + interface without the need to delete it. + + #. Unlock the worker host: + + .. code-block:: none + + system host-unlock diff --git a/doc/source/deploy_install_guides/r6_release/openstack/hybrid-cluster-c7a3134b6f2a.rst b/doc/source/deploy_install_guides/r6_release/openstack/hybrid-cluster-c7a3134b6f2a.rst new file mode 100644 index 000000000..da5a9b699 --- /dev/null +++ b/doc/source/deploy_install_guides/r6_release/openstack/hybrid-cluster-c7a3134b6f2a.rst @@ -0,0 +1,45 @@ +.. _hybrid-cluster-c7a3134b6f2a: + +============== +Hybrid Cluster +============== + +A Hybrid Cluster occurs when the hosts with a worker function (|AIO| +controllers and worker nodes) are split between two groups, one running +|prod-os| for hosting |VM| payloads and the other for hosting containerized +payloads. + +The host labels are used to define each worker function on the Hybrid Cluster +setup. For example, a standard configuration (2 controllers and 2 computes) can +be split into (2 controllers, 1 openstack-compute and 1 kubernetes-worker). + +----------- +Limitations +----------- + +- Worker function on |AIO| controllers MUST both be either + Kubernetes or OpenStack. + + - Hybrid Cluster does not apply to |AIO-SX| or |AIO-DX| setups. + +- A worker must have only one function, either it is OpenStack compute or + k8s-only worker, never both at the same time. + + - The ``sriov`` and ``sriovdp`` labels cannot coexist on the same host, + in order to prevent the |SRIOV| device plugin from conflicting with the + OpenStack |SRIOV| driver. + + - No host will assign |VMs| and application containers to application cores + at the same time. + +- Standard Controllers cannot have ``openstack-compute-node`` label; + only |AIO| Controllers can have ``openstack-compute-node`` label. + +- Taints must be added to OpenStack compute hosts (i.e. worker nodes or + |AIO|-Controller nodes with the ``openstack-compute-node`` label) to + prevent + end users' hosted containerized workloads/pods from being scheduled on + OpenStack compute hosts. See :ref:`Add Taints to OpenStack Node in + Hybrid Cluster + `. + diff --git a/doc/source/deploy_install_guides/r6_release/openstack/index-install-r6-os-adc44604968c.rst b/doc/source/deploy_install_guides/r6_release/openstack/index-install-r6-os-adc44604968c.rst index 36938726d..cf0ecaeef 100644 --- a/doc/source/deploy_install_guides/r6_release/openstack/index-install-r6-os-adc44604968c.rst +++ b/doc/source/deploy_install_guides/r6_release/openstack/index-install-r6-os-adc44604968c.rst @@ -16,3 +16,14 @@ StarlingX is independent of deployment configuration. install access uninstall_delete + +-------------- +Hybrid Cluster +-------------- + +.. toctree:: + :maxdepth: 1 + + hybrid-cluster-c7a3134b6f2a + add-taints-to-openstack-node-in-hybrid-cluster-e8b37e8d1b48 + convert-worker-nodes-0007b1532308