diff --git a/doc/source/deploy/deploy-edgeworker-nodes.rst b/doc/source/deploy/deploy-edgeworker-nodes.rst new file mode 100644 index 000000000..c80282045 --- /dev/null +++ b/doc/source/deploy/deploy-edgeworker-nodes.rst @@ -0,0 +1,261 @@ +======================= +Deploy Edgeworker Nodes +======================= + +.. contents:: + :local: + :depth: 1 + +------------ +Introduction +------------ + +Edgeworker is a new personality of node introduced in stx.5.0 [1]_. Ubuntu +is supported for a node with edgeworker personality as its operating system [2]_. +Edgeworkers are typically small systems running dedicated workloads that can now +be managed by |prod|. + +The edgeworker personality is targeted for nodes with below criteria: + +- A node is running a customized OS, or +- A node is running a Type1 hypervisor, or +- A node does not meet |prod| worker’s minimum requirements. + +These nodes can be added to a running |prod| cluster in minutes. During the +provisioning stage, Kubernetes components will be provisioned on the edgeworker +nodes. + +The |prod| controller manages edgeworker nodes as normal worker nodes including +the following functionalities: + +- Container orchestration by Kubernetes +- Distributed storage cluster management by Rook-ceph + +.. rubric:: Notes + +.. [1] Edgeworker is an experimental feature in |prefix|.5.0. +.. [2] The OS of an edgeworker node is not installed by |prod| controller,instead, a node with installed OS is required prior to the node provisioning step. Only Ubuntu is supported in stx.5.0. + +--------------------- +Hardware Requirements +--------------------- + +The minimum requirements for edgeworker host (bare metal or virtual machine) +are: + ++-------------------------+-------------------------------+ +| Minimum Requirement | Edgeworker Node | ++=========================+===============================+ +| Minimum processor class | Intel® Core™ Processor Family | ++-------------------------+-------------------------------+ +| Minimum memory | 16GB | ++-------------------------+-------------------------------+ +| Primary disk | 256GB SSD or NVMe | ++-------------------------+-------------------------------+ +| Minimum network ports | - Mgmt/Cluster: 1 x 1000Base | +| | - Data: 0 or more x 10GE | ++-------------------------+-------------------------------+ + +------------------------ +Prepare Edgeworker Nodes +------------------------ + +Prior to provisioning edgeworker nodes, the setup must be prepared in the +following condition: + +- |prod| AIO-Duplex or Standard setup deployed. +- Edgeworker node cabled for power and powered on. +- Edgeworker node cabled for networking and connected to mgmt switch. +- Set primary disk as 1st boot device in BIOS setting of the edgeworker node. +- Ubuntu 18.04(or above) has been installed in edgeworker node. +- Openssh-server and python package installed. + +The network connection of edgeworker nodes to the existing |prod| cluster is +shown in below figure. + +.. image:: figures/edgeworker-deployment.png + :width: 800 + +-------------------------- +Provision Edgeworker Nodes +-------------------------- + +By connecting a node’s network interface to the management network switch, a +node will be detected and will appear in the list of hosts managed by system +inventory (with personality set to None). + +.. code-block:: bash + + [sysadmin@controller-0 ~(keystone_admin)]$ system host-list + +----+--------------+-------------+----------------+-------------+--------------+ + | id | hostname | personality | administrative | operational | availability | + +----+--------------+-------------+----------------+-------------+--------------+ + | 1 | controller-0 | controller | unlocked | enabled | available | + | 2 | controller-1 | controller | unlocked | enabled | available | + | 3 | None | None | locked | disabled | offline | + +----+--------------+-------------+----------------+-------------+--------------+ + +#. Update the node’s hostname and personality by `system host-update`. + + .. code-block:: bash + + [sysadmin@controller-0 ~(keystone_admin)]$ system host-update 3 hostname=edgeworker-0 personality=edgeworker + +-----------------------+--------------------------------------+ + | Property | Value | + +-----------------------+--------------------------------------+ + | action | none | + | administrative | locked | + | availability | offline | + | bm_ip | None | + | bm_type | None | + | bm_username | None | + | boot_device | /dev/sda | + | capabilities | {} | + | clock_synchronization | ntp | + | config_applied | None | + | config_status | None | + | config_target | None | + | console | ttyS0,115200 | + | created_at | 2021-02-05T02:12:42.136121+00:00 | + | device_image_update | None | + | hostname | edgeworker-0 | + | id | 3 | + | install_output | text | + | install_state | None | + | install_state_info | None | + | inv_state | None | + | invprovision | None | + | location | {} | + | mgmt_ip | 192.188.204.21 | + | mgmt_mac | 52:54:00:76:e6:cb | + | operational | disabled | + | personality | edgeworker | + | reboot_needed | False | + | reserved | False | + | rootfs_device | /dev/sda | + | serialid | None | + | software_load | 20.12 | + | task | None | + | tboot | false | + | ttys_dcd | None | + | updated_at | None | + | uptime | 0 | + | uuid | a2c8f3ec-1dc4-4a0a-b471-6e5c01abc187 | + | vim_progress_status | None | + +-----------------------+--------------------------------------+ + + Alternatively, if the node is not shown in host list automatically, you can + also add the host with `system host-add`. + + .. code-block:: bash + + [sysadmin@controller-0 ~(keystone_admin)]$ system host-add -n edgeworker-0 -p edgeworker -m + +#. Check pre-requisites of the edgeworker node: + + .. note:: + + We suggest using the same username and password for all the edgeworker nodes. + + .. code-block:: bash + + EWUSER= + EWPASSWD= + SYSADMINPASSWD= + EWNODE=(edgeworker-0) + PYTHONBIN=() + for NODE in ${EWNODE[@]}; do + EDGEWORKER=$(system host-show ${NODE} | awk '/mgmt_ip/{print$4}'); + PYTHONBIN+=($(sshpass -p ${EWPASSWD} ssh -o "StrictHostKeyChecking no" ${EWUSER}@${EDGEWORKER} which python)); + done + ERR=0 + for i in ${PYTHONBIN[@]}; do + if [ -z "$i" ]; then + ERR=1 + break + fi + done + if [ $ERR == 1 ]; then + echo "Error, python should be intalled on edgeworker nodes." + else + echo "Pre-requisites check passed." + fi + + If the following error occurrs, refresh the |DHCP| client for the mgmt interface on the edgeworker nodes to get the right ip address assigned. + + .. code-block:: none + + ssh: connect to host XX.XX.XX.XX port 22: No route to host + +#. Create an edgeworker inventory file with the variables from the last step. + + .. code-block:: none + + tee ./edgeworker_inventory.yml << EOF + all: + hosts: + localhost: + ansible_connection: local + children: + edgeworker: + hosts: + EOF + for i in ${!EWNODE[@]}; do + echo " ${EWNODE[$i]}:" + echo " ansible_ssh_user: ${EWUSER}" + echo " ansible_ssh_pass: ${EWPASSWD}" + echo " ansible_become_pass: ${EWPASSWD}" + echo " ansible_python_interpreter: ${PYTHONBIN[$i]}" + done | tee -a edgeworker_inventory.yml + tee -a ./edgeworker_inventory.yml << EOF + vars: + ansible_ssh_user: sysadmin + ansible_ssh_pass: ${SYSADMINPASSWD} + ansible_become_pass: ${SYSADMINPASSWD} + EOF + +#. Provision the edgeworker node using ansible playbook. You can provision + edgeworker node one at a time or multiple nodes at once. Run the playbook + with the underlying inventory + + .. code-block:: bash + + ansible-playbook -i ./edgeworker_inventory.yml /usr/share/ansible/stx-ansible/playbooks/provision_edgeworker.yml + + After the provisioning, the edgeworker node will be Ready in Kubernetes + cluster. + + .. code-block:: bash + + TASK [provision-edgeworker/prepare-edgeworker/kubernetes : set_fact] ********************************************************************************************************* + ok: [edgeworker-0] + + TASK [provision-edgeworker/prepare-edgeworker/kubernetes : Pull k8s gcr images from controller registry to edgeworker-0] ***************************************************** + changed: [edgeworker-0] + + RUNNING HANDLER [provision-edgeworker/prepare-edgeworker/kubernetes : Restart kubelet] *************************************************************************************** + changed: [edgeworker-0] + + PLAY [localhost] ************************************************************************************************************************************************************* + + TASK [provision-edgeworker/prepare-controller/cleanup : Cleanup temporary files] ********************************************************************************************* + changed: [localhost] + + TASK [provision-edgeworker/prepare-controller/cleanup : Remove the provision flag] ******************************************************************************************* + changed: [localhost] + + PLAY RECAP ******************************************************************************************************************************************************************* + edgeworker-0 : ok=54 changed=10 unreachable=0 failed=0 + localhost : ok=31 changed=19 unreachable=0 failed=0 + + .. code-block:: bash + + [sysadmin@controller-0 ~(keystone_admin)]$ kubectl get node + NAME STATUS ROLES AGE VERSION + controller-0 Ready master 42h v1.18.1 + controller-1 Ready master 42h v1.18.1 + edgeworker-0 Ready 21m v1.18.1 + +.. note:: + + The edgeworker nodes will remain locked/disabled/offline in the host inventory list in stx5.0. diff --git a/doc/source/deploy/figures/edgeworker-deployment.png b/doc/source/deploy/figures/edgeworker-deployment.png new file mode 100644 index 000000000..6cffd848f Binary files /dev/null and b/doc/source/deploy/figures/edgeworker-deployment.png differ diff --git a/doc/source/deploy/index.rst b/doc/source/deploy/index.rst index 6257c7d59..f87549732 100644 --- a/doc/source/deploy/index.rst +++ b/doc/source/deploy/index.rst @@ -17,3 +17,4 @@ Deployment Configurations deployment-and-configuration-options-standard-configuration-with-controller-storage standard-configuration-with-dedicated-storage worker-function-performance-profiles + deploy-edgeworker-nodes diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst index eb17eb4dc..a5dbad978 100644 --- a/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/aio_duplex_extend.rst @@ -44,6 +44,11 @@ Install software on worker nodes This initiates the install of software on worker nodes. This can take 5-10 minutes, depending on the performance of the host machine. + .. Note:: + + A node with EdgeWorker personality is also available, please refer to + :doc:`/deploy/deploy-edgeworker-nodes` + #. Wait for the install of software on the worker nodes to complete, for the worker nodes to reboot, and for both to show as locked/disabled/online in 'system host-list'. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst index e44fa891a..2bbb5837f 100644 --- a/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/controller_storage_install_kubernetes.rst @@ -396,6 +396,11 @@ Install software on controller-1 and worker nodes system host-update 4 personality=worker hostname=worker-1 + .. Note:: + + A node with EdgeWorker personality is also available, please refer to + :doc:`/deploy/deploy-edgeworker-nodes` + #. Wait for the software installation on controller-1, worker-0, and worker-1 to complete, for all servers to reboot, and for all to show as locked/disabled/online in 'system host-list'. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst index 70695ad1f..0d4128cb7 100644 --- a/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/dedicated_storage_install_kubernetes.rst @@ -142,6 +142,11 @@ Install software on controller-1, storage nodes, and worker nodes This initiates the install of software on worker-0 and worker-1. + .. Note:: + + A node with EdgeWorker personality is also available, please refer to + :doc:`/deploy/deploy-edgeworker-nodes` + #. Wait for the software installation on controller-1, storage-0, storage-1, worker-0, and worker-1 to complete, for all servers to reboot, and for all to show as locked/disabled/online in 'system host-list'. diff --git a/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst index 94d9fde8f..dc93eb57d 100644 --- a/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst +++ b/doc/source/deploy_install_guides/r5_release/bare_metal/rook_storage_install_kubernetes.rst @@ -396,6 +396,11 @@ Install software on controller-1 and worker nodes system host-update 5 personality=worker hostname=storage-0 system host-update 6 personality=worker hostname=storage-1 + .. Note:: + + A node with EdgeWorker personality is also available, please refer to + :doc:`/deploy/deploy-edgeworker-nodes` + #. Wait for the software installation on controller-1, worker-0, and worker-1 to complete, for all servers to reboot, and for all to show as locked/disabled/online in 'system host-list'. diff --git a/doc/source/deploy_install_guides/r5_release/virtual/controller_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/virtual/controller_storage_install_kubernetes.rst index a23a43e03..2605e715b 100644 --- a/doc/source/deploy_install_guides/r5_release/virtual/controller_storage_install_kubernetes.rst +++ b/doc/source/deploy_install_guides/r5_release/virtual/controller_storage_install_kubernetes.rst @@ -331,6 +331,11 @@ Install software on controller-1 and worker nodes system host-update 4 personality=worker hostname=worker-1 + .. Note:: + + A node with EdgeWorker personality is also available, please refer to + :doc:`/deploy/deploy-edgeworker-nodes` + #. Wait for the software installation on controller-1, worker-0, and worker-1 to complete, for all virtual servers to reboot, and for all to show as locked/disabled/online in 'system host-list'. diff --git a/doc/source/deploy_install_guides/r5_release/virtual/dedicated_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/virtual/dedicated_storage_install_kubernetes.rst index 762076a9a..dec82f4b2 100644 --- a/doc/source/deploy_install_guides/r5_release/virtual/dedicated_storage_install_kubernetes.rst +++ b/doc/source/deploy_install_guides/r5_release/virtual/dedicated_storage_install_kubernetes.rst @@ -184,6 +184,11 @@ Install software on controller-1, storage nodes, and worker nodes This initiates software installation on worker-0 and worker-1. + .. Note:: + + A node with EdgeWorker personality is also available, please refer to + :doc:`/deploy/deploy-edgeworker-nodes` + #. Wait for the software installation on controller-1, storage-0, storage-1, worker-0, and worker-1 to complete, for all virtual servers to reboot, and for all to show as locked/disabled/online in 'system host-list'. diff --git a/doc/source/deploy_install_guides/r5_release/virtual/rook_storage_install_kubernetes.rst b/doc/source/deploy_install_guides/r5_release/virtual/rook_storage_install_kubernetes.rst index 9f46c180c..1de086c18 100644 --- a/doc/source/deploy_install_guides/r5_release/virtual/rook_storage_install_kubernetes.rst +++ b/doc/source/deploy_install_guides/r5_release/virtual/rook_storage_install_kubernetes.rst @@ -256,6 +256,11 @@ Install software on controller-1 and worker nodes This initiates software installation on worker-0 and worker-1. + .. Note:: + + A node with EdgeWorker personality is also available, please refer to + :doc:`/deploy/deploy-edgeworker-nodes` + #. Wait for the software installation on controller-1, storage-0, storage-1, worker-0, and worker-1 to complete, for all virtual servers to reboot, and for all to show as locked/disabled/online in 'system host-list'.