diff --git a/doc/source/operations/index.rst b/doc/source/operations/index.rst index 9e86786aa..c751f85e3 100644 --- a/doc/source/operations/index.rst +++ b/doc/source/operations/index.rst @@ -35,7 +35,6 @@ Kubernetes Operation k8s_persistent_vol_claims k8s_sriov_config k8s_gpu_device_plugin - kata_container ------------------- OpenStack Operation diff --git a/doc/source/operations/kata_container.rst b/doc/source/operations/kata_container.rst deleted file mode 100644 index 74aa361a5..000000000 --- a/doc/source/operations/kata_container.rst +++ /dev/null @@ -1,156 +0,0 @@ -=============== -Kata Containers -=============== - -This guide describes how to run Kata Containers with Kubernetes on StarlingX. - -.. contents:: - :local: - :depth: 1 - --------- -Overview --------- - -StarlingX has supported Kata Containers in master since January 2020, and coming -Release 4.0 will include it. Release 3.0 and before will not support it. - -To support Kata Containers, pods are created by containerd instead of Docker. -Also containerd is configured to support both runc and Kata Containers, while the -default runtime is still runc. If you want to launch a pod with Kata Containers, -you must declare it explicitly. - ---------------------------------- -Run Kata Containers in Kubernetes ---------------------------------- - -There are two methods to run Kata Containers in Kubernetes: by runtime class or -by annotation. Runtime class is supported in Kubernetes since v1.12.0, and it is -the recommended method for running Kata Containers. - -To run by runtime class, create a RuntimeClass with ``handler`` set to ``kata``. -Then reference this class in the pod spec, as shown in the following example: - -:: - - kind: RuntimeClass - apiVersion: node.k8s.io/v1 - metadata: - name: kata-containers - handler: kata - --- - apiVersion: v1 - kind: Pod - metadata: - name: busybox-runtime - spec: - runtimeClassName: kata-containers - containers: - - name: busybox - command: - - sleep - - "3600" - image: busybox - -To run a pod with Kata Containers by annotation, set ``io.kubernetes.cri.untrusted-workload`` -to ``true`` in the annotations section of a pod spec. - -.. note:: - - This method is deprecated and may not be supported in future - Kubernetes releases. We recommend using the RuntimeClass method. - -Example of using annotation: - -:: - - apiVersion: v1 - kind: Pod - metadata: - name: busybox-untrusted - annotations: - io.kubernetes.cri.untrusted-workload: "true" - spec: - containers: - - name: busybox - command: - - sleep - - "3600" - image: busybox - --------------------------------- -Kata configuration in containerd --------------------------------- -.. note:: - - No action is required for end user. This section just shows the - configuration in containerd to support Kata Containers. - -Containerd's configuration file ``/etc/containerd/config.toml`` is customized -to support Kata Containers. - -For RuntimeClass, here is the configuration in config.toml: - -:: - - [plugins.cri.containerd.runtimes.kata] - runtime_type = "io.containerd.kata.v2" - -For annotation, here is the configuration in config.toml: - -:: - - [plugins.cri.containerd.runtimes.untrusted] - runtime_type = "io.containerd.kata.v2" - -In this example, ``kata.v2`` means ``shimv2(containerd-shim-kata-v2)``, which -helps Kubernetes to launch pods and OCI-compatible containers with one shim per -pod. - -------------------------- -Check Kata Containers use -------------------------- - -Here are two methods to check whether the pod is running with Kata Containers -or not: - -#. Run ``uname -a`` in both container and host. The host kernel version should - be 4.18.0, while the container kernel version should be 4.19 or higher. - For normal container, the host kernel version is the same as the container. - -#. Run ``ps aux`` in the host. A normal container is triggered by - containerd-shim-runc-v1, while Kata Containers is triggered by - containerd-shim-kata-v2. - ----------- -References ----------- - -* For technical details about how Kata Containers is implemented on StarlingX, - refer to: - - * Spec file: https://opendev.org/starlingx/specs/src/branch/master/doc/source/specs/stx-3.0/approved/containerization-2006145-kata-containers-integration.rst - * Story: https://storyboard.openstack.org/#!/story/2006145 - * Patches: https://review.opendev.org/#/q/topic:kata+projects:starlingx - -* Kata Containers is supported for Kubernetes only, since Kubernetes is the - only supported container orchestration tool in StarlingX. Kata Container - support for Docker is not implemented in StarlingX. The Docker runtime also - may be removed in future releases of StarlingX, since all containers in - StarlingX are run by Kubernetes at this time. - - To try Kata Containers with Docker in StarlingX, refer to this link: - https://github.com/kata-containers/documentation/blob/master/install/docker/centos-docker-install.md - -* To support Kata Containers, the CRI runtime in Kubernetes was switched from - ``dockershim`` to ``containerd``. This means you cannot view/operate a - Kubernetes container with the Docker client. You must use ``crictl`` - instead, which supports commands that are similar to Docker commands. There - is no difference in kubectl commands before and after the switch to - containerd. - -* More information is available at: - - * Kata Containers: https://katacontainers.io/ - * containerd: https://containerd.io/ - * Kubernetes RuntimeClass: https://kubernetes.io/docs/concepts/containers/runtime-class/ diff --git a/doc/source/planning/kubernetes/container-security-df8a251ec03f.rst b/doc/source/planning/kubernetes/container-security-df8a251ec03f.rst index 5edd28ab1..815b69180 100644 --- a/doc/source/planning/kubernetes/container-security-df8a251ec03f.rst +++ b/doc/source/planning/kubernetes/container-security-df8a251ec03f.rst @@ -63,17 +63,13 @@ and volumes through pod security policies, for example: Kata Containers --------------- -.. note:: - - Kata Containers will not be supported in |prod-long| |prod-ver|. - Kata containers are an optional capability on |prod| that provide a secure container runtime with lightweight virtual machines that feel and perform like containers, but provide stronger workload isolation. For improved performance -with relation to isolation, Kata containers leverages hardware-enforced isolation -with virtualization VT extensions. +with relation to isolation, Kata containers leverages hardware-enforced +isolation with virtualization VT extensions. -For more information, see :ref:`starlingx-kubernetes-user-tutorials-overview`. +For more information, see :ref:`kata_container`. --------------------- Pod Security Policies diff --git a/doc/source/usertasks/kubernetes/kata-containers-overview.rst b/doc/source/usertasks/kubernetes/kata-containers-overview.rst index 0eb0e841a..3d956373c 100644 --- a/doc/source/usertasks/kubernetes/kata-containers-overview.rst +++ b/doc/source/usertasks/kubernetes/kata-containers-overview.rst @@ -6,14 +6,84 @@ Kata Containers Overview ======================== -.. note:: - - Kata Containers will not be supported in |prod-long| |prod-ver|. - -|prod| uses a **containerd** :abbr:`CRI (Container Runtime Interface)` that supports -both runc and Kata Container runtimes. The default runtime is runc. If you want -to launch a pod that uses the Kata Container runtime, you must declare it -explicitly. +|prod| uses a **containerd** :abbr:`CRI (Container Runtime Interface)` that +supports both runc and Kata Container runtimes. The default runtime is runc. If +you want to launch a pod that uses the Kata Container runtime, you must declare +it explicitly. You can see more details on how to specify the use of Kata +Container in :ref:`specifying-kata-container-runtime-in-pod-spec`. For more information about Kata containers, see `https://katacontainers.io/ `__. + + +This guide describes how to run Kata Containers with Kubernetes on StarlingX. + +.. _kata_container: + +--------------------------------- +Run Kata Containers in Kubernetes +--------------------------------- + +There are two methods to run Kata Containers in Kubernetes: by runtime class or +by annotation. Runtime class is supported in Kubernetes since v1.12.0 or +higher, and it is the recommended method for running Kata Containers. + +To run by runtime class, create a RuntimeClass with ``handler`` set to ``kata``. +Then reference this class in the pod spec, as shown in the following example: + +.. code-block:: none + + kind: RuntimeClass + apiVersion: node.k8s.io/v1 + metadata: + name: kata-containers + handler: kata + --- + apiVersion: v1 + kind: Pod + metadata: + name: busybox-runtime + spec: + runtimeClassName: kata-containers + containers: + - name: busybox + command: + - sleep + - "3600" + image: busybox + +Alternatively, set the ``io.kubernetes.cri.untrusted-workload`` to ``true``, to +run a pod with Kata Containers. + +.. note:: + + This method is deprecated and may not be supported in future Kubernetes + releases. We recommend using the RuntimeClass method. + +Example of using annotation: + +.. code-block:: none + + apiVersion: v1 + kind: Pod + metadata: + name: busybox-untrusted + annotations: + io.kubernetes.cri.untrusted-workload: "true" + spec: + containers: + - name: busybox + command: + - sleep + - "3600" + image: busybox + + +------------------------- +Check Kata Containers use +------------------------- + +You can check whether the pod is running with Kata Containers or not by running +``ps aux`` in the host. A normal container is triggered by +``containerd-shim-runc-v1``, while Kata Containers is triggered by +``containerd-shim-kata-v2``. diff --git a/doc/source/usertasks/kubernetes/known-limitations.rst b/doc/source/usertasks/kubernetes/known-limitations.rst index 17ca8086d..052b015fd 100644 --- a/doc/source/usertasks/kubernetes/known-limitations.rst +++ b/doc/source/usertasks/kubernetes/known-limitations.rst @@ -6,10 +6,6 @@ Known Kata Container Limitations ================================ -.. note:: - - Kata Containers will not be supported in |prod-long| |prod-ver|. - This section describes the known limitations when using Kata containers. .. _known-limitations-section-tsh-tl1-zlb: @@ -19,11 +15,12 @@ SR-IOV Support -------------- A minimal **kernel** and **rootfs** for Kata containers are shipped with -|prod-long|, and are present at /usr/share/kata-containers/. To enable certain -kernel features such as :abbr:`IOMMU (I/O memory management unit)`, and desired -network kernel modules, a custom kernel image, and rootfs has to be built. For -more information, see `https://katacontainers.io/ -`__. +|prod-long|, and can be found at ``/usr/share/kata-containers/``. To enable +certain kernel features such as :abbr:`IOMMU (I/O memory management unit)`, and +desired network kernel modules, a custom kernel image, and rootfs has to be +built. However, many tests carried out using this solution were unsuccessful. +More details of this solution can be seen in: +https://bugs.launchpad.net/starlingx/+bug/1867927. .. _known-limitations-section-ngp-ty3-bmb: @@ -42,11 +39,17 @@ Hugepages .. _known-limitations-ul-uhz-xy3-bmb: -- Similar to the SR-IOV limitation, hugepage support must be configured in a +- Similar to the |SRIOV| limitation, hugepage support must be configured in a custom Kata kernel. - The size and number of hugepages must be written using the :command:`io.katacontainers.config.hypervisor.kernel_params` annotation. -- Creating a **hugetlbfs** mount for hugepages in the Kata container is +- Creating a ``hugetlbfs`` mount for hugepages in the Kata container is specific to the end user application. + +.. note:: + + This solution does not work in version v3.1.3 of the Kata Containers. A + pod can be started with hugepages specified, but none are actually + allocated to the Kata |VM|. \ No newline at end of file diff --git a/doc/source/usertasks/kubernetes/specifying-kata-container-runtime-in-pod-spec.rst b/doc/source/usertasks/kubernetes/specifying-kata-container-runtime-in-pod-spec.rst index 95edbf1e0..722edf220 100644 --- a/doc/source/usertasks/kubernetes/specifying-kata-container-runtime-in-pod-spec.rst +++ b/doc/source/usertasks/kubernetes/specifying-kata-container-runtime-in-pod-spec.rst @@ -6,66 +6,62 @@ Specify Kata Container Runtime in Pod Spec ========================================== -.. note:: - - Kata Containers will not be supported in |prod-long| |prod-ver|. - You can specify the use of Kata Container runtime in your pod specification by runtime class or by annotation. .. rubric:: |proc| -* Do one of the following: +Choose one of the methods below: - .. table:: - :widths: auto +.. table:: + :widths: auto - +--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+ - | **To use the runtime class method:** | #. Create a RuntimeClass with handler set to kata. | - | | | - | | #. Reference this class in the pod spec, as shown in the following example: | - | | | - | | .. code-block:: none | - | | | - | | kind: RuntimeClass | - | | apiVersion: node.k8s.io/v1 | - | | metadata: | - | | name: kata-containers | - | | handler: kata | - | | --- | - | | apiVersion: v1 | - | | kind: Pod | - | | metadata: | - | | name: busybox-runtime | - | | spec: | - | | runtimeClassName: kata-containers | - | | containers: | - | | - name: busybox | - | | command: | - | | - sleep | - | | - "3600" | - | | image: busybox | - +--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+ - | **To use the annotation method:** | Set io.kubernetes.cri.untrusted-workload to true in the annotations section of a pod spec. | - | | | - | | For example: | - | | | - | | .. code-block:: none | - | | | - | | apiVersion: v1 | - | | kind: Pod | - | | metadata: | - | | name: busybox-untrusted | - | | annotations: | - | | io.kubernetes.cri.untrusted-workload: "true" | - | | spec: | - | | containers: | - | | - name: busybox | - | | command: | - | | - sleep | - | | - "3600" | - | | image: busybox | - | | | - | | .. note:: | - | | This method is deprecated and may not be supported in future releases. | - +--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+ + +--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+ + | **To use the runtime class method:** | #. Create a RuntimeClass with handler set to kata. | + | | | + | | #. Reference this class in the pod spec, as shown in the following example: | + | | | + | | .. code-block:: none | + | | | + | | kind: RuntimeClass | + | | apiVersion: node.k8s.io/v1 | + | | metadata: | + | | name: kata-containers | + | | handler: kata | + | | --- | + | | apiVersion: v1 | + | | kind: Pod | + | | metadata: | + | | name: busybox-runtime | + | | spec: | + | | runtimeClassName: kata-containers | + | | containers: | + | | - name: busybox | + | | command: | + | | - sleep | + | | - "3600" | + | | image: busybox | + +--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+ + | **To use the annotation method:** | Set io.kubernetes.cri.untrusted-workload to true in the annotations section of a pod spec. | + | | | + | | For example: | + | | | + | | .. code-block:: none | + | | | + | | apiVersion: v1 | + | | kind: Pod | + | | metadata: | + | | name: busybox-untrusted | + | | annotations: | + | | io.kubernetes.cri.untrusted-workload: "true" | + | | spec: | + | | containers: | + | | - name: busybox | + | | command: | + | | - sleep | + | | - "3600" | + | | image: busybox | + | | | + | | .. note:: | + | | This method is deprecated and may not be supported in future releases. | + +--------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------+ \ No newline at end of file