.. WARNING: Add no lines of text between the label immediately following .. and the title. .. _qat-device-plugin-configuration-616551306371: =============================== QAT Device Plugin Configuration =============================== Intel® QuickAssist Technology (Intel® QAT) accelerates cryptographic workloads by offloading the data to hardware that is capable of optimizing those functions. This section describes how to enable and consume the Intel |QAT| device plugin in |prod|. .. rubric:: |prereq| - The host should have Intel |QAT| hardware. Supported |QAT| devices are 4940 and 4942. After |prod| is installed, do the following verification to ensure |QAT| devices are configured. - Verify |QAT| |SRIOV| physical functions are configured. .. code-block:: $ for i in 4942 4940; do lspci -d 8086:$i; done - Verify |QAT| |SRIOV| virtual functions are configured. .. code-block:: $ for i in 4943 4941; do lspci -d 8086:$i; done $ sudo /etc/init.d/qat_service status # Must list all the virtual functions Checking status of all devices. There is 34 QAT acceleration device(s) in the system: qat_dev0 - type: 4xxx, inst_id: 0, node_id: 0, bsf: 0000:f3:00.0, #accel: 1 #engines: 9 state: up qat_dev1 - type: 4xxx, inst_id: 1, node_id: 0, bsf: 0000:f7:00.0, #accel: 1 #engines: 9 state: up qat_dev2 - type: 4xxxvf, inst_id: 0, node_id: 0, bsf: 0000:f3:00.1, #accel: 1 #engines: 1 state: up qat_dev3 - type: 4xxxvf, inst_id: 1, node_id: 0, bsf: 0000:f3:00.2, #accel: 1 #engines: 1 state: up - Verify the |QAT| driver ``vfio_pci`` is installed. .. code-block:: $ lsmod | grep vfio_pci vfio_pci 69632 0 vfio_virqfd 16384 1 vfio_pci fio 45056 4 intel_qat,vfio_mdev,vfio_iommu_type1,vfio_pci irqbypass 16384 3 intel_qat,vfio_pci,kvm - Node Feature Discovery application must be installed, using the following commands. .. code-block:: ~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/node-feature-discovery-.tgz ~(keystone_admin)]$ system application-apply node-feature-discovery Replace ```` with the latest version number. Enable Intel QAT Device Plugin ------------------------------ The following steps should be performed to enable the Intel |QAT| device plugin for discovering and advertising |QAT| VF (Virtual Functions) resources to Kubernetes host. #. Locate the application tarball in the ``/usr/local/share/applications/helm`` directory. For example: ``/usr/local/share/applications/helm/intel-device-plugins-operator-.tgz`` #. Upload the application using the following command. .. code-block:: ~(keystone_admin)]$ system application-upload intel-device-plugins-operator-.tgz Replace ```` with the latest version number. #. Verify that the application has been uploaded successfully. .. code-block:: ~(keystone_admin)]$ system application-list #. Check the Helm chart status. .. code-block:: ~(keystone_admin)]$ system helm-override-list intel-device-plugins-operator --long #. Enable QAT helm chart. .. code-block:: ~(keystone_admin)]$ system helm-chart-attribute-modify --enabled true intel-device-plugins-operator intel-device-plugins-qat intel-device-plugins-operator #. Apply the application. .. code-block:: ~(keystone_admin)]$ system application-apply intel-device-plugins-operator #. Monitor the status of the application. .. code-block:: ~(keystone_admin)]$ watch -n 5 system application-list OR .. code-block:: none ~(keystone_admin)]$ watch kubectl get pods -n intel-device-plugins-operator #. Check the pods. .. code-block:: $ kubectl get pods -n intel-device-plugins-operator NAME READY STATUS RESTARTS AGE intel-qat-plugin-qatdeviceplugin-sample-g8n45 1/1 Running 0 34s inteldeviceplugins-controller-manager-74f4c 2/2 Running 0 64s #. Verify |QAT| devices by checking the node's resource allocations. The |QAT| 4940 device and the |QAT| 4942 device each have 16 virtual functions. If both devices are present, the following command will display a total of 32 virtual functions: .. code-block:: $ kubectl describe node | grep qat.intel.com/sym-dc Capacity: --- qat.intel.com/sym-dc: 32 --- Allocatable: --- qat.intel.com/sym-dc: 32 --- Use Intel QAT Device Plugin --------------------------- This section describes the steps for using |QAT| device plugin. #. Deploy a pod using the following sample POD specification file. The pod specification file can be modified for required resource request and limit. The ``qat.intel.com/sym-dc: `` field is used to configure the requested |QAT| virtual functions. For a |DPDK|-based workload, you may need to add a hugepage request and limit. ``qat-dpdk.yaml`` .. code-block:: yaml kind: Pod apiVersion: v1 metadata: name: dpdk-test-crypto-perf spec: containers: - name: crypto-perf image: intel/crypto-perf:devel imagePullPolicy: IfNotPresent command: [ "/bin/bash", "-c", "--" ] args: [ "while true; do sleep 300000; done;" ] volumeMounts: - mountPath: /dev/hugepages name: hugepage - mountPath: /var/run/dpdk name: dpdk-runtime resources: requests: cpu: "3" memory: "128Mi" qat.intel.com/sym-dc: '4' hugepages-2Mi: "128Mi" limits: cpu: "3" memory: "128Mi" qat.intel.com/sym-dc: '4' hugepages-2Mi: "128Mi" securityContext: readOnlyRootFilesystem: true allowPrivilegeEscalation: false capabilities: add: ["IPC_LOCK"] restartPolicy: Never volumes: - name: dpdk-runtime emptyDir: medium: Memory - name: hugepage emptyDir: medium: HugePages Apply the pod specification file to create ``dpdk-test-crypto-perf`` pod. .. code-block:: $ kubectl apply -f qat-dpdk.yaml #. Verify the pod status and the allocated |QAT| virtual functions. .. code-block:: $ kubectl get pods NAME READY STATUS RESTARTS AGE dpdk-test-crypto-perf 1/1 Running 0 27m $ kubectl describe pod dpdk-test-crypto-perf Requests: --- qat.intel.com/sym-dc: 4 --- $ kubectl describe node Allocated resources: --- qat.intel.com/sym-dc: 4 --- For more information, see: `Demos and Testing `__.