docs/doc/source/node_management/kubernetes/qat-device-plugin-configuration-616551306371.rst

QAT Device Plugin Configuration

Intel® QuickAssist Technology (Intel® QAT) accelerates cryptographic workloads by offloading the data to hardware that is capable of optimizing those functions.

This section describes how to enable and consume the Intel device plugin in .

• The host should have Intel hardware. Supported devices are 4940 and 4942. After is installed, do the following verification to ensure devices are configured.

  • Verify physical functions are configured.

    $ for i in 4942 4940; do lspci -d 8086:$i; done

  • Verify virtual functions are configured.

    $ for i in 4943 4941; do lspci -d 8086:$i; done
    
    $ sudo /etc/init.d/qat_service status # Must list all the virtual functions
    
    Checking status of all devices.
    
    There is 34 QAT acceleration device(s) in the system:
    
    qat_dev0 - type: 4xxx, inst_id: 0, node_id: 0, bsf: 0000:f3:00.0,
    #accel: 1 #engines: 9 state: up
    
    qat_dev1 - type: 4xxx, inst_id: 1, node_id: 0, bsf: 0000:f7:00.0,
    #accel: 1 #engines: 9 state: up
    
    qat_dev2 - type: 4xxxvf, inst_id: 0, node_id: 0, bsf: 0000:f3:00.1,
    #accel: 1 #engines: 1 state: up
    
    qat_dev3 - type: 4xxxvf, inst_id: 1, node_id: 0, bsf: 0000:f3:00.2,
    #accel: 1 #engines: 1 state: up

  • Verify the driver vfio_pci is installed.

    $ lsmod | grep vfio_pci
    
    vfio_pci 69632 0
    vfio_virqfd 16384 1 vfio_pci
    fio 45056 4 intel_qat,vfio_mdev,vfio_iommu_type1,vfio_pci
    irqbypass 16384 3 intel_qat,vfio_pci,kvm

• Node Feature Discovery application must be installed, using the following commands.

  ~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/node-feature-discovery-<version>.tgz
  ~(keystone_admin)]$ system application-apply node-feature-discovery

  Replace <version> with the latest version number.

Enable Intel QAT Device Plugin

The following steps should be performed to enable the Intel device plugin for discovering and advertising VF (Virtual Functions) resources to Kubernetes host.

1. Locate the application tarball in the /usr/local/share/applications/helm directory. For example:

   /usr/local/share/applications/helm/intel-device-plugins-operator-<version>.tgz

2. Upload the application using the following command.

   ~(keystone_admin)]$ system application-upload intel-device-plugins-operator-<version>.tgz

   Replace <version> with the latest version number.

3. Verify that the application has been uploaded successfully.

   ~(keystone_admin)]$ system application-list

4. Check the Helm chart status.

   ~(keystone_admin)]$ system helm-override-list intel-device-plugins-operator --long

5. Enable QAT helm chart.

   ~(keystone_admin)]$ system helm-chart-attribute-modify --enabled true intel-device-plugins-operator intel-device-plugins-qat intel-device-plugins-operator

6. Apply the application.

   ~(keystone_admin)]$ system application-apply intel-device-plugins-operator

7. Monitor the status of the application.

   ~(keystone_admin)]$ watch -n 5 system application-list

   OR

   ~(keystone_admin)]$ watch kubectl get pods -n intel-device-plugins-operator

8. Check the pods.

   $ kubectl get pods -n intel-device-plugins-operator
   
   NAME                                          READY STATUS  RESTARTS AGE
   
   intel-qat-plugin-qatdeviceplugin-sample-g8n45 1/1   Running 0        34s
   inteldeviceplugins-controller-manager-74f4c   2/2   Running 0        64s

9. Verify devices by checking the node's resource allocations. The 4940 device and the 4942 device each have 16 virtual functions. If both devices are present, the following command will display a total of 32 virtual functions:

   $ kubectl describe node <node name> | grep qat.intel.com/sym-dc
   
   Capacity:
   ---
   qat.intel.com/sym-dc: 32
   ---
   Allocatable:
   ---
   qat.intel.com/sym-dc: 32
   ---

Use Intel QAT Device Plugin

This section describes the steps for using device plugin.

1. Deploy a pod using the following sample POD specification file. The pod specification file can be modified for required resource request and limit.

   The qat.intel.com/sym-dc: <number of devices> field is used to configure the requested virtual functions.

   For a -based workload, you may need to add a hugepage request and limit.

   qat-dpdk.yaml

   kind: Pod
   apiVersion: v1
   metadata:
     name: dpdk-test-crypto-perf
   spec:
     containers:
     - name: crypto-perf
       image: intel/crypto-perf:devel
       imagePullPolicy: IfNotPresent
       command: [ "/bin/bash", "-c", "--" ]
       args: [ "while true; do sleep 300000; done;" ]
       volumeMounts:
       - mountPath: /dev/hugepages
         name: hugepage
       - mountPath: /var/run/dpdk
         name: dpdk-runtime
       resources:
         requests:
           cpu: "3"
           memory: "128Mi"
           qat.intel.com/sym-dc: '4'
           hugepages-2Mi: "128Mi"
         limits:
           cpu: "3"
           memory: "128Mi"
           qat.intel.com/sym-dc: '4'
           hugepages-2Mi: "128Mi"
       securityContext:
         readOnlyRootFilesystem: true
         allowPrivilegeEscalation: false
         capabilities:
           add:
             ["IPC_LOCK"]
     restartPolicy: Never
     volumes:
     - name: dpdk-runtime
       emptyDir:
         medium: Memory
     - name: hugepage
       emptyDir:
         medium: HugePages

   Apply the pod specification file to create dpdk-test-crypto-perf pod.

   $ kubectl apply -f qat-dpdk.yaml

2. Verify the pod status and the allocated virtual functions.

   $ kubectl get pods
   
   NAME                  READY STATUS  RESTARTS AGE
   dpdk-test-crypto-perf 1/1   Running 0        27m
   
   $ kubectl describe pod dpdk-test-crypto-perf
   
   Requests:
   ---
   qat.intel.com/sym-dc: 4
   ---
   
   $ kubectl describe node <controller-name>
   
   Allocated resources:
   ---
   qat.intel.com/sym-dc: 4
   ---

For more information, see: Demos and Testing.