Document PCI IRQ Affinity Agent operation

This commit includes a documentation about how the agent operates
and how to configure OpenStack to operate properly with it.

Story: 2009299
Task: 43806

Signed-off-by: Heitor Matsui <HeitorVieira.Matsui@windriver.com>
Change-Id: I5969b1a415daf97dbad02b09d8429f2d54efd533

README.rst

@@ -0,0 +1,94 @@
+
+=========
+utilities
+=========
+
+This file serves as documentation for the components and features included
+on the utilities repository.
+
+PCI IRQ Affinity Agent
+----------------------
+
+While in OpenStack it is possible to enable instances to use PCI devices,
+the interrupts generated by these devices may be handled by host CPUs that
+are unrelated to the instance, and this can lead to a performance that is
+lower than it could be if the device interrupts were handled by the instance
+CPUs.
+
+The agent only acts over instances with dedicated vCPUs. For instances using
+shared vCPUs no action will be taken by the agent.
+
+The expected outcome from the agent operation is achieving a higher
+performance by assigning the instances core to handle the interrupts from
+PCI devices used by these instances and avoid interrupts consuming excessive
+cycles from the platform cores.
+
+Agent operation
+~~~~~~~~~~~~~~~
+
+The agent operates by listening to RabbitMQ notifications from Nova. When
+an instance is created or moved to the host, the agent checks for an specific
+flavor spec (detailed below) and if it does then it queries libvirt to map
+the instance vCPUs into pCPUs from the host.
+
+Once the agent has the CPU mapping, it determines the IRQ for each PCI
+device used by the instance, and then it loops over all PCI devices and
+determines which host NUMA node is associated with the device, the pCPUs
+that are associated with the NUMA node and finally set the CPU affinity
+for the IRQs of the PCI device based on the pCPU list.
+
+There is also a periodic audit that runs every minute and loops over the
+existing IRQs, so that if there are new IRQs that weren't mapped before
+the agent maps them, and if there are PCI devices that aren't associated
+to an instance that they were before, their IRQ affinity is reset to the
+default value.
+
+Flavor spec
+~~~~~~~~~~~
+
+The PCI IRQ Affinity Agent uses a specific flavor spec for PCI interrupt
+affining, that is used to determine which vCPUs assigned to the instance
+must handle the interrupts from the PCI devices:
+
+- ``hw:pci_irq_affinity_mask=<vcpus_cpulist>``
+
+Where ``vcpus_cpulist`` can assume a comma-separated list of values that
+can be expressed as:
+
+- ``int``: the vCPU expressed by ``int`` will be assigned to handle the
+  interruptions from the PCI devices
+- ``int1-int2``: the vCPUs between ``int1`` and ``int2`` (inclusive) will
+  be used to handle the interruptions from the PCI devices
+- ``^int``: the vCPU expressed by ``int`` will not be assigned to handle the
+  interruptions from the PCI devices and shall be used to exclude a vCPU
+  that was included in a previous range
+
+**NOTE**: ``int`` must be a value between ``0`` and ``flavor.vcpus - 1``
+
+Example: ``hw_pci_irq_affinity_mask=1-4,^3,6`` means that vCPUs with indexes
+``1,2,4 and 6`` from the vCPU list that Nova allocates to the instance will
+be assigned to handle interruptions from the PCI devices.
+
+Limitations
+~~~~~~~~~~~
+
+- No CPU affining is performed for instances using shared CPUs (i.e.,
+  when using flavor spec ``hw:cpu_policy=shared``)
+- No CPU affining will be performed when invalid ranges are specified on
+  the flavor spec, the agent instead will log error messages indicating
+  the problem
+
+Agent packaging
+~~~~~~~~~~~~~~~
+
+The agent code resides on the ``starlingx/utilities`` repo, along with the
+spec and docker_image files that are used to build a CentOS image with the
+agent wheel installed on it.
+
+The agent is deployed by Armada along with the other OpenStack helm charts;
+refer to `PCI IRQ Affinity Agent helm chart`_ on
+``starlingx/openstack-armada-app`` repository.
+
+.. _`PCI IRQ Affinity Agent helm chart`: https://opendev.org/starlingx
+   /openstack-armada-app/src/branch/master/stx-openstack-helm
+   /stx-openstack-helm/helm-charts/pci-irq-affinity-agent