diff --git a/doc/source/node_management/figures/ptj1538163621290.png b/doc/source/node_management/figures/ptj1538163621290.png new file mode 100644 index 000000000..8533efa17 Binary files /dev/null and b/doc/source/node_management/figures/ptj1538163621290.png differ diff --git a/doc/source/node_management/openstack/configuring-pci-passthrough-ethernet-interfaces.rst b/doc/source/node_management/openstack/configuring-pci-passthrough-ethernet-interfaces.rst index 85465e33e..f0d924459 100644 --- a/doc/source/node_management/openstack/configuring-pci-passthrough-ethernet-interfaces.rst +++ b/doc/source/node_management/openstack/configuring-pci-passthrough-ethernet-interfaces.rst @@ -13,7 +13,9 @@ considerations. .. rubric:: |context| -You can specify interfaces when you launch an instance. + +Configure a |PCI| Passthrough Ethernet Interface on a host and request it for an +instance at boot/create time. .. rubric:: |prereq| @@ -28,7 +30,7 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to .. rubric:: |proc| -#. Log in as the **admin** user to the |os-prod-hor| interface. +#. Log in as the **admin** user to the |prod-p| |prod-hor-long|. #. Lock the compute node you want to configure. @@ -65,11 +67,43 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to ~(keystone_admin)$ system host-if-modify -c pci-passthrough compute-0 enp0s3 ~(keystone_admin)$ system interface-datanetwork-assign compute-0 -#. Create the **net0** project network + +#. Check if the Ethernet interface supports |SRIOV| + + + #. Check the host port associated with the configured |PCI|-passthrough interface. + + .. code-block:: none + + ~(keystone_admin)$ system host-if-list | grep pci-passthrough + + #. Describe the target port to check the value of sriov_totalvfs on it. + If the value is None, the Ethernet interface does not support |SRIOV|. + Otherwise, it does. + + .. code-block:: none + + ~(keystone_admin)$ system host-port-show | grep sriov_totalvfs + + #. For Ethernet interfaces without |SRIOV| support, there is a known limitation + reported `here `__. + This limitation is overcome with a specific step later on this procedure. + + .. note:: + + It will be required to know if the Ethernet interface supports or not |SRIOV| later in this procedure. + + +#. For Ethernet interfaces that support |SRIOV|, create the **net0** project network + + .. note:: + + If the Ethernet interface **DOES NOT** support |SRIOV|, **skip** this step. + + Log in as the **admin** user to the |os-prod-hor-long|. Select **Admin** \> **Network** \> **Networks**, select the Networks tab, and then click **Create Network**. Fill in the Create Network dialog box as illustrated below. You must ensure that: - - **project1** has access to the project network, either assigning it as the owner, as in the illustration \(using **Project**\), or by enabling the shared flag. @@ -80,7 +114,6 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to .. image:: /node_management/figures/bek1516655307871.png - Click the **Next** button to proceed to the Subnet tab. Click the **Next** button to proceed to the Subnet Details tab. @@ -88,6 +121,8 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to #. Configure the access switch. Refer to the OEM documentation to configure the access switch. + Log in as the **admin** user to the |prod-p| |prod-hor-long|. + Configure the physical port on the access switch used to connect to Ethernet interface **enp0s3** as an access port with default |VLAN| ID of 10. Traffic across the connection is therefore untagged, and effectively @@ -103,10 +138,14 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to #. Unlock the compute node. -#. Create a neutron port with a |VNIC| type, direct-physical. +#. For Ethernet interfaces that support |SRIOV|, + create a neutron port with a |VNIC| type, direct-physical. - The neutron port can also be created from the |CLI|, using the following - command. First, you must set up the environment and determine the correct + .. note:: + + If the Ethernet interface **DOES NOT** support |SRIOV|, **skip** this step. + + First, you must set up the environment and determine the correct network |UUID| to use with the port. .. code-block:: none @@ -119,16 +158,45 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to You have now created a port to be used when launching the server in the next step. -#. Launch the virtual machine, specifying the port uuid created in *Step 7*. +#. For Ethernet interfaces that do not support |SRIOV|, the following Nova + configuration is required. + + .. note:: + + If the Ethernet interface **DOES** support |SRIOV|, **skip** this step. + + - Get the Ethernet interface ``vendor_id`` and ``product_id``: + + .. code-block:: none + + ~(keystone_admin)$ source /etc/platform/openrc + ~(keystone_admin)$ system host-port-show | grep -E '(pvendor |pdevice )' + + - Use the retrieved IDs to create a |PCI| alias with ``"device_type":"type-PCI"``, + as peer :ref:`Configure a PCI Alias in Nova `. + + - Configure a flavor with the extra spec key ``pci_passthrough:alias`` pointing to + the previously created |PCI| alias, as peer + :ref:`Configure a Flavor to Use a Generic PCI Device ` + +#. Launch the virtual machine .. note:: You will need to source to the same project selected in the Create - Network 'net0' in *step 4*. + Network 'net0' step. - .. code-block:: none + - For Ethernet interfaces with |SRIOV| support: specify the port uuid created - ~(keystone_admin)$ openstack server create --flavor --image --nic port-id= + .. code-block:: none + + ~(keystone_admin)$ openstack server create --flavor --image --nic port-id= + + - For Ethernet interfaces without |SRIOV| support: specify the created flavor to use the |PCI| device + + .. code-block:: none + + ~(keystone_admin)$ openstack server create --flavor --image For more information, see the Neutron documentation at: `https://docs.openstack.org/neutron/train/admin/config-sriov.html diff --git a/doc/source/node_management/openstack/index-node-mgmt-os-ccb47338adbc.rst b/doc/source/node_management/openstack/index-node-mgmt-os-ccb47338adbc.rst index 7c0c05941..7d98d239a 100644 --- a/doc/source/node_management/openstack/index-node-mgmt-os-ccb47338adbc.rst +++ b/doc/source/node_management/openstack/index-node-mgmt-os-ccb47338adbc.rst @@ -22,13 +22,13 @@ PCI Device Access for VMs .. toctree:: :maxdepth: 1 - sr-iov-encryption-acceleration - configuring-pci-passthrough-ethernet-interfaces pci-passthrough-ethernet-interface-devices + configuring-pci-passthrough-ethernet-interfaces configure-pci-passthrough-interface-to-nvidia-gpu - configuring-a-flavor-to-use-a-generic-pci-device generic-pci-passthrough - pci-device-access-for-vms pci-sr-iov-ethernet-interface-devices + sr-iov-encryption-acceleration + pci-device-access-for-vms + configuring-a-flavor-to-use-a-generic-pci-device exposing-a-generic-pci-device-for-use-by-vms exposing-a-generic-pci-device-using-the-cli diff --git a/doc/source/node_management/openstack/pci-sr-iov-ethernet-interface-devices.rst b/doc/source/node_management/openstack/pci-sr-iov-ethernet-interface-devices.rst index 8f38b9be3..a13df244e 100644 --- a/doc/source/node_management/openstack/pci-sr-iov-ethernet-interface-devices.rst +++ b/doc/source/node_management/openstack/pci-sr-iov-ethernet-interface-devices.rst @@ -2,9 +2,9 @@ .. vic1596720744539 .. _pci-sr-iov-ethernet-interface-devices: -===================================== -PCI SR-IOV Ethernet Interface Devices -===================================== +=============================================== +Configure PCI SR-IOV Ethernet Interface Devices +=============================================== A |SRIOV| ethernet interface is a physical |PCI| ethernet |NIC| that implements hardware-based virtualization mechanisms to expose multiple virtual network @@ -22,9 +22,6 @@ an independent ethernet interface. When compared with a |PCI| Passthrough ethernet interface, a |SRIOV| ethernet interface: - -.. _pci-sr-iov-ethernet-interface-devices-ul-tyq-ymg-rr: - - Provides benefits similar to those of a |PCI| Passthrough ethernet interface, including lower latency packet processing. @@ -40,22 +37,134 @@ interface: - Provides a similar configuration workflow when used on |prod-os|. -The configuration of a |PCI| |SRIOV| ethernet interface is identical to +The configuration of a |PCI| |SRIOV| ethernet interface is almost identical to :ref:`Configure PCI Passthrough ethernet Interfaces -` except that +` and will be detailed bellow. + +.. rubric:: |context| -.. _pci-sr-iov-ethernet-interface-devices-ul-ikt-nvz-qmb: +Configure a |PCI| |SRIOV| on a host and request it for an +instance at boot/create time. -- you use **pci-sriov** instead of **pci-passthrough** when defining the - network type of an interface +.. rubric:: |prereq| -- the segmentation ID of the project network\(s\) used is more significant +.. note:: + + To use |PCI| passthrough or |SRIOV| devices, you must have Intel VT-x and + Intel VT-d features enabled in the BIOS. + +The exercise assumes that the underlying data network **group0-data0** exists +already, and that |VLAN| ID 10 is a valid segmentation ID assigned to +**project1**. + +.. rubric:: |proc| + +#. Log in as the **admin** user to the |prod-p| |prod-hor-long|. + +#. Lock the compute node you want to configure. + +#. Configure the Ethernet interface to be used as a PCI passthrough interface. + + + #. Select **Admin** \> **Platform** \> **Host Inventory** from the left-hand pane. + + #. Select the **Hosts** tab. + + #. Click the name of the compute host. + + #. Select the **Interfaces** tab. + + #. Click the **Edit Interface** button associated with the interface you + want to configure. + + + The Edit Interface dialog appears. + + .. image:: /node_management/figures/ptj1538163621290.png + + Select **pci-sriov**, from the **Interface Class** drop-down, and + then select the data network to attach the interface. + + You may also need to change the |MTU|. + + The interface can also be configured from the |CLI| as illustrated below: + + .. code-block:: none + + ~(keystone_admin)$ system host-if-modify -c pci-sriov compute-0 enp0s3 + ~(keystone_admin)$ system interface-datanetwork-assign compute-0 + +#. Create the **net0** project network + + Log in as the **admin** user to the |os-prod-hor-long|. + + Select **Admin** \> **Network** \> **Networks**, select the Networks tab, and then click **Create Network**. Fill in the Create Network dialog box as illustrated below. You must ensure that: + + - **project1** has access to the project network, either assigning it as + the owner, as in the illustration \(using **Project**\), or by enabling + the shared flag. + + - The segmentation ID is set to 10. + + + .. image:: /node_management/figures/bek1516655307871.png + + The segmentation ID of the project network\(s\) used is more significant here since this identifies the particular |VF| of the |SRIOV| interface -- when creating the neutron port, you must use ``--vnic-typedirect`` + Click the **Next** button to proceed to the Subnet tab. -- when creating a neutron port backed by an |SRIOV| |VF|, you must use - ``--vnic-type direct`` + Click the **Next** button to proceed to the Subnet Details tab. +#. Configure the access switch. Refer to the OEM documentation to configure + the access switch. + Log in as the **admin** user to the |prod-p| |prod-hor-long|. + + Configure the physical port on the access switch used to connect to + Ethernet interface **enp0s3** as an access port with default |VLAN| ID of 10. + Traffic across the connection is therefore untagged, and effectively + integrated into the targeted project network. + + You can also use a trunk port on the access switch so that it handles + tagged packets as well. However, this opens the possibility for guest + applications to join other project networks using tagged packets with + different |VLAN| IDs, which might compromise the security of the system. + See |os-intro-doc|: :ref:`L2 Access Switches + ` for other details regarding the + configuration of the access switch. + +#. Unlock the compute node. + +#. Create a neutron port with a |VNIC| type, direct-physical. + + First, you must set up the environment and determine the correct + network |UUID| to use with the port. + + .. code-block:: none + + ~(keystone_admin)$ source /etc/platform/openrc + ~(keystone_admin)$ OS_AUTH_URL=http://keystone.openstack.svc.cluster.local/v3 + ~(keystone_admin)$ openstack network list | grep net0 + ~(keystone_admin)$ openstack port create --network --vnic-type direct + + You have now created a port to be used when launching the server in the + next step. + +#. Launch the virtual machine + + .. note:: + + You will need to source to the same project selected in the Create + Network 'net0' step. + + - Specify the port uuid created + + .. code-block:: none + + ~(keystone_admin)$ openstack server create --flavor --image --nic port-id= + + For more information, see the Neutron documentation at: + `https://docs.openstack.org/neutron/train/admin/config-sriov.html + `__.