PCI-PT configuration when SR-IOV is not available (stx 7.0, stx8, ds7)
There is a known limitation [1] and NICs that do not support SR-IOV require a different procedure [2] when configuring PCI-PT. This change adds a note on checking SR-IOV support for the target NIC, when configuring PCI-Passthrough for it, and adds the necessary steps for the configuration to work properly whit this type of NIC. For completeness, it also duplicates the PCI-PT example for when configuring PCI SRIOV Ethernet Interfaces, with the necessary changes to the procedure. [1] https://bugs.launchpad.net/starlingx/+bug/1836682 [2] https://wiki.openstack.org/wiki/StarlingX/Networking#Useful_Networking_Commands Partial-bug: 1836682 Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com> Change-Id: I7258ab34cb7ce69a2f4b82c682f72d9467d95c70
This commit is contained in:
Thales Elero Cervi
Juanita-Balaraj
parent
0d1d90eb4b
commit
bc0870eade
BIN
doc/source/node_management/figures/ptj1538163621290.png
Normal file
BIN
doc/source/node_management/figures/ptj1538163621290.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 103 KiB |
@ -13,7 +13,9 @@ considerations.
|
||||
|
||||
.. rubric:: |context|
|
||||
|
||||
You can specify interfaces when you launch an instance.
|
||||
|
||||
Configure a |PCI| Passthrough Ethernet Interface on a host and request it for an
|
||||
instance at boot/create time.
|
||||
|
||||
.. rubric:: |prereq|
|
||||
|
||||
@ -28,7 +30,7 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
#. Log in as the **admin** user to the |os-prod-hor| interface.
|
||||
#. Log in as the **admin** user to the |prod-p| |prod-hor-long|.
|
||||
|
||||
#. Lock the compute node you want to configure.
|
||||
|
||||
@ -65,11 +67,43 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
|
||||
~(keystone_admin)$ system host-if-modify -c pci-passthrough compute-0 enp0s3
|
||||
~(keystone_admin)$ system interface-datanetwork-assign compute-0 <enp0s3_interface_uuid> <group0_data0_data_network_uuid>
|
||||
|
||||
#. Create the **net0** project network
|
||||
|
||||
#. Check if the Ethernet interface supports |SRIOV|
|
||||
|
||||
|
||||
#. Check the host port associated with the configured |PCI|-passthrough interface.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ system host-if-list <host-name> | grep pci-passthrough
|
||||
|
||||
#. Describe the target port to check the value of sriov_totalvfs on it.
|
||||
If the value is None, the Ethernet interface does not support |SRIOV|.
|
||||
Otherwise, it does.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ system host-port-show <host-name> <port-name> | grep sriov_totalvfs
|
||||
|
||||
#. For Ethernet interfaces without |SRIOV| support, there is a known limitation
|
||||
reported `here <https://bugs.launchpad.net/starlingx/+bug/1836682>`__.
|
||||
This limitation is overcome with a specific step later on this procedure.
|
||||
|
||||
.. note::
|
||||
|
||||
It will be required to know if the Ethernet interface supports or not |SRIOV| later in this procedure.
|
||||
|
||||
|
||||
#. For Ethernet interfaces that support |SRIOV|, create the **net0** project network
|
||||
|
||||
.. note::
|
||||
|
||||
If the Ethernet interface **DOES NOT** support |SRIOV|, **skip** this step.
|
||||
|
||||
Log in as the **admin** user to the |os-prod-hor-long|.
|
||||
|
||||
Select **Admin** \> **Network** \> **Networks**, select the Networks tab, and then click **Create Network**. Fill in the Create Network dialog box as illustrated below. You must ensure that:
|
||||
|
||||
|
||||
- **project1** has access to the project network, either assigning it as
|
||||
the owner, as in the illustration \(using **Project**\), or by enabling
|
||||
the shared flag.
|
||||
@ -80,7 +114,6 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
|
||||
.. image:: /node_management/figures/bek1516655307871.png
|
||||
|
||||
|
||||
|
||||
Click the **Next** button to proceed to the Subnet tab.
|
||||
|
||||
Click the **Next** button to proceed to the Subnet Details tab.
|
||||
@ -88,6 +121,8 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
|
||||
#. Configure the access switch. Refer to the OEM documentation to configure
|
||||
the access switch.
|
||||
|
||||
Log in as the **admin** user to the |prod-p| |prod-hor-long|.
|
||||
|
||||
Configure the physical port on the access switch used to connect to
|
||||
Ethernet interface **enp0s3** as an access port with default |VLAN| ID of 10.
|
||||
Traffic across the connection is therefore untagged, and effectively
|
||||
@ -103,10 +138,14 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
|
||||
|
||||
#. Unlock the compute node.
|
||||
|
||||
#. Create a neutron port with a |VNIC| type, direct-physical.
|
||||
#. For Ethernet interfaces that support |SRIOV|,
|
||||
create a neutron port with a |VNIC| type, direct-physical.
|
||||
|
||||
The neutron port can also be created from the |CLI|, using the following
|
||||
command. First, you must set up the environment and determine the correct
|
||||
.. note::
|
||||
|
||||
If the Ethernet interface **DOES NOT** support |SRIOV|, **skip** this step.
|
||||
|
||||
First, you must set up the environment and determine the correct
|
||||
network |UUID| to use with the port.
|
||||
|
||||
.. code-block:: none
|
||||
@ -119,16 +158,45 @@ already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
|
||||
You have now created a port to be used when launching the server in the
|
||||
next step.
|
||||
|
||||
#. Launch the virtual machine, specifying the port uuid created in *Step 7*.
|
||||
#. For Ethernet interfaces that do not support |SRIOV|, the following Nova
|
||||
configuration is required.
|
||||
|
||||
.. note::
|
||||
|
||||
If the Ethernet interface **DOES** support |SRIOV|, **skip** this step.
|
||||
|
||||
- Get the Ethernet interface ``vendor_id`` and ``product_id``:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ source /etc/platform/openrc
|
||||
~(keystone_admin)$ system host-port-show <host-name> <port-name> | grep -E '(pvendor |pdevice )'
|
||||
|
||||
- Use the retrieved IDs to create a |PCI| alias with ``"device_type":"type-PCI"``,
|
||||
as peer :ref:`Configure a PCI Alias in Nova <configuring-a-pci-alias-in-nova>`.
|
||||
|
||||
- Configure a flavor with the extra spec key ``pci_passthrough:alias`` pointing to
|
||||
the previously created |PCI| alias, as peer
|
||||
:ref:`Configure a Flavor to Use a Generic PCI Device <configuring-a-flavor-to-use-a-generic-pci-device>`
|
||||
|
||||
#. Launch the virtual machine
|
||||
|
||||
.. note::
|
||||
|
||||
You will need to source to the same project selected in the Create
|
||||
Network 'net0' in *step 4*.
|
||||
Network 'net0' step.
|
||||
|
||||
.. code-block:: none
|
||||
- For Ethernet interfaces with |SRIOV| support: specify the port uuid created
|
||||
|
||||
~(keystone_admin)$ openstack server create --flavor <flavor_name> --image <image_name> --nic port-id=<port_uuid> <name>
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ openstack server create --flavor <flavor_name> --image <image_name> --nic port-id=<port_uuid> <name>
|
||||
|
||||
- For Ethernet interfaces without |SRIOV| support: specify the created flavor to use the |PCI| device
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ openstack server create --flavor <pci_flavor_name> --image <image_name>
|
||||
|
||||
For more information, see the Neutron documentation at:
|
||||
`https://docs.openstack.org/neutron/train/admin/config-sriov.html
|
||||
|
||||
@ -22,13 +22,13 @@ PCI Device Access for VMs
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
sr-iov-encryption-acceleration
|
||||
configuring-pci-passthrough-ethernet-interfaces
|
||||
pci-passthrough-ethernet-interface-devices
|
||||
configuring-pci-passthrough-ethernet-interfaces
|
||||
configure-pci-passthrough-interface-to-nvidia-gpu
|
||||
configuring-a-flavor-to-use-a-generic-pci-device
|
||||
generic-pci-passthrough
|
||||
pci-device-access-for-vms
|
||||
pci-sr-iov-ethernet-interface-devices
|
||||
sr-iov-encryption-acceleration
|
||||
pci-device-access-for-vms
|
||||
configuring-a-flavor-to-use-a-generic-pci-device
|
||||
exposing-a-generic-pci-device-for-use-by-vms
|
||||
exposing-a-generic-pci-device-using-the-cli
|
||||
|
||||
@ -2,9 +2,9 @@
|
||||
.. vic1596720744539
|
||||
.. _pci-sr-iov-ethernet-interface-devices:
|
||||
|
||||
=====================================
|
||||
PCI SR-IOV Ethernet Interface Devices
|
||||
=====================================
|
||||
===============================================
|
||||
Configure PCI SR-IOV Ethernet Interface Devices
|
||||
===============================================
|
||||
|
||||
A |SRIOV| ethernet interface is a physical |PCI| ethernet |NIC| that implements
|
||||
hardware-based virtualization mechanisms to expose multiple virtual network
|
||||
@ -22,9 +22,6 @@ an independent ethernet interface.
|
||||
When compared with a |PCI| Passthrough ethernet interface, a |SRIOV| ethernet
|
||||
interface:
|
||||
|
||||
|
||||
.. _pci-sr-iov-ethernet-interface-devices-ul-tyq-ymg-rr:
|
||||
|
||||
- Provides benefits similar to those of a |PCI| Passthrough ethernet interface,
|
||||
including lower latency packet processing.
|
||||
|
||||
@ -40,22 +37,134 @@ interface:
|
||||
- Provides a similar configuration workflow when used on |prod-os|.
|
||||
|
||||
|
||||
The configuration of a |PCI| |SRIOV| ethernet interface is identical to
|
||||
The configuration of a |PCI| |SRIOV| ethernet interface is almost identical to
|
||||
:ref:`Configure PCI Passthrough ethernet Interfaces
|
||||
<configure-pci-passthrough-ethernet-interfaces>` except that
|
||||
<configure-pci-passthrough-ethernet-interfaces>` and will be detailed bellow.
|
||||
|
||||
.. rubric:: |context|
|
||||
|
||||
|
||||
.. _pci-sr-iov-ethernet-interface-devices-ul-ikt-nvz-qmb:
|
||||
Configure a |PCI| |SRIOV| on a host and request it for an
|
||||
instance at boot/create time.
|
||||
|
||||
- you use **pci-sriov** instead of **pci-passthrough** when defining the
|
||||
network type of an interface
|
||||
.. rubric:: |prereq|
|
||||
|
||||
- the segmentation ID of the project network\(s\) used is more significant
|
||||
.. note::
|
||||
|
||||
To use |PCI| passthrough or |SRIOV| devices, you must have Intel VT-x and
|
||||
Intel VT-d features enabled in the BIOS.
|
||||
|
||||
The exercise assumes that the underlying data network **group0-data0** exists
|
||||
already, and that |VLAN| ID 10 is a valid segmentation ID assigned to
|
||||
**project1**.
|
||||
|
||||
.. rubric:: |proc|
|
||||
|
||||
#. Log in as the **admin** user to the |prod-p| |prod-hor-long|.
|
||||
|
||||
#. Lock the compute node you want to configure.
|
||||
|
||||
#. Configure the Ethernet interface to be used as a PCI passthrough interface.
|
||||
|
||||
|
||||
#. Select **Admin** \> **Platform** \> **Host Inventory** from the left-hand pane.
|
||||
|
||||
#. Select the **Hosts** tab.
|
||||
|
||||
#. Click the name of the compute host.
|
||||
|
||||
#. Select the **Interfaces** tab.
|
||||
|
||||
#. Click the **Edit Interface** button associated with the interface you
|
||||
want to configure.
|
||||
|
||||
|
||||
The Edit Interface dialog appears.
|
||||
|
||||
.. image:: /node_management/figures/ptj1538163621290.png
|
||||
|
||||
Select **pci-sriov**, from the **Interface Class** drop-down, and
|
||||
then select the data network to attach the interface.
|
||||
|
||||
You may also need to change the |MTU|.
|
||||
|
||||
The interface can also be configured from the |CLI| as illustrated below:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ system host-if-modify -c pci-sriov compute-0 enp0s3
|
||||
~(keystone_admin)$ system interface-datanetwork-assign compute-0 <enp0s3_interface_uuid> <group0_data0_data_network_uuid>
|
||||
|
||||
#. Create the **net0** project network
|
||||
|
||||
Log in as the **admin** user to the |os-prod-hor-long|.
|
||||
|
||||
Select **Admin** \> **Network** \> **Networks**, select the Networks tab, and then click **Create Network**. Fill in the Create Network dialog box as illustrated below. You must ensure that:
|
||||
|
||||
- **project1** has access to the project network, either assigning it as
|
||||
the owner, as in the illustration \(using **Project**\), or by enabling
|
||||
the shared flag.
|
||||
|
||||
- The segmentation ID is set to 10.
|
||||
|
||||
|
||||
.. image:: /node_management/figures/bek1516655307871.png
|
||||
|
||||
The segmentation ID of the project network\(s\) used is more significant
|
||||
here since this identifies the particular |VF| of the |SRIOV| interface
|
||||
|
||||
- when creating the neutron port, you must use ``--vnic-typedirect``
|
||||
Click the **Next** button to proceed to the Subnet tab.
|
||||
|
||||
- when creating a neutron port backed by an |SRIOV| |VF|, you must use
|
||||
``--vnic-type direct``
|
||||
Click the **Next** button to proceed to the Subnet Details tab.
|
||||
|
||||
#. Configure the access switch. Refer to the OEM documentation to configure
|
||||
the access switch.
|
||||
|
||||
Log in as the **admin** user to the |prod-p| |prod-hor-long|.
|
||||
|
||||
Configure the physical port on the access switch used to connect to
|
||||
Ethernet interface **enp0s3** as an access port with default |VLAN| ID of 10.
|
||||
Traffic across the connection is therefore untagged, and effectively
|
||||
integrated into the targeted project network.
|
||||
|
||||
You can also use a trunk port on the access switch so that it handles
|
||||
tagged packets as well. However, this opens the possibility for guest
|
||||
applications to join other project networks using tagged packets with
|
||||
different |VLAN| IDs, which might compromise the security of the system.
|
||||
See |os-intro-doc|: :ref:`L2 Access Switches
|
||||
<network-planning-l2-access-switches>` for other details regarding the
|
||||
configuration of the access switch.
|
||||
|
||||
#. Unlock the compute node.
|
||||
|
||||
#. Create a neutron port with a |VNIC| type, direct-physical.
|
||||
|
||||
First, you must set up the environment and determine the correct
|
||||
network |UUID| to use with the port.
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ source /etc/platform/openrc
|
||||
~(keystone_admin)$ OS_AUTH_URL=http://keystone.openstack.svc.cluster.local/v3
|
||||
~(keystone_admin)$ openstack network list | grep net0
|
||||
~(keystone_admin)$ openstack port create --network <uuid_of_net0> --vnic-type direct <port_name>
|
||||
|
||||
You have now created a port to be used when launching the server in the
|
||||
next step.
|
||||
|
||||
#. Launch the virtual machine
|
||||
|
||||
.. note::
|
||||
|
||||
You will need to source to the same project selected in the Create
|
||||
Network 'net0' step.
|
||||
|
||||
- Specify the port uuid created
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
~(keystone_admin)$ openstack server create --flavor <flavor_name> --image <image_name> --nic port-id=<port_uuid> <name>
|
||||
|
||||
For more information, see the Neutron documentation at:
|
||||
`https://docs.openstack.org/neutron/train/admin/config-sriov.html
|
||||
<https://docs.openstack.org/neutron/train/admin/config-sriov.html>`__.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user