Add Intel N3000 FPGA Guide
Include sections for Device Flashing & FEC interface Closes-Bug: 1884776 Closes-Bug: 1885053 Change-Id: If34e9e11b6ac1670334eb130a1d42bd5820b7e7a Signed-off-by: MCamp859 <maryx.camp@intel.com>
This commit is contained in:
MCamp859
parent
f94ff78453
commit
31a46c273c
@ -13,6 +13,7 @@ General Configuration
|
|||||||
cert_config
|
cert_config
|
||||||
host_config
|
host_config
|
||||||
host_interface_network_config
|
host_interface_network_config
|
||||||
|
intel_n3000_fpga
|
||||||
docker_proxy_config
|
docker_proxy_config
|
||||||
system_config
|
system_config
|
||||||
time_sync_config
|
time_sync_config
|
||||||
|
|||||||
330
doc/source/configuration/intel_n3000_fpga.rst
Normal file
330
doc/source/configuration/intel_n3000_fpga.rst
Normal file
@ -0,0 +1,330 @@
|
|||||||
|
=====================================
|
||||||
|
Host FPGA Configuration - Intel N3000
|
||||||
|
=====================================
|
||||||
|
|
||||||
|
This guide describes how to configure and integrate an Intel N3000
|
||||||
|
:abbr:`FPGA (Field Programmable Gate Array)` :abbr:`PAC (Programmable Acceleration Card)`
|
||||||
|
on StarlingX.
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:local:
|
||||||
|
:depth: 1
|
||||||
|
|
||||||
|
--------
|
||||||
|
Overview
|
||||||
|
--------
|
||||||
|
|
||||||
|
The `Intel FPGA PAC N3000 <https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/intel-fpga-pac-n3000/overview.html>`_ contains two Intel
|
||||||
|
XL710 NICs, memory, and an Intel FPGA. The system discovers and inventories the
|
||||||
|
device as a NIC, with the XL710 ports available in the host port list and host
|
||||||
|
interface list.
|
||||||
|
|
||||||
|
--------------------
|
||||||
|
Update device images
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
The Intel FPGA PAC N3000 as shipped from the factory is expected to have
|
||||||
|
production BMC and factory images. The following procedure describes how to
|
||||||
|
update the user image on a host.
|
||||||
|
|
||||||
|
Device image types:
|
||||||
|
|
||||||
|
* root-key: The root-key image sets the main authentication key on the
|
||||||
|
hardware.
|
||||||
|
|
||||||
|
* functional-key: The functional device image performs the desired work on
|
||||||
|
behalf of the application. If a rootkey device image has been written to the
|
||||||
|
hardware, then the functional image will only be accepted if it has been
|
||||||
|
signed by a :abbr:`CSK (code-signing key)` generated from the root key which has
|
||||||
|
not been revoked.
|
||||||
|
|
||||||
|
* key-revocation: The key-revocation device image will revoke a CSK. If a
|
||||||
|
root-key device image has been written to the hardware, then the
|
||||||
|
key-revocation device image will only be accepted if it has been signed by
|
||||||
|
the root key.
|
||||||
|
|
||||||
|
The following items are specific to the Intel FPGA PAC N3000:
|
||||||
|
|
||||||
|
* The root-key image is called the *root entry hash bitstream* and can only be
|
||||||
|
set once.
|
||||||
|
* The functional device image is known as the *user image*.
|
||||||
|
* The key-revocation device image is known as the *CSK ID cancellation
|
||||||
|
bitstream*.
|
||||||
|
* CSKs are revoked by specifying an integer CSK ID.
|
||||||
|
* 128 CSK ID cancellation slots exist.
|
||||||
|
|
||||||
|
|
||||||
|
For the Intel FPGA PAC N3000, a CSK is revoked by specifying an integer ID, and
|
||||||
|
all CSKs with that ID will be revoked. Writing the root-key device image or a
|
||||||
|
key-revocation device image is essentially permanent. Reverting to factory
|
||||||
|
status requires physical access to the card and specialized equipment.
|
||||||
|
|
||||||
|
#. Upload the device image.
|
||||||
|
|
||||||
|
To upload a root-key device image:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
~(keystone_admin)$ system device-image-upload imagefile root-key pci_vendor
|
||||||
|
pci_device --key-signature key_signature --name imagename --description
|
||||||
|
description --image-version version
|
||||||
|
|
||||||
|
To upload a functional device image:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
~(keystone_admin)$ system device-image-upload imagefile functional pci_vendor
|
||||||
|
pci_device --functional bitstream_id --name imagename --description description
|
||||||
|
-image-version version
|
||||||
|
|
||||||
|
To upload a revocation key device image:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
~(keystone_admin)$ system device-image-upload imagefile key-revocation
|
||||||
|
pci_vendor pci_device --revoke-key-id revoke_key_id --name imagename --
|
||||||
|
description description --image-version version
|
||||||
|
|
||||||
|
where:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
imagefile # The filepath of the binary device image file.
|
||||||
|
pci_vendor # The hexadecimal string identifying the PCI vendor ID of the device this image applies to.
|
||||||
|
pci_device # The hexadecimal string identifying the PCI device ID of the device this image applies to.
|
||||||
|
key_signature # A hexadecimal string identifying the root key device image.
|
||||||
|
revoke_key_id # A decimal key ID for the key revocation device image.
|
||||||
|
bitstream_id # A hexadecimal string of the functional device image.
|
||||||
|
name # The name of the device image (optional).
|
||||||
|
description # The description of the device image (optional).
|
||||||
|
image-version # The version of the device image (optional).
|
||||||
|
|
||||||
|
#. Assign a device label to the device.
|
||||||
|
|
||||||
|
Labels are key-value pairs that are assigned to host PCI devices and are
|
||||||
|
used to specify attributes of the devices. Labels can be used to identify
|
||||||
|
certain properties of the PCI devices where the same device image can be
|
||||||
|
used.
|
||||||
|
|
||||||
|
The command syntax is:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
system host-device-label-assign [--overwrite] hostname_or_id pci_name_or_address
|
||||||
|
name=value [name=value ...]
|
||||||
|
|
||||||
|
Overwrite the label using the ``--overwrite`` option. This option
|
||||||
|
is not allowed while the image update is in progress after running
|
||||||
|
``host-device-image-update``. Once assigned, a device label can be
|
||||||
|
referenced by multiple ``device-image-apply`` commands.
|
||||||
|
|
||||||
|
#. Apply the device image on one or all supported devices.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
A device firmware update in progress alarm is raised once the first
|
||||||
|
device image is applied.
|
||||||
|
|
||||||
|
The ``system device-image-state-list`` will show the status of the device as
|
||||||
|
pending.
|
||||||
|
|
||||||
|
* Apply a device image to all supported devices:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
~(keystone_admin)$ system device-image-apply image_uuid
|
||||||
|
|
||||||
|
* Alternatively, apply a device image to devices with a specified label:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
~(keystone_admin)$ system device-image-apply image_uuid key1=value1
|
||||||
|
|
||||||
|
#. Write pending device images on the host to hardware.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
~(keystone)admin)$ system host-device-image-update hostname
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
This operation currently supports one pending device image at a time.
|
||||||
|
|
||||||
|
Any previously-attempted device image writes for this host that are in a
|
||||||
|
failed state will be reset to pending and retried.
|
||||||
|
|
||||||
|
Root and revocation key updates can be expected to take 1-2 minutes.
|
||||||
|
|
||||||
|
Functional image updates can take approximately 40 minutes for the Intel
|
||||||
|
FPGA PAC N3000.
|
||||||
|
|
||||||
|
* Once a device update is complete, ``system device-image-state-list``
|
||||||
|
will show the status as completed for that device/image.
|
||||||
|
|
||||||
|
* Once all pending device updates for the host are complete,
|
||||||
|
``system host-show hostname`` will again display an empty string for
|
||||||
|
``device_image_update``.
|
||||||
|
|
||||||
|
#. Lock and unlock the host.
|
||||||
|
|
||||||
|
#. (Optional) Upload, apply, and update any additional key-revocation device
|
||||||
|
images or functional device images as needed.
|
||||||
|
|
||||||
|
New device images can be uploaded as needed, and already-uploaded images can
|
||||||
|
be applied with new labels. Devices can also have new labels applied to
|
||||||
|
them, and any device images with matching labels will be automatically
|
||||||
|
applied.
|
||||||
|
|
||||||
|
--------------------------
|
||||||
|
Device management commands
|
||||||
|
--------------------------
|
||||||
|
|
||||||
|
This section lists the commands used to control the Intel FPGA PAC N3000.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
Listing uploaded device images
|
||||||
|
system device-image-list
|
||||||
|
|
||||||
|
Listing device labels
|
||||||
|
system host-device-label-list hostname_or_id devicename_or_address
|
||||||
|
|
||||||
|
Removing device labels
|
||||||
|
system host-device-label-remove hostname_or_id key
|
||||||
|
|
||||||
|
Remove a device image
|
||||||
|
To remove a device image from all devices
|
||||||
|
system device-image-remove image_uuid
|
||||||
|
To remove the device image from all devices with a matching label
|
||||||
|
system device-image-remove image_uuid key1=value1
|
||||||
|
|
||||||
|
Initiating a Device Image Update for a Host
|
||||||
|
system host-device-image-update hostname_or_host_ID
|
||||||
|
|
||||||
|
Displaying the status of device images
|
||||||
|
system device-image-state-list
|
||||||
|
|
||||||
|
-------------------------------
|
||||||
|
Enable Forward Error Correction
|
||||||
|
-------------------------------
|
||||||
|
|
||||||
|
The Intel FPGA PAC N3000 supports :abbr:`FEC (forward error correction)`
|
||||||
|
capabilities, which are exposed as a PCI device. The PCI device can be used by a
|
||||||
|
`DPDK <https://www.dpdk.org/>`_ enabled container application to perform accelerated 5G LDPC encoding and
|
||||||
|
decoding operations.
|
||||||
|
|
||||||
|
After the FPGA device is programmed, the list of host devices shows the FEC
|
||||||
|
device with device ID 0xd8f, as shown below.
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
system host-device-list <worker-node>
|
||||||
|
|
||||||
|
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
|
||||||
|
| name | address | class id | vendor id | device id | class name | vendor name | device name | numa_node | enabled |
|
||||||
|
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
|
||||||
|
| pci_0000_b7_00_0 | 0000:b7:00.0 | 120000 | 8086 | 0d8f | Processing accelerators | Intel Corporation | Device 0d8f | 1 | True |
|
||||||
|
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
|
||||||
|
...
|
||||||
|
|
||||||
|
To enable the FEC device for SR-IOV, set the number of virtual functions (VFs)
|
||||||
|
and set the appropriate userspace drivers for the physical function (PF) and VF.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
system host-lock <worker>
|
||||||
|
system host-device-modify <worker> <name> --driver <vf driver> --vf-driver <vf driver> -N <number of vfs>
|
||||||
|
system host-unlock <worker>
|
||||||
|
|
||||||
|
The supported PF driver(s) are:
|
||||||
|
- igb_uio
|
||||||
|
|
||||||
|
The supported VF driver(s) are currently:
|
||||||
|
- igb_uio
|
||||||
|
- vfio
|
||||||
|
|
||||||
|
To pass the FEC device to a container, enter the following requests/limits
|
||||||
|
into the pod specification:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
intel.com/intel_fpga_fec: '<number of vfs>'
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Pod
|
||||||
|
metadata:
|
||||||
|
name: 5gnr
|
||||||
|
annotations:
|
||||||
|
k8s.v1.cni.cncf.io/networks: '[
|
||||||
|
{ "name": "sriov1" }
|
||||||
|
]'
|
||||||
|
spec:
|
||||||
|
restartPolicy: Never
|
||||||
|
containers:
|
||||||
|
- name: 5gnr
|
||||||
|
image: "5gnr-image"
|
||||||
|
volumeMounts:
|
||||||
|
- mountPath: /mnt/huge-1048576kB
|
||||||
|
name: hugepage
|
||||||
|
stdin: true
|
||||||
|
tty: true
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
memory: 4Gi
|
||||||
|
intel.com/intel_fpga_fec: '1'
|
||||||
|
intel.com/pci_sriov_net_datanetwork_a: '1'
|
||||||
|
limits:
|
||||||
|
hugepages-1Gi: 4Gi
|
||||||
|
memory: 4Gi
|
||||||
|
intel.com/intel_fpga_fec: '1'
|
||||||
|
intel.com/pci_sriov_net_datanetwork_a: '1'
|
||||||
|
volumes:
|
||||||
|
- name: hugepage
|
||||||
|
emptyDir:
|
||||||
|
medium: HugePages
|
||||||
|
|
||||||
|
-------------------------
|
||||||
|
Configure NICs for SR-IOV
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
You can configure the Intel XL710 NICs for SR-IOV by first identifying the NICs
|
||||||
|
on the Intel FPGA PAC N3000 using the following command:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
system host-port-list <worker>
|
||||||
|
|
||||||
|
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
|
||||||
|
| uuid | name | type | pci address | device | processor | accelerated | device type |
|
||||||
|
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
|
||||||
|
| 6c79c0d0-0463-4551-a19a-24d52a9403c6 | enp177s0f0 | ethernet | 0000:b1:00.0 | 0 | 1 | False | Device [0d58] |
|
||||||
|
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
|
||||||
|
...
|
||||||
|
|
||||||
|
Next, set the number of virtual functions (VFs) and set the appropriate
|
||||||
|
userspace drivers for the VF.
|
||||||
|
|
||||||
|
For example:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
system host-lock <worker>
|
||||||
|
system host-if-list -a <worker>
|
||||||
|
system host-if-modify <worker> <interface name or uuid> -c pci-sriov --vf-driver <vf driver> -N <number of vfs>
|
||||||
|
system interface-datanetwork-assign <worker> <interface> <datanetwork>
|
||||||
|
system host-unlock <worker>
|
||||||
|
|
||||||
|
The supported VF driver(s) are currently:
|
||||||
|
- vfio
|
||||||
|
- netdevice
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
If ``--vf-driver`` is not specified in the ``system host-if-modify``
|
||||||
|
command, then ``netdevice`` (kernel driver) will be assigned.
|
||||||
Loading…
x
Reference in New Issue
Block a user