Add Intel N3000 FPGA Guide
Include sections for Device Flashing & FEC interface Closes-Bug: 1884776 Closes-Bug: 1885053 Change-Id: If34e9e11b6ac1670334eb130a1d42bd5820b7e7a Signed-off-by: MCamp859 <maryx.camp@intel.com>
This commit is contained in:
MCamp859
parent
f94ff78453
commit
31a46c273c
@ -13,6 +13,7 @@ General Configuration
|
||||
cert_config
|
||||
host_config
|
||||
host_interface_network_config
|
||||
intel_n3000_fpga
|
||||
docker_proxy_config
|
||||
system_config
|
||||
time_sync_config
|
||||
|
||||
330
doc/source/configuration/intel_n3000_fpga.rst
Normal file
330
doc/source/configuration/intel_n3000_fpga.rst
Normal file
@ -0,0 +1,330 @@
|
||||
=====================================
|
||||
Host FPGA Configuration - Intel N3000
|
||||
=====================================
|
||||
|
||||
This guide describes how to configure and integrate an Intel N3000
|
||||
:abbr:`FPGA (Field Programmable Gate Array)` :abbr:`PAC (Programmable Acceleration Card)`
|
||||
on StarlingX.
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
--------
|
||||
Overview
|
||||
--------
|
||||
|
||||
The `Intel FPGA PAC N3000 <https://www.intel.com/content/www/us/en/programmable/products/boards_and_kits/dev-kits/altera/intel-fpga-pac-n3000/overview.html>`_ contains two Intel
|
||||
XL710 NICs, memory, and an Intel FPGA. The system discovers and inventories the
|
||||
device as a NIC, with the XL710 ports available in the host port list and host
|
||||
interface list.
|
||||
|
||||
--------------------
|
||||
Update device images
|
||||
--------------------
|
||||
|
||||
The Intel FPGA PAC N3000 as shipped from the factory is expected to have
|
||||
production BMC and factory images. The following procedure describes how to
|
||||
update the user image on a host.
|
||||
|
||||
Device image types:
|
||||
|
||||
* root-key: The root-key image sets the main authentication key on the
|
||||
hardware.
|
||||
|
||||
* functional-key: The functional device image performs the desired work on
|
||||
behalf of the application. If a rootkey device image has been written to the
|
||||
hardware, then the functional image will only be accepted if it has been
|
||||
signed by a :abbr:`CSK (code-signing key)` generated from the root key which has
|
||||
not been revoked.
|
||||
|
||||
* key-revocation: The key-revocation device image will revoke a CSK. If a
|
||||
root-key device image has been written to the hardware, then the
|
||||
key-revocation device image will only be accepted if it has been signed by
|
||||
the root key.
|
||||
|
||||
The following items are specific to the Intel FPGA PAC N3000:
|
||||
|
||||
* The root-key image is called the *root entry hash bitstream* and can only be
|
||||
set once.
|
||||
* The functional device image is known as the *user image*.
|
||||
* The key-revocation device image is known as the *CSK ID cancellation
|
||||
bitstream*.
|
||||
* CSKs are revoked by specifying an integer CSK ID.
|
||||
* 128 CSK ID cancellation slots exist.
|
||||
|
||||
|
||||
For the Intel FPGA PAC N3000, a CSK is revoked by specifying an integer ID, and
|
||||
all CSKs with that ID will be revoked. Writing the root-key device image or a
|
||||
key-revocation device image is essentially permanent. Reverting to factory
|
||||
status requires physical access to the card and specialized equipment.
|
||||
|
||||
#. Upload the device image.
|
||||
|
||||
To upload a root-key device image:
|
||||
|
||||
::
|
||||
|
||||
~(keystone_admin)$ system device-image-upload imagefile root-key pci_vendor
|
||||
pci_device --key-signature key_signature --name imagename --description
|
||||
description --image-version version
|
||||
|
||||
To upload a functional device image:
|
||||
|
||||
::
|
||||
|
||||
~(keystone_admin)$ system device-image-upload imagefile functional pci_vendor
|
||||
pci_device --functional bitstream_id --name imagename --description description
|
||||
-image-version version
|
||||
|
||||
To upload a revocation key device image:
|
||||
|
||||
::
|
||||
|
||||
~(keystone_admin)$ system device-image-upload imagefile key-revocation
|
||||
pci_vendor pci_device --revoke-key-id revoke_key_id --name imagename --
|
||||
description description --image-version version
|
||||
|
||||
where:
|
||||
|
||||
::
|
||||
|
||||
imagefile # The filepath of the binary device image file.
|
||||
pci_vendor # The hexadecimal string identifying the PCI vendor ID of the device this image applies to.
|
||||
pci_device # The hexadecimal string identifying the PCI device ID of the device this image applies to.
|
||||
key_signature # A hexadecimal string identifying the root key device image.
|
||||
revoke_key_id # A decimal key ID for the key revocation device image.
|
||||
bitstream_id # A hexadecimal string of the functional device image.
|
||||
name # The name of the device image (optional).
|
||||
description # The description of the device image (optional).
|
||||
image-version # The version of the device image (optional).
|
||||
|
||||
#. Assign a device label to the device.
|
||||
|
||||
Labels are key-value pairs that are assigned to host PCI devices and are
|
||||
used to specify attributes of the devices. Labels can be used to identify
|
||||
certain properties of the PCI devices where the same device image can be
|
||||
used.
|
||||
|
||||
The command syntax is:
|
||||
|
||||
::
|
||||
|
||||
system host-device-label-assign [--overwrite] hostname_or_id pci_name_or_address
|
||||
name=value [name=value ...]
|
||||
|
||||
Overwrite the label using the ``--overwrite`` option. This option
|
||||
is not allowed while the image update is in progress after running
|
||||
``host-device-image-update``. Once assigned, a device label can be
|
||||
referenced by multiple ``device-image-apply`` commands.
|
||||
|
||||
#. Apply the device image on one or all supported devices.
|
||||
|
||||
.. note::
|
||||
|
||||
A device firmware update in progress alarm is raised once the first
|
||||
device image is applied.
|
||||
|
||||
The ``system device-image-state-list`` will show the status of the device as
|
||||
pending.
|
||||
|
||||
* Apply a device image to all supported devices:
|
||||
|
||||
::
|
||||
|
||||
~(keystone_admin)$ system device-image-apply image_uuid
|
||||
|
||||
* Alternatively, apply a device image to devices with a specified label:
|
||||
|
||||
::
|
||||
|
||||
~(keystone_admin)$ system device-image-apply image_uuid key1=value1
|
||||
|
||||
#. Write pending device images on the host to hardware.
|
||||
|
||||
::
|
||||
|
||||
~(keystone)admin)$ system host-device-image-update hostname
|
||||
|
||||
.. note::
|
||||
|
||||
This operation currently supports one pending device image at a time.
|
||||
|
||||
Any previously-attempted device image writes for this host that are in a
|
||||
failed state will be reset to pending and retried.
|
||||
|
||||
Root and revocation key updates can be expected to take 1-2 minutes.
|
||||
|
||||
Functional image updates can take approximately 40 minutes for the Intel
|
||||
FPGA PAC N3000.
|
||||
|
||||
* Once a device update is complete, ``system device-image-state-list``
|
||||
will show the status as completed for that device/image.
|
||||
|
||||
* Once all pending device updates for the host are complete,
|
||||
``system host-show hostname`` will again display an empty string for
|
||||
``device_image_update``.
|
||||
|
||||
#. Lock and unlock the host.
|
||||
|
||||
#. (Optional) Upload, apply, and update any additional key-revocation device
|
||||
images or functional device images as needed.
|
||||
|
||||
New device images can be uploaded as needed, and already-uploaded images can
|
||||
be applied with new labels. Devices can also have new labels applied to
|
||||
them, and any device images with matching labels will be automatically
|
||||
applied.
|
||||
|
||||
--------------------------
|
||||
Device management commands
|
||||
--------------------------
|
||||
|
||||
This section lists the commands used to control the Intel FPGA PAC N3000.
|
||||
|
||||
::
|
||||
|
||||
Listing uploaded device images
|
||||
system device-image-list
|
||||
|
||||
Listing device labels
|
||||
system host-device-label-list hostname_or_id devicename_or_address
|
||||
|
||||
Removing device labels
|
||||
system host-device-label-remove hostname_or_id key
|
||||
|
||||
Remove a device image
|
||||
To remove a device image from all devices
|
||||
system device-image-remove image_uuid
|
||||
To remove the device image from all devices with a matching label
|
||||
system device-image-remove image_uuid key1=value1
|
||||
|
||||
Initiating a Device Image Update for a Host
|
||||
system host-device-image-update hostname_or_host_ID
|
||||
|
||||
Displaying the status of device images
|
||||
system device-image-state-list
|
||||
|
||||
-------------------------------
|
||||
Enable Forward Error Correction
|
||||
-------------------------------
|
||||
|
||||
The Intel FPGA PAC N3000 supports :abbr:`FEC (forward error correction)`
|
||||
capabilities, which are exposed as a PCI device. The PCI device can be used by a
|
||||
`DPDK <https://www.dpdk.org/>`_ enabled container application to perform accelerated 5G LDPC encoding and
|
||||
decoding operations.
|
||||
|
||||
After the FPGA device is programmed, the list of host devices shows the FEC
|
||||
device with device ID 0xd8f, as shown below.
|
||||
|
||||
::
|
||||
|
||||
system host-device-list <worker-node>
|
||||
|
||||
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
|
||||
| name | address | class id | vendor id | device id | class name | vendor name | device name | numa_node | enabled |
|
||||
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
|
||||
| pci_0000_b7_00_0 | 0000:b7:00.0 | 120000 | 8086 | 0d8f | Processing accelerators | Intel Corporation | Device 0d8f | 1 | True |
|
||||
+------------------+--------------+----------+-----------+-----------+---------------------------+-------------------------+-------------------------------------+-----------+---------+
|
||||
...
|
||||
|
||||
To enable the FEC device for SR-IOV, set the number of virtual functions (VFs)
|
||||
and set the appropriate userspace drivers for the physical function (PF) and VF.
|
||||
|
||||
For example:
|
||||
|
||||
::
|
||||
|
||||
system host-lock <worker>
|
||||
system host-device-modify <worker> <name> --driver <vf driver> --vf-driver <vf driver> -N <number of vfs>
|
||||
system host-unlock <worker>
|
||||
|
||||
The supported PF driver(s) are:
|
||||
- igb_uio
|
||||
|
||||
The supported VF driver(s) are currently:
|
||||
- igb_uio
|
||||
- vfio
|
||||
|
||||
To pass the FEC device to a container, enter the following requests/limits
|
||||
into the pod specification:
|
||||
|
||||
::
|
||||
|
||||
intel.com/intel_fpga_fec: '<number of vfs>'
|
||||
|
||||
For example:
|
||||
|
||||
::
|
||||
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: 5gnr
|
||||
annotations:
|
||||
k8s.v1.cni.cncf.io/networks: '[
|
||||
{ "name": "sriov1" }
|
||||
]'
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
containers:
|
||||
- name: 5gnr
|
||||
image: "5gnr-image"
|
||||
volumeMounts:
|
||||
- mountPath: /mnt/huge-1048576kB
|
||||
name: hugepage
|
||||
stdin: true
|
||||
tty: true
|
||||
resources:
|
||||
requests:
|
||||
memory: 4Gi
|
||||
intel.com/intel_fpga_fec: '1'
|
||||
intel.com/pci_sriov_net_datanetwork_a: '1'
|
||||
limits:
|
||||
hugepages-1Gi: 4Gi
|
||||
memory: 4Gi
|
||||
intel.com/intel_fpga_fec: '1'
|
||||
intel.com/pci_sriov_net_datanetwork_a: '1'
|
||||
volumes:
|
||||
- name: hugepage
|
||||
emptyDir:
|
||||
medium: HugePages
|
||||
|
||||
-------------------------
|
||||
Configure NICs for SR-IOV
|
||||
-------------------------
|
||||
|
||||
You can configure the Intel XL710 NICs for SR-IOV by first identifying the NICs
|
||||
on the Intel FPGA PAC N3000 using the following command:
|
||||
|
||||
::
|
||||
|
||||
system host-port-list <worker>
|
||||
|
||||
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
|
||||
| uuid | name | type | pci address | device | processor | accelerated | device type |
|
||||
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
|
||||
| 6c79c0d0-0463-4551-a19a-24d52a9403c6 | enp177s0f0 | ethernet | 0000:b1:00.0 | 0 | 1 | False | Device [0d58] |
|
||||
+--------------------------------------+------------+----------+--------------+--------+-----------+-------------+------------------------------------------------+
|
||||
...
|
||||
|
||||
Next, set the number of virtual functions (VFs) and set the appropriate
|
||||
userspace drivers for the VF.
|
||||
|
||||
For example:
|
||||
|
||||
::
|
||||
|
||||
system host-lock <worker>
|
||||
system host-if-list -a <worker>
|
||||
system host-if-modify <worker> <interface name or uuid> -c pci-sriov --vf-driver <vf driver> -N <number of vfs>
|
||||
system interface-datanetwork-assign <worker> <interface> <datanetwork>
|
||||
system host-unlock <worker>
|
||||
|
||||
The supported VF driver(s) are currently:
|
||||
- vfio
|
||||
- netdevice
|
||||
|
||||
.. note::
|
||||
|
||||
If ``--vf-driver`` is not specified in the ``system host-if-modify``
|
||||
command, then ``netdevice`` (kernel driver) will be assigned.
|
||||
Loading…
Reference in New Issue
Block a user