Add Broadcom NetXtreme-E drivers
This commit introduces Broadcom's bnxt_en and bnxt_re out-of-tree
drivers, version 220.0.13.0, to StarlingX, and disables the in-tree
bnxt_en driver. (For clarity, the main driver package file name is
"bcm_220.0.83.0.tar.gz".)
There is a need to build Broadcom's driver against the version of the
Infiniband/RDMA code base bundled with Mellanox OFED (OpenFabrics
Enterprise Distribution), because the in-tree kernel Infiniband/RDMA
modules are disabled, and Mellanox OFED's out-of-tree Infiniband/RDMA
modules replace the in-tree versions.
A custom spec file was used, despite the fact that the Broadcom device
driver package includes a source RPM file. This was necessary due to two
reasons:
- The source RPM file has whitespace between its field labels (such as
Name, Version, Release, and so on) and the values of these fields.
(Example: "Name : bnxt_en") This unfortunately breaks "build-pkgs
--edit", which cannot handle the whitespace during the extraction of
the source RPM.
- More importantly, the spec file included by Broadcom has the following
lines in its %prep section, which does not use the %setup macro:
%prep
#%setup
mkdir -p source
tar -xzvf %{SOURCE0} -C source/
mv -f source/netxtreme-%name-%{version}-220.0.13.0/* source/
rm -rf source/netxtreme-%name-%{version}-220.0.13.0/
cp ./source/%{bnxt_re_driver}/README.TXT
./source/%{bnxt_re_driver}/%{bnxt_re_driver}_README.TXT
cp ./source/%{bnxt_en_driver}/README.TXT
./source/%{bnxt_en_driver}/%{bnxt_en_driver}_README.TXT
mkdir -p obj
This is problematic, because the "build-pkgs --edit" procedure
extracts the source RPM multiple times into the same directory to
ensure that the extraction is stable. However, the %prep section
quoted above does not remove the 'source' directory before attempting
to populate it. (Unfortunately, this issue is not patchable using a
meta-patch, because the extraction occurs even before the application
of meta-patches.) This in turn triggers the following errors:
+ mv -f source/netxtreme-bnxt_en-1.10.2-220.0.13.0/Makefile \
source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_en \
source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_en-1.10.2-220.0.13.0 \
source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_re \
source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_re-220.0.12.0 \
source/
mv: cannot move 'source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_en-1.10.2-220.0.13.0' \
to 'source/bnxt_en-1.10.2-220.0.13.0': Directory not empty
mv: cannot move 'source/netxtreme-bnxt_en-1.10.2-220.0.13.0/bnxt_re-220.0.12.0' \
to 'source/bnxt_re-220.0.12.0': Directory not empty
error: Bad exit status from .../std/srpm_work/bnxt_en/var/tmp/rpm-tmp.cDTpOP (%prep)
Bad exit status from .../std/srpm_work/bnxt_en/var/tmp/rpm-tmp.cDTpOP (%prep)
The changes made to the spec file are as follows:
- Reliance on Red Hat's kernel module packaging RPM spec macros (such as
%kernel_module_package) is removed, because the aforementioned macros
make use of the kmodtool helper program, which automatically generates
an RPM post-installation and post-uninstallation sections making use
of the weak-modules utility, which StarlingX does not use/support. It
is not possible to disable this behaviour of the kmodtool script
("/usr/lib/rpm/redhat/kmodtool") with a command line option or an
environment variable.
- The spec file is reorganized to build the kernel module against only a
single kernel "flavour", as StarlingX does not use the kernel flavour
concept.
- The %build process is modified so that instead of using the top-level
makefile (which does not take certain variables as arguments), the
bnxt_en- and bnxt_re-specific makefiles are called with make
separately. This allows passing KDIR, KVER and OFED_VERSION make
variables as command line arguments to make.
- The use of the devtoolset-8 software collection was added to align to
the kernel's build toolchain.
- The spec file is modified to detect the OFED version and pass the
version to the bnxt_re makefile as the OFED_VERSION environment
variable. This is necessary to automatically adapt to the version of
OFED installed in the build environment.
- The %post and %postun scriptlets are modified to make them similar to
the ones used by i40e, which passes the target kernel's version to
depmod. The original %post only had 'depmod -a', which can be
problematic when installing the package for a kernel that is not
currently running.
- Kernel module signing was added to %install. This shell snippet was
adapted from i40e.
- Finally, retpolines were enabled by default, given that StarlingX
enables retpolines in its kernel packages as well.
In terms of source code modifications, the following changes were made
to bnxt_en and bnxt_re:
- The bnxt_en and bnxt_re kernel modules are patched to make use of the
irq_update_affinity_hint function instead of the irq_set_affinity_hint
function, so that the irqaffinity= kernel command line option takes
effect for the affinities of the IRQs managed by the two drivers.
However, it should be noted the changes in bnxt_en are not truly
necessary, because pre-existing conditional compilation directives
leave out the modified code when targeting kernels >= v4.15 or kernels
that have the "IRQ matrix allocator" feature. The changes are kept for
completeness, nevertheless.
For further details, please see the patch description.
- A compilation failure caused by the definition of a placeholder macro
aimed to stand in for the bpf_warn_invalid_xdp_action function in the
kernel tree is resolved by removing the macro definition.
Testing:
- An ISO image was successfully built with a monolithic incremental
build procedure.
- Using a virtual machine in All-in-One simplex configuration, the
bnxt_en and bnxt_re kernel modules were confirmed to be successfully
loaded manually using, for example, "modprobe -v -a bnxt_en bnxt_re".
Runtime tests for these drivers were unfortunately very limited,
because the necessary hardware has not yet been acquired. We intend to
resolve integration issues when hardware is acquired.
Story: 2009915
Task: 44762
Depends-On: https://review.opendev.org/c/starlingx/kernel/+/833463
Depends-On: https://review.opendev.org/c/starlingx/tools/+/833462
Change-Id: Ieb4147335410160a781382b0b6b01e2015468a3b
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
This commit is contained in:
M. Vefa Bicakci
parent
ec7d0d4768
commit
0ae9f6bd53
@ -4,3 +4,4 @@ mstflint-4.16.0-1.55103.src.rpm
|
||||
rdma-core-55mlnx37-1.55103.src.rpm
|
||||
opae-intel-fpga-driver-2.0.1-10.src.rpm
|
||||
ice_comms-1.3.31.0.zip
|
||||
netxtreme-bnxt_en-1.10.2-220.0.13.0.tar.gz
|
||||
|
||||
@ -60,3 +60,7 @@ rdma-core
|
||||
|
||||
# mellanox firmware tool
|
||||
mstflint
|
||||
|
||||
# Broadcom device driver
|
||||
kmod-bnxt_en
|
||||
kmod-bnxt_en-rt
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
kernel-std
|
||||
kernel-modules/bnxt_en
|
||||
kernel-modules/intel-iavf
|
||||
kernel-modules/intel-i40e
|
||||
kernel-modules/intel-ice
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
kernel-rt
|
||||
kernel-modules/bnxt_en
|
||||
kernel-modules/mlnx-ofa_kernel
|
||||
kernel-modules/intel-iavf
|
||||
kernel-modules/intel-i40e
|
||||
|
||||
@ -14,3 +14,4 @@ opae-intel-fpga-driver-2.0.1-8.tar.gz#opae-intel-fpga-driver-2.0.1-8#https://git
|
||||
!dpdk-kmods-2a9f0f72a2d926382634cf8f1de10e1acf57542b.tar.gz#dpdk-kmods#git://dpdk.org/dpdk-kmods/#git#2a9f0f72a2d926382634cf8f1de10e1acf57542b##
|
||||
!OPAE_1.3.7-5_el7.zip#OPAE#https://github.com/OPAE/opae-sdk/releases/download/1.3.7-5/OPAE_1.3.7-5_el7.zip#http_script##unused
|
||||
!800-Series-DDP-Comms-Package-1.3.31.0.zip#unused-field#https://downloadmirror.intel.com/713853/800%20Series%20DDP%20Comms%20Package%201.3.31.0.zip#http_script##unused
|
||||
!bcm_220.0.83.0.tar.gz#bcm_220.0.83.0#https://docs.broadcom.com/docs-and-downloads/ethernet-network-adapters/NXE/BRCM_220.0.83.0/bcm_220.0.83.0.tar.gz#http_script#85bdfc30b4bd3e184e3b57a48055c11085e3b97593f7b4a8347fa50a9d571336#unused
|
||||
|
||||
135
kernel-modules/bnxt_en/centos/bnxt_en.spec
Normal file
135
kernel-modules/bnxt_en/centos/bnxt_en.spec
Normal file
@ -0,0 +1,135 @@
|
||||
%if "%{?_tis_build_type}" == "rt"
|
||||
%define bt_ext -rt
|
||||
%else
|
||||
%undefine bt_ext
|
||||
%endif
|
||||
|
||||
%define kmod_name bnxt_en
|
||||
%define upstream_release 220.0.13.0
|
||||
|
||||
## Condition to check if retpolines are needed.
|
||||
## Default: retpolines are enabled.
|
||||
%bcond_without retpoline
|
||||
|
||||
%define kernel_module_package_buildreqs kernel%{?bt_ext}-devel
|
||||
|
||||
# Disable the building of debug package(s).
|
||||
%define debug_package %{nil}
|
||||
|
||||
Name: kmod-%{kmod_name}%{?bt_ext}
|
||||
Version: 1.10.2
|
||||
Release: %{upstream_release}%{?_tis_dist}.%{tis_patch_ver}
|
||||
Group: System Environment/Base
|
||||
Packager: StarlingX Community
|
||||
Vendor: Broadcom Inc
|
||||
URL: https://www.broadcom.com/support/download-search?pg=&pf=&pn=&pa=&po=&dk=bnxt_en&pl=
|
||||
License: GPLv2
|
||||
Summary: Broadcom NetXtreme Gigabit Ethernet Driver
|
||||
|
||||
Provides: %{kmod_name}-kmod = %{?epoch:%{epoch}:}%{version}-%{release}
|
||||
BuildRequires: devtoolset-8-build
|
||||
BuildRequires: devtoolset-8-binutils
|
||||
BuildRequires: devtoolset-8-gcc
|
||||
BuildRequires: devtoolset-8-make
|
||||
BuildRequires: kernel%{?bt_ext}-devel findutils
|
||||
BuildRequires: mlnx-ofa_kernel%{?bt_ext}-devel mlnx-ofa_kernel%{?bt_ext}-source
|
||||
Requires: mlnx-ofa_kernel%{?bt_ext}-modules
|
||||
Requires(post): /usr/sbin/depmod
|
||||
Requires(postun): /usr/sbin/depmod
|
||||
|
||||
Source0: netxtreme-%{kmod_name}-%{version}-%{upstream_release}.tar.gz
|
||||
|
||||
Patch0001: 0001-bnxt_re-Makefile-Adapt-to-mlnx-ofa_kernel-for-Starli.patch
|
||||
Patch0002: 0002-bnxt_en-bnxt_compat.h-Fix-up-a-build-failure.patch
|
||||
Patch0003: 0003-bnxt_en-bnxt_re-Use-irq_update_affinity_hint.patch
|
||||
|
||||
#define some build variables required
|
||||
%define bnxt_en_driver bnxt_en
|
||||
%define bnxt_re_driver bnxt_re
|
||||
|
||||
%define kversion %(rpm -q kernel%{?bt_ext}-devel | sort --version-sort | tail -1 | sed 's/kernel%{?bt_ext}-devel-//')
|
||||
|
||||
%description
|
||||
This package provides the %{bnxt_en_driver} and %{bnxt_re_driver} kernel
|
||||
modules built for the Linux kernel for the %{_target_cpu} family of processors.
|
||||
|
||||
%prep
|
||||
%autosetup -p 1 -n netxtreme-%{kmod_name}-%{version}-%{upstream_release}
|
||||
cp %{bnxt_re_driver}/README.TXT %{bnxt_re_driver}/%{bnxt_re_driver}_README.TXT
|
||||
cp %{bnxt_en_driver}/README.TXT %{bnxt_en_driver}/%{bnxt_en_driver}_README.TXT
|
||||
|
||||
%build
|
||||
%if 0%{?rhel} == 7
|
||||
source scl_source enable devtoolset-8 || :
|
||||
%endif
|
||||
|
||||
export EXTRA_CFLAGS+='-DVERSION=\"%{version}\"'
|
||||
|
||||
%if %{with retpoline}
|
||||
find . -name *.c -print0 | xargs -0 sed -i '/MODULE_LICENSE(/a MODULE_INFO(retpoline, "Y");'
|
||||
%endif
|
||||
|
||||
# Regenerate MANIFEST HASH
|
||||
if [ -e bnxt_en/MANIFEST ]; then
|
||||
pushd bnxt_en >/dev/null
|
||||
sha512sum $(cat MANIFEST | cut -c 131-) > MANIFEST
|
||||
popd >/dev/null
|
||||
fi
|
||||
|
||||
%{__make} %{_smp_mflags} -C bnxt_en \
|
||||
KVER=%{kversion} \
|
||||
KDIR=%{_usrsrc}/kernels/%{kversion}
|
||||
|
||||
OFED_VERSION="$(ls -1vd %{_usrsrc}/ofa_kernel-* | xargs -r -n1 basename | \
|
||||
sed -e 's@^ofa_kernel-@@' | sort --version-sort | tail -n1)"
|
||||
if test -z "${OFED_VERSION}"; then
|
||||
echo "Error: Could not detect the OFED version"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
%{__make} %{_smp_mflags} -C bnxt_re \
|
||||
KVER=%{kversion} \
|
||||
KDIR=%{_usrsrc}/kernels/%{kversion} \
|
||||
OFED_VERSION="${OFED_VERSION}"
|
||||
|
||||
%install
|
||||
export KVER=%{kversion}
|
||||
|
||||
dest_dir_bnxt_en=${RPM_BUILD_ROOT}/lib/modules/${KVER}/extra/%{kmod_name}/
|
||||
src_dir_bnxt_en=$(find . -name %{bnxt_en_driver}.ko)
|
||||
dest_dir_bnxt_re=${RPM_BUILD_ROOT}/lib/modules/${KVER}/extra/%{kmod_name}/drivers/infiniband/hw/%{bnxt_re_driver}/
|
||||
src_dir_bnxt_re=$(find . -name %{bnxt_re_driver}.ko)
|
||||
mkdir -p ${dest_dir_bnxt_en}
|
||||
mkdir -p ${dest_dir_bnxt_re}
|
||||
install -m 744 ${src_dir_bnxt_en} ${dest_dir_bnxt_en}
|
||||
install -m 744 ${src_dir_bnxt_re} ${dest_dir_bnxt_re}
|
||||
|
||||
### Adapted from i40e
|
||||
# Strip the modules(s).
|
||||
find %{buildroot} -type f -name \*.ko -exec %{__strip} --strip-debug \{\} \;
|
||||
|
||||
# Always Sign the modules(s).
|
||||
# If the module signing keys are not defined, define them here.
|
||||
%{!?privkey: %define privkey /usr/src/kernels/%{kversion}/signing_key.pem}
|
||||
%{!?pubkey: %define pubkey /usr/src/kernels/%{kversion}/signing_key.x509}
|
||||
for module in $(find %{buildroot} -type f -name \*.ko);
|
||||
do /usr/src/kernels/%{kversion}/scripts/sign-file \
|
||||
sha256 %{privkey} %{pubkey} $module;
|
||||
done
|
||||
|
||||
%files
|
||||
%defattr(644,root,root,755)
|
||||
/lib/modules/%{kversion}/extra/%{kmod_name}/
|
||||
|
||||
%post
|
||||
if [ -e "/boot/System.map-%{kversion}" ]; then
|
||||
/usr/sbin/depmod -aeF "/boot/System.map-%{kversion}" "%{kversion}" > /dev/null || :
|
||||
fi
|
||||
|
||||
%postun
|
||||
/usr/sbin/depmod -a >/dev/null 2>&1 || true
|
||||
|
||||
%clean
|
||||
rm -rf %{buildroot}
|
||||
|
||||
%changelog
|
||||
11
kernel-modules/bnxt_en/centos/build_srpm.data
Normal file
11
kernel-modules/bnxt_en/centos/build_srpm.data
Normal file
@ -0,0 +1,11 @@
|
||||
COPY_LIST=" \
|
||||
$PATCHES_BASE/* \
|
||||
$STX_BASE/downloads/netxtreme-bnxt_en-1.10.2-220.0.13.0.tar.gz \
|
||||
"
|
||||
|
||||
OPT_DEP_LIST_FOR_BUILD_TYPE[std]=$GIT_BASE/kernel-std
|
||||
OPT_DEP_LIST_FOR_BUILD_TYPE[rt]=$GIT_BASE/kernel-rt
|
||||
BASE_SRCREV_FOR_PATH[$GIT_BASE/kernel-std]=1378583af6e040bc974f67850cd44e0c62d42f81
|
||||
BASE_SRCREV_FOR_PATH[$GIT_BASE/kernel-rt]=1378583af6e040bc974f67850cd44e0c62d42f81
|
||||
PKG_BASE_SRCREV=1378583af6e040bc974f67850cd44e0c62d42f81
|
||||
TIS_PATCH_VER=PKG_GITREVCOUNT+OTHER_GITREVCOUNT
|
||||
@ -0,0 +1,117 @@
|
||||
From b4de4f0d87331a3df820ca78a7196b41a5ee74d5 Mon Sep 17 00:00:00 2001
|
||||
From: "M. Vefa Bicakci" <vefa.bicakci@windriver.com>
|
||||
Date: Fri, 4 Feb 2022 18:27:53 -0500
|
||||
Subject: [PATCH] bnxt_re/Makefile: Adapt to mlnx-ofa_kernel for StarlingX
|
||||
|
||||
This patch adapts the bnxt_re kernel module's Makefile to
|
||||
mlnx-ofa_kernel for StarlingX.
|
||||
|
||||
Here is a summary of the changes:
|
||||
|
||||
- $(shell uname -r) is replaced with $(KVER) to ensure that the target
|
||||
kernel version is selected instead of the build host's kernel version.
|
||||
|
||||
- To target Mellanox's OFED distribution instead of compat-rdma, the
|
||||
references to "/usr/src/compat-rdma" are replaced with
|
||||
"/usr/src/ofa_kernel".
|
||||
|
||||
- Mellanox's OFED distribution has version numbers in the form 5.x, so
|
||||
the Makefile version match patterns are adjusted.
|
||||
|
||||
- "AUTOCONF_H += -include $(LINUXSRC)/include/linux/kconfig.h" line is
|
||||
added avoid the following kernel module build errors indicating that
|
||||
IS_ENABLED is unintendedly undefined:
|
||||
|
||||
include/asm-generic/bitops/instrumented-non-atomic.h: In function \
|
||||
'__instrument_read_write_bitop':
|
||||
include/asm-generic/bitops/instrumented-non-atomic.h:63:6: error: \
|
||||
implicit declaration of function 'IS_ENABLED' \
|
||||
[-Werror=implicit-function-declaration]
|
||||
if (IS_ENABLED(CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC)) {
|
||||
^~~~~~~~~~
|
||||
|
||||
- The line that reads
|
||||
"KBUILD_EXTRA_SYMBOLS := $(BNXT_PEER_MEM_INC)/Module.symvers"
|
||||
is guarded to avoid the following build warning, which is encountered
|
||||
as BNXT_PEER_MEM_INC is not defined by default:
|
||||
WARNING: Symbol version dump "/Module.symvers" is missing.
|
||||
|
||||
- OFA_BUILD_PATH and OFA_KERNEL_PATH are adjusted to refer to the kernel
|
||||
source code belonging to the target kernel version in
|
||||
"/usr/src/kernels/", because StarlingX's build environment does not
|
||||
populate /lib/modules/<kver>/ with symbolic links for kernel source
|
||||
and build trees.
|
||||
|
||||
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
|
||||
---
|
||||
bnxt_re-220.0.12.0/Makefile | 30 ++++++++++++++++++------------
|
||||
1 file changed, 18 insertions(+), 12 deletions(-)
|
||||
|
||||
diff --git a/bnxt_re-220.0.12.0/Makefile b/bnxt_re-220.0.12.0/Makefile
|
||||
index 532fb8025465..d13a61379acb 100644
|
||||
--- a/bnxt_re-220.0.12.0/Makefile
|
||||
+++ b/bnxt_re-220.0.12.0/Makefile
|
||||
@@ -32,7 +32,7 @@ ifeq ($(shell ls /lib/modules/$(KVER)/build > /dev/null 2>&1 && echo build),)
|
||||
# SuSE source RPMs
|
||||
_KVER=$(shell echo $(KVER) | cut -d "-" -f1,2)
|
||||
_KFLA=$(shell echo $(KVER) | cut -d "-" -f3)
|
||||
- _ARCH=$(shell file -b /lib/modules/$(shell uname -r)/build | cut -d "/" -f5)
|
||||
+ _ARCH=$(shell file -b /lib/modules/$(KVER)/build | cut -d "/" -f5)
|
||||
ifeq ($(_ARCH),)
|
||||
_ARCH=$(__ARCH)
|
||||
endif
|
||||
@@ -90,34 +90,40 @@ ifeq ($(OFED_VERSION), )
|
||||
endif
|
||||
|
||||
#find OFED version and compat-includes
|
||||
-ofed_major=$(filter OFED-3.% OFED-4.%, $(OFED_VERSION))
|
||||
+ofed_major=$(filter OFED-3.% OFED-4.% 5.%, $(OFED_VERSION))
|
||||
ifneq ($(ofed_major), )
|
||||
-exists=$(shell if [ -e /usr/src/compat-rdma$(OFED_VERSION) ];\
|
||||
+exists=$(shell if [ -e /usr/src/ofa_kernel-$(OFED_VERSION) ];\
|
||||
then echo y; fi)
|
||||
ifeq ($(exists), )
|
||||
-$(shell ln -s /usr/src/compat-rdma\
|
||||
- /usr/src/compat-rdma$(OFED_VERSION))
|
||||
+$(shell ln -s /usr/src/ofa_kernel\
|
||||
+ /usr/src/ofa_kernel-$(OFED_VERSION))
|
||||
endif
|
||||
-OFA_BUILD_PATH=/usr/src/compat-rdma$(OFED_VERSION)
|
||||
-OFA_KERNEL_PATH=/usr/src/compat-rdma$(OFED_VERSION)
|
||||
+OFA_BUILD_PATH=/usr/src/ofa_kernel/default
|
||||
+OFA_KERNEL_PATH=/usr/src/ofa_kernel-$(OFED_VERSION)/source
|
||||
EXTRA_CFLAGS += -DOFED_3_x
|
||||
-ofed_4_17_x=$(filter OFED-4.17%, $(ofed_major))
|
||||
+ofed_4_17_x=$(filter OFED-4.17% 5.%, $(ofed_major))
|
||||
ifneq ($(ofed_4_17_x), )
|
||||
EXTRA_CFLAGS += -D__OFED_BUILD__
|
||||
endif
|
||||
EXTRA_CFLAGS += -include $(OFA_KERNEL_PATH)/include/linux/compat-2.6.h
|
||||
|
||||
AUTOCONF_H = -include $(shell /bin/ls -1 $(LINUX)/include/*/autoconf.h 2> /dev/null | head -1)
|
||||
+AUTOCONF_H += -include $(LINUXSRC)/include/linux/kconfig.h
|
||||
endif #end non 3.x OFED
|
||||
+
|
||||
+ifneq ($(BNXT_PEER_MEM_INC),)
|
||||
KBUILD_EXTRA_SYMBOLS := $(BNXT_PEER_MEM_INC)/Module.symvers
|
||||
+else
|
||||
+KBUILD_EXTRA_SYMBOLS :=
|
||||
+endif
|
||||
|
||||
ifeq (OFED-NATIVE, $(findstring OFED-NATIVE, $(OFED_VERSION)))
|
||||
ifeq ($(KERNELRELEASE),)
|
||||
-OFA_BUILD_PATH =/lib/modules/`uname -r`/build/
|
||||
-OFA_KERNEL_PATH=/lib/modules/`uname -r`/source/
|
||||
+OFA_BUILD_PATH=/usr/src/kernels/$(KVER)/
|
||||
+OFA_KERNEL_PATH=/usr/src/kernels/$(KVER)/
|
||||
else
|
||||
-OFA_BUILD_PATH =/lib/modules/$(KERNELRELEASE)/build/
|
||||
-OFA_KERNEL_PATH=/lib/modules/$(KERNELRELEASE)/source/
|
||||
+OFA_BUILD_PATH=/usr/src/kernels/$(KERNELRELEASE)/
|
||||
+OFA_KERNEL_PATH=/usr/src/kernels/$(KERNELRELEASE)/
|
||||
endif
|
||||
else
|
||||
# Add OFED symbols only if external OFED is used
|
||||
--
|
||||
2.29.2
|
||||
|
||||
@ -0,0 +1,45 @@
|
||||
From c6e38e23c1a2d45efcab73793ebe28f8fcb504e4 Mon Sep 17 00:00:00 2001
|
||||
From: "M. Vefa Bicakci" <vefa.bicakci@windriver.com>
|
||||
Date: Fri, 4 Feb 2022 19:44:06 -0500
|
||||
Subject: [PATCH] bnxt_en/bnxt_compat.h: Fix up a build failure
|
||||
|
||||
This commit resolves the following build failure:
|
||||
|
||||
In file included from ./include/net/sock.h:59,
|
||||
from ./include/linux/tcp.h:19,
|
||||
from ./include/linux/ipv6.h:87,
|
||||
from ./include/net/dsfield.h:12,
|
||||
from ./include/net/ip_tunnels.h:13,
|
||||
from ./include/net/dst_metadata.h:6,
|
||||
from /builddir/build/BUILD/netxtreme-bnxt_en-1.10.2-219.0.55.0/bnxt_en-1.10.2-219.0.55.0/bnxt.h:30,
|
||||
from /builddir/build/BUILD/netxtreme-bnxt_en-1.10.2-219.0.55.0/bnxt_en-1.10.2-219.0.55.0/bnxt_hwrm.c:26:
|
||||
./include/linux/filter.h:980:1: error: useless type name in empty declaration [-Werror]
|
||||
void bpf_warn_invalid_xdp_action(u32 act);
|
||||
^~~~
|
||||
cc1: all warnings being treated as errors
|
||||
|
||||
The problem arises from the fact that the function
|
||||
bpf_warn_invalid_xdp_action is already defined in the kernel. Having a
|
||||
placeholder macro definition with the same name causes the compilation
|
||||
warning (which turns into an error due to -Werror).
|
||||
|
||||
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
|
||||
---
|
||||
bnxt_en-1.10.2-220.0.13.0/bnxt_compat.h | 1 -
|
||||
1 file changed, 1 deletion(-)
|
||||
|
||||
diff --git a/bnxt_en-1.10.2-220.0.13.0/bnxt_compat.h b/bnxt_en-1.10.2-220.0.13.0/bnxt_compat.h
|
||||
index 63d27f2e8979..aeb15fc9ab2a 100644
|
||||
--- a/bnxt_en-1.10.2-220.0.13.0/bnxt_compat.h
|
||||
+++ b/bnxt_en-1.10.2-220.0.13.0/bnxt_compat.h
|
||||
@@ -1596,7 +1596,6 @@ enum xdp_action {
|
||||
|
||||
#ifndef HAVE_BPF_TRACE
|
||||
#define trace_xdp_exception(dev, xdp_prog, act)
|
||||
-#define bpf_warn_invalid_xdp_action(act)
|
||||
#endif
|
||||
|
||||
#ifdef HAVE_XDP_RXQ_INFO
|
||||
--
|
||||
2.29.2
|
||||
|
||||
@ -0,0 +1,134 @@
|
||||
From f278ab69adb9e59041d2b8e78b6005587f94d0e3 Mon Sep 17 00:00:00 2001
|
||||
From: "M. Vefa Bicakci" <vefa.bicakci@windriver.com>
|
||||
Date: Wed, 2 Mar 2022 14:14:12 -0500
|
||||
Subject: [PATCH] bnxt_en, bnxt_re: Use irq_update_affinity_hint
|
||||
|
||||
This commit updates the bnxt_en and bnxt_re device drivers to use the
|
||||
irq_update_affinity_hint function instead of the irq_set_affinity_hint
|
||||
function, so that StarlingX's CPU affinity requirements for interrupts
|
||||
are met. (Note that the StarlingX kernel has been patched to add the
|
||||
irq_update_affinity_hint API function, which originally did not exist in
|
||||
kernel version 5.10.)
|
||||
|
||||
irq_set_affinity_hint sets the IRQ affinity hint as well as the actual
|
||||
IRQ affinity, which causes the IRQ affinity to not align with the value
|
||||
of the irqaffinity= command line option.
|
||||
|
||||
StarlingX's user-space initialization scripts set the IRQ affinities
|
||||
too, but if a network interface is initialized after the completion of
|
||||
the initialization scripts, then the network adapter's preferred IRQ
|
||||
affinity passed to irq_set_affinity_hint takes effect. The latter is
|
||||
undesirable if the CPU affinity of an interrupt includes CPUs that are
|
||||
in the set of isolated CPUs configured by StarlingX, as isolated CPUs
|
||||
should not service unrelated interrupts.
|
||||
|
||||
To resolve the aforementioned issues, this patch updates the bnxt_en and
|
||||
bnxt_re drivers to use irq_update_affinity_hint, which only sets the IRQ
|
||||
affinity hint, and not the actual IRQ affinity. This allows interrupts
|
||||
to be affined to the CPUs listed in the irqaffinity= command line
|
||||
argument by default, and eliminates the need to set the IRQ affinities
|
||||
after the initialization of network interfaces.
|
||||
|
||||
Finally, please note that the changes in bnxt_en were confirmed to not
|
||||
be compiled with StarlingX's 5.10-based kernel, because of the following
|
||||
conditional compilation pre-processor directives that are above the
|
||||
irq_set_affinity_hint/irq_update_affinity_hint call site:
|
||||
|
||||
#if (!defined(CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR) || \
|
||||
KERNEL_VERSION(4, 15, 0) > LINUX_VERSION_CODE) && \
|
||||
(defined(HAVE_CPUMASK_LOCAL_FIRST) || \
|
||||
defined(HAVE_CPUMASK_LOCAL_SPREAD))
|
||||
|
||||
/* ... */
|
||||
cpumask_set_cpu(cpumask_local_spread(i, numa_node),
|
||||
irq->cpu_mask);
|
||||
/* ... */
|
||||
rc = irq_update_affinity_hint(irq->vector, irq->cpu_mask);
|
||||
/* ... */
|
||||
|
||||
The IRQ matrix allocator is enabled in StarlingX's kernel, which also
|
||||
has a version greater than 4.15. 4.15 happens to be the kernel version
|
||||
with which the IRQ matrix allocator was introduced. In other words, the
|
||||
call site in question is not compiled at all with the out-of-tree
|
||||
bnxt_en driver. The in-tree version of bnxt_en does not have conditional
|
||||
compilation directives around the same code (i.e., the code is always
|
||||
compiled in).
|
||||
|
||||
Our imperfect understanding of the intent is that the out-of-tree
|
||||
bnxt_en driver's authors decided to rely on the IRQ matrix allocator's
|
||||
default IRQ vector distribution behaviour (which appears to occur at a
|
||||
lower level) instead of setting the IRQ affinity hints explicitly.
|
||||
|
||||
Thankfully, the lack of a call irq_update_affinity_hint does not prevent
|
||||
the irqaffinity= command line argument from taking effect for such
|
||||
interrupts. This was confirmed with a virtual machine and the e1000e
|
||||
driver, which also manages multiple MSI-X interrupts and which also does
|
||||
not explicitly set the IRQ affinities nor affinity hints. In our
|
||||
experiments, the IRQ affinities of the emulated Intel network adapter
|
||||
correctly matched the CPU list set by the irqaffinity= command line
|
||||
argument.
|
||||
|
||||
Despite what is discussed above, the code changes in bnxt_en are kept
|
||||
for completeness.
|
||||
|
||||
Signed-off-by: M. Vefa Bicakci <vefa.bicakci@windriver.com>
|
||||
---
|
||||
bnxt_en-1.10.2-220.0.13.0/bnxt.c | 6 +++---
|
||||
bnxt_re-220.0.12.0/qplib_fp.c | 6 +++---
|
||||
2 files changed, 6 insertions(+), 6 deletions(-)
|
||||
|
||||
diff --git a/bnxt_en-1.10.2-220.0.13.0/bnxt.c b/bnxt_en-1.10.2-220.0.13.0/bnxt.c
|
||||
index b0b99338c425..a2f55dfcd0c3 100644
|
||||
--- a/bnxt_en-1.10.2-220.0.13.0/bnxt.c
|
||||
+++ b/bnxt_en-1.10.2-220.0.13.0/bnxt.c
|
||||
@@ -10716,7 +10716,7 @@ static void bnxt_free_irq(struct bnxt *bp)
|
||||
(defined(HAVE_CPUMASK_LOCAL_FIRST) || \
|
||||
defined(HAVE_CPUMASK_LOCAL_SPREAD))
|
||||
if (irq->have_cpumask) {
|
||||
- irq_set_affinity_hint(irq->vector, NULL);
|
||||
+ irq_update_affinity_hint(irq->vector, NULL);
|
||||
free_cpumask_var(irq->cpu_mask);
|
||||
irq->have_cpumask = 0;
|
||||
}
|
||||
@@ -10787,10 +10787,10 @@ static int bnxt_request_irq(struct bnxt *bp)
|
||||
break;
|
||||
}
|
||||
#endif
|
||||
- rc = irq_set_affinity_hint(irq->vector, irq->cpu_mask);
|
||||
+ rc = irq_update_affinity_hint(irq->vector, irq->cpu_mask);
|
||||
if (rc) {
|
||||
netdev_warn(bp->dev,
|
||||
- "Set affinity failed, IRQ = %d\n",
|
||||
+ "Update affinity failed, IRQ = %d\n",
|
||||
irq->vector);
|
||||
break;
|
||||
}
|
||||
diff --git a/bnxt_re-220.0.12.0/qplib_fp.c b/bnxt_re-220.0.12.0/qplib_fp.c
|
||||
index ef5f279fcb79..60d2eda3727d 100644
|
||||
--- a/bnxt_re-220.0.12.0/qplib_fp.c
|
||||
+++ b/bnxt_re-220.0.12.0/qplib_fp.c
|
||||
@@ -492,7 +492,7 @@ void bnxt_qplib_nq_stop_irq(struct bnxt_qplib_nq *nq, bool kill)
|
||||
bnxt_qplib_ring_nq_db(&nq->nq_db.dbinfo, res->cctx, false);
|
||||
/* Sync with last running IRQ handler */
|
||||
synchronize_irq(nq->msix_vec);
|
||||
- irq_set_affinity_hint(nq->msix_vec, NULL);
|
||||
+ irq_update_affinity_hint(nq->msix_vec, NULL);
|
||||
free_irq(nq->msix_vec, nq);
|
||||
|
||||
/* Cleanup Tasklet */
|
||||
@@ -546,10 +546,10 @@ int bnxt_qplib_nq_start_irq(struct bnxt_qplib_nq *nq, int nq_indx,
|
||||
|
||||
cpumask_clear(&nq->mask);
|
||||
cpumask_set_cpu(nq_indx, &nq->mask);
|
||||
- rc = irq_set_affinity_hint(nq->msix_vec, &nq->mask);
|
||||
+ rc = irq_update_affinity_hint(nq->msix_vec, &nq->mask);
|
||||
if (rc)
|
||||
dev_warn(&res->pdev->dev,
|
||||
- "QPLIB: set affinity failed; vector: %d nq_idx: %d\n",
|
||||
+ "QPLIB: update affinity failed; vector: %d nq_idx: %d\n",
|
||||
nq->msix_vec, nq_indx);
|
||||
nq->requested = true;
|
||||
bnxt_qplib_ring_nq_db(&nq->nq_db.dbinfo, res->cctx, true);
|
||||
--
|
||||
2.29.2
|
||||
|
||||
@ -1089,3 +1089,5 @@ CONFIG_GVE=m
|
||||
# upgrade to 5.10.99
|
||||
# CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
|
||||
# CONFIG_FRAMEBUFFER_CONSOLE_LEGACY_ACCELERATION is not set
|
||||
|
||||
# CONFIG_BNXT is not set
|
||||
|
||||
@ -1597,3 +1597,5 @@ CONFIG_PCI_REALLOC_ENABLE_AUTO=y
|
||||
# upgrade to 5.10.99
|
||||
# CONFIG_BPF_UNPRIV_DEFAULT_OFF is not set
|
||||
# CONFIG_FRAMEBUFFER_CONSOLE_LEGACY_ACCELERATION is not set
|
||||
|
||||
# CONFIG_BNXT is not set
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user