airship/maas
Code Issues Proposed changes

Adjust Redfish retry interval

Browse Source 
Patch the redfish driver to retry requests less aggressively, using
values that match the IPMI driver [0], instead of the defaults [1].

This helps prevent HTTP 409 and HTTP 500 errors.

0: https://git.launchpad.net/maas/tree/src/provisioningserver/drivers/power/ipmi.py?h=2.8.6#n234
1: https://git.launchpad.net/maas/tree/src/provisioningserver/drivers/power/__init__.py?h=2.8.6#n42

Change-Id: Ia41aafd04a6b8439e04fdd6d9f867a79f74789e1
This commit is contained in:
Phil Sphicas 2021-07-30 21:46:02 +00:00 committed by Andrii Ostapenko
parent d6d9b4c857
commit 2e94c847ac
2 changed files with 15 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
images/maas-rack-controller/2.8_redfish_retries.patch Normal file
View File

@ -0,0 +1,12 @@
diff --git a/src/provisioningserver/drivers/power/redfish.py b/src/provisioningserver/drivers/power/redfish.py
index 27f63545a..9c39d577e 100644
--- a/src/provisioningserver/drivers/power/redfish.py
+++ b/src/provisioningserver/drivers/power/redfish.py
@@ -149,6 +149,7 @@ class RedfishPowerDriver(RedfishPowerDriverBase):
make_setting_field("node_id", "Node ID", scope=SETTING_SCOPE.NODE),
]
ip_extractor = make_ip_extractor("power_address")
+ wait_time = (4, 8, 16, 32)
def detect_missing_packages(self):
# no required packages

3
images/maas-rack-controller/Dockerfile
View File

@ -58,10 +58,13 @@ COPY 2.8_nic_filter.patch /tmp/2.8_nic_filter.patch
COPY 2.8_secure_headers.patch /tmp/2.8_secure_headers.patch
# Patch so maas knows that "BMC error" is retriable
COPY 2.8_ipmi_error.patch /tmp/2.8_ipmi_error.patch
# Patch to space redfish request retries apart a bit, to avoid overwhelming the BMC
COPY 2.8_redfish_retries.patch /tmp/2.8_redfish_retries.patch
RUN cd /usr/lib/python3/dist-packages/provisioningserver/utils && patch network.py < /tmp/2.8_nic_filter.patch
RUN cd /usr/lib/python3/dist-packages/twisted/web && patch server.py < /tmp/2.8_secure_headers.patch
RUN cd /usr/lib/python3/dist-packages/provisioningserver/drivers/power && patch ipmi.py < /tmp/2.8_ipmi_error.patch
RUN cd /usr/lib/python3/dist-packages/provisioningserver/drivers/power && patch redfish.py < /tmp/2.8_redfish_retries.patch
# echo journalctl logs to the container's stdout
COPY scripts/journalctl-to-tty.service /etc/systemd/system/journalctl-to-tty.service