When Ironic's CI went to Ubuntu Noble, IPv6 testing broke.
Ultimately, that was because the job was changed from BIOS
boot mode to UEFI.
Which in theory was a good thing.
But in reviewing the EDK2 known bugs[0], it is clear IPv6 testing
moving forward needs to be manual and cannot be done automatically.
For the reader's context, EDK2 is the source of the firmware used
for Libvirt UEFI VMs.
In attempting to work through the issues, we discovered:
- EDK2 doesn't care about stateless (auto discovery) mode, and
thus doesn't work when that is set. It attempts to solicit
an address, which is an open bug [1] from 2019, opened by
an Ironic contributor.
- EDK2 *always* attempts PXE v4 first, which takes 60 seconds to
timeout, before attempting PXE v6.
- PXE v6 *might* work before it times out, which takes 5 minutes to
occur, if the dhcp server has cleared leases, but there is no
guarentee. In most cases the result is the DHCP server views the
address as handed out and that there are no free addresses available
to supply. This is also rooted in bug [2], also opened in 2019.
- EDK2 then switches gears and tries HTTPBoot... simiarly, but
the way it does so, noted in bug [3], is also incompatible
with dnsmasq.
There are additional bugs, but one sort of gets the idea. Some of this
is compounded by aspects like dnsmasq attempting to be strict about
responding to requests in the DHCPv6 model. A different DHCP server
*might* demonstrate a little differently, but fundimentally the same
underlying issues in EDK2 will make testing difficult.
In attempting to fix this issue, we also attempted to revert back to
BIOS mode. This mode uses iPXE ROM images built for QEMU, yet we
quickly discovered these pre-built ROM images lacked IPv6 support
in Ubuntu Noble. This likely a regression of Ubuntu, but bug tracking
points directly to Upstream iPXE which is not valid as it is a compile
time option.
Testing the ROMs showed only DHCPv4 being attempted and IPv6 router
advertisements being entirely ignored. In a sense, if it did work,
it would still kind of be cheating as the iPXE ROM is able to skip
the first part of the complexity related to PXE in general.
In other words, it is not an entirely realistic test
when compared to Bare Metal.
As such, we don't have a forward path to "fix" this CI job as is.
We know the code works. We know vendor firmware sometimes has quarks like
needing stateless or stateful operation, We know Ironic does the
right thing... within it's capabilities. We just can't test this in
CI.
[0]: https://github.com/tianocore/edk2/issues?q=is%3Aissue%20state%3Aopen%20%20ipv6
[1]: https://github.com/tianocore/edk2/issues/9832
[2]: https://github.com/tianocore/edk2/issues/9828
[3]: https://github.com/tianocore/edk2/issues/9689
Change-Id: Ifc25bc1e1abb949892a1297a313d63f74937c9a1
be2bd0ea04