IPv4/IPv6 Dual Stack Networking

Story: 2011027
Task: 50147

Change-Id: Ic8688741c99af0c85c62e293e8f21f19831a51cf
Signed-off-by: Suzana Fernandes <Suzana.Fernandes@windriver.com>
This commit is contained in:
Suzana Fernandes 2024-09-23 11:46:27 +00:00 committed by Juanita-Balaraj
parent 8ba09ff4d7
commit 0055b64777
4 changed files with 411 additions and 137 deletions

View File

@ -129,41 +129,21 @@ procedure is performed only on the subcloud.
.. rubric:: |proc|
#. Delete the admin address pool.
.. code-block:: none
~(keystone_admin)]$ system addrpool-delete <admin-address-pool-uuid>
.. note::
This will automatically delete the admin network and unassign it from the
admin interface.
#. Create a new admin network address pool.
#. Modify the admin network address pool.
For example:
.. code-block:: none
~(keystone_admin)]$ system addrpool-add --floating-address 192.168.103.2 --controller0-address 192.168.103.3 --controller1-address 192.168.103.4 --gateway-address 192.168.103.1 admin 192.168.103.0 24
#. Create a new admin network.
For example:
.. code-block:: none
~(keystone_admin)]$ system network-add admin admin false <admin-address-pool-uuid>
#. Assign the new admin network to the admin interface.
.. code-block:: none
~(keystone_admin)]$ system interface-network-assign controller-0 enp0s9 admin
~(keystone_admin)]$ system addrpool-modify <admin-pool-uuid> \
--network 192.168.103.0 --prefix 24 \
--floating-address 192.168.103.2 \
--controller0-address 192.168.103.3 \
--controller1-address 192.168.103.4 \
--gateway-address 192.168.103.1
#. On the system controller, perform the following:
#. Unmanage the subcloud.
.. code-block:: none
@ -200,104 +180,18 @@ procedure is performed only on the subcloud.
.. rubric:: |proc|
#. Delete the current admin address pool.
#. Modify the admin network address pool.
For example:
.. code-block:: none
~(keystone_admin)]$ system addrpool-delete <admin-address-pool-uuid>
.. note::
- During the creation of a new network, you will get the error message
``ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)`` whenever you
connect to the subcloud bootstrap address. This error will disappear
when the network creation is completed. The **250.001** alarm will be raised
after deleting the address pool. Wait for the alarm to be clear before proceeding.
- The system will automatically delete the admin network and
interface-network association while deleting the address pool.
- The endpoints associated with |prod-dc| management will revert to the
endpoints of the management network. This can take a few minutes and
can be obtained using the :command:`openstack endpoint list` command on the
subcloud.
.. _swacktocontroller1:
#. Swact to controller-1.
.. code-block:: none
~(keystone_admin)]$ system host-swact controller-0
.. _lockcontroller0:
#. Lock controller-0 and delete the current admin interface.
.. code-block:: none
~(keystone_admin)]$ system host-lock controller-0
.. code-block:: none
~(keystone_admin)]$ system host-if-delete controller-0 <admin interface>
#. Create a network interface for the new admin network.
.. code-block:: none
~(keystone_admin)]$ system host-if-add -V <interface vlan> -c platform controller-0 <new-admin-interface> vlan <interface port>
.. _unlockcontroller0:
#. Unlock controller-0.
.. code-block:: none
~(keystone_admin)]$ system host-unlock controller-0
#. Wait for the controller-0 to be available and for the alarms to be clear.
.. _swacktocontroller0:
#. Swact to controller-0.
.. code-block:: none
~(keystone_admin)]$ system host-swact controller-1
Now, repeat steps :ref:`3 <lockcontroller0>` to :ref:`5 <unlockcontroller0>` pointing to controller-1. Wait for the
controller-1 to be available.
.. note::
You need to perform steps :ref:`2 <swacktocontroller1>` to :ref:`7 <swacktocontroller0>` only when a new admin interface that is
different from the original one needs to be used. That means, you can
skip to step :ref:`8 <createnewadminnetworkaddresspool>` only if the subnet (addressing) parameters need to be
changed.
.. _createnewadminnetworkaddresspool:
#. Create a new admin network address pool.
Example:
.. code-block:: none
-~(keystone_admin)]$ system addrpool-add --floating-address 192.168.103.2 --controller0-address 192.168.103.3 --controller1-address 192.168.103.4 --gateway-address 192.168.103.1 admin 192.168.103.0 24
#. Create a new admin network.
Example:
.. code-block:: none
~(keystone_admin)]$ system network-add admin admin false <admin-address-pool-uuid>
#. Assign the new admin network to the admin interfaces of controller-0 and controller-1.
.. code-block:: none
~(keystone_admin)]$ system interface-network-assign controller-0 <new-admin-interface> admin
.. code-block:: none
~(keystone_admin)]$ system interface-network-assign controller-1 <new-admin-interface> admin
~(keystone_admin)]$ system addrpool-modify <admin-pool-uuid> \
--network 192.168.103.0 --prefix 24 \
--floating-address 192.168.103.2 \
--controller0-address 192.168.103.3 \
--controller1-address 192.168.103.4 \
--gateway-address 192.168.103.1
.. note::
@ -306,14 +200,6 @@ procedure is performed only on the subcloud.
can be obtained using the :command:`openstack endpoint list` command on
the subcloud.
#. Before going back to the system controller, perform an additional lock/unlock
on both controller-0 and controller-1 and the hosts should be available again.
.. note::
This is needed for the new admin network that uses a different interface
from the original.
#. On the system controller, perform the following:
#. Unmanage the subcloud.

View File

@ -28,6 +28,102 @@ Horizon Web Interface. You can use IPv4 or IPv6 addresses.
addresses in their |SANs|. However, any optional applications must have
their certificate |SANs| updated manually.
------------------------------
OAM Address Pool Configuration
------------------------------
The system supports the dual-stack (IPv4 and IPv6) values for the affected pools.
To modify the values, use the following command:
.. code-block:: none
system addrpool-modify [--name <name>] [--network <network address>] \
[--prefix <network prefix>] [--ranges <ranges>] \
[--order <sequential | random>] \
[--floating-address <floating address>] \
[--controller0-address <controller0 address>] \
[--controller1-address <controller1 address>] \
[--gateway-address <gateway address>] \
<address_pool uuid>
To view the existing |OAM| pools, use the following command.
.. code-block:: none
~(keystone_admin)]# system network-addrpool-list
+-------...----------------+-----------------+-----------------------------+
| uuid ... | network_name | addrpool_name |
+-------...----------------+-----------------+-----------------------------+
| 54911d...b6-62266582a8f4 | admin | admin-ipv6 |
| 7a54f0...61-55eb6b12b1c5 | cluster-host | cluster-host-subnet-ipv6 |
| 12204f...24-1ea6400da629 | cluster-pod | cluster-pod-subnet-ipv6 |
| 6d6795...3c-2506fdf386b1 | cluster-service | cluster-service-subnet-ipv6 |
| f82081...48-662f8c48466c | mgmt | management-ipv6 |
| 7fbd42...4a-4807cf039a0e | multicast | multicast-subnet-ipv6 |
| 0f1ab4...0c-bca519d752b6 | oam | oam-ipv4 |
| a1d33e...f1-a5523c2bf079 | oam | oam-ipv6 |
| 55a0d2...a0-eff1e72ff05a | pxeboot | pxeboot |
+-------...----------------+-----------------+-----------------------------+
Get the |OAM| pool uuid with the following command:
.. code-block:: none
~(keystone_admin)]# system addrpool-list --nowrap
+--------------------------------------+-----------------------------+---------------+--------+-...-+------------------+---------------------+---------------------+-----------------+
| uuid | name | network | prefix | ... | floating_address | controller0_address | controller1_address | gateway_address |
+--------------------------------------+-----------------------------+---------------+--------+-...-+------------------+---------------------+---------------------+-----------------+
| 7ac2e794-184f-4b40-8529-e07b52bfd2c2 | admin-ipv6 | 3333:: | 64 | ... | 3333::2 | 3333::3 | 3333::4 | None |
| 6250edb8-15f5-4204-80f1-8e54b9e28a5a | cluster-host-subnet-ipv6 | fd02:: | 64 | ... | fd02::1 | fd02::2 | fd02::3 | None |
| f4c9560c-47e5-46bd-aff5-18642831b1da | cluster-pod-subnet-ipv6 | fd03:: | 64 | ... | None | None | None | None |
| a6366aab-b3c1-4947-97e5-f5171e0e2f3e | cluster-service-subnet-ipv6 | fd04:: | 112 | ... | None | None | None | None |
| 412aebff-9a86-40b1-a379-752f00a0c3a0 | management-ipv6 | fd01:: | 64 | ... | fd01::1 | fd01::2 | fd01::3 | None |
| 7c445f38-067c-4b3c-a511-d8e00da5791c | multicast-subnet-ipv6 | ff08::1:1:0 | 124 | ... | None | None | None | None |
| 4affb8fb-6b67-467a-9674-cfa6743ca5c5 | oam-ipv4 | 20.20.20.0 | 24 | ... | 20.20.20.1 | 20.20.20.2 | 20.20.20.3 | None |
| b46512d7-5404-4daa-a64d-fc510e0c5864 | oam-ipv6 | fd00:: | 64 | ... | fd00::a14:103 | fd00::a14:104 | fd00::a14:105 | fd00::1 |
| 05fde56d-f26a-4ea4-8b32-1ebf868743e2 | pxeboot | 169.254.202.0 | 24 | ... | 169.254.202.1 | 169.254.202.2 | 169.254.202.3 | None |
+--------------------------------------+-----------------------------+---------------+--------+-...-+------------------+---------------------+---------------------+-----------------+
~(keystone_admin)]# system addrpool-show b46512d7-5404-4daa-a64d-fc510e0c5864
+---------------------+---------------------------------------+
| Property | Value |
+---------------------+---------------------------------------+
| uuid | b46512d7-5404-4daa-a64d-fc510e0c5864 |
| name | oam-ipv6 |
| network | fd00:: |
| prefix | 64 |
| order | random |
| ranges | ['fd00::1-fd00::ffff:ffff:ffff:fffe'] |
| floating_address | fd00::a14:103 |
| controller0_address | fd00::a14:104 |
| controller1_address | fd00::a14:105 |
| gateway_address | fd00::1 |
+---------------------+---------------------------------------+
Modify the pool, in this example ``oam-ipv6``, using the following parameters:
.. note::
If the system is dual-stack open a ssh session via the other |OAM| network
configured before proceding, to not loose the connection.
.. code-block:: none
system addrpool-modify b46512d7-5404-4daa-a64d-fc510e0c5864 \
--floating-address fd00::3 \
--controller0-address fd00::4 \
--controller1-address fd00::5 \
--gateway-address fd00::2
For |AIO-SX| ``controller0-address`` and ``controller1-address`` do not take effect.
-------------------------
Legacy OAM Configuration
-------------------------
The set of commands below can also be used instead of ``addrpool`` commands. It
affects only the |OAM| primary address pool. If using CLI a warning is displayed
showing the command is deprecated, but can still be used.
To view the existing |OAM| IP configuration, use the following command.
.. code-block:: none
@ -47,7 +143,7 @@ To view the existing |OAM| IP configuration, use the following command.
| uuid | 2818e7c4-f730-43bd-b33d-eaff53a92ee1 |
+-----------------+--------------------------------------+
To change the OAM IP subnet, floating IP address, gateway IP address, or
To change the |OAM| IP subnet, floating IP address, gateway IP address, or
controller IP addresses, use the following command syntax.
.. code-block:: none
@ -69,7 +165,7 @@ For example:
oam_c1_ip=10.10.10.4
.. note::
On AIO Simplex systems, the
On |AIO-SX| systems, the
oam_floating_ip, oam_c0\_ip and oam_c0\_ip parameters are not
supported. To change the |OAM| IP address of a Simplex System, the parameter
oam_ip must be used in combination with oam_gateway_ip and oam_subnet.
@ -84,13 +180,16 @@ For example:
If you change the IP address version (IPv4 or IPv6), ensure that the
same version is used for the DNS and NTP servers.
.. rubric:: |postreq|
After changing the |OAM| server configuration, you must lock and unlock the
controllers. This process requires a swact on the controllers. Then you must
lock and unlock the worker nodes one at a time, ensuring that sufficient
resources are available to migrate any running instances.
.. note::
On AIO Simplex systems you do not need to lock and unlock the host. The
On |AIO-SX| systems you do not need to lock and unlock the host. The
changes are applied automatically.
In the |prod-dc| system, if the system controller |OAM| address is changed
@ -108,8 +207,6 @@ points to the new value. Run the following command on all subclouds:
Adjust the necessary parameters based on the altered values in the system controller.
.. rubric:: |postreq|
(Optional) If you are running Docker proxy on your system, you need to
change the **no_proxy** service parameter using the CLI to append the new
OAM IP, using the following steps:
@ -128,10 +225,13 @@ OAM IP, using the following steps:
#. To apply the changes, use the :command:`system service-parameter-apply docker`
command.
#. Lock and unlock the controller(s).
#. If it is not |AIO-SX|, then lock and unlock the controller(s), otherwise the
|OAM| modification will be applied immediately.
.. seealso::
:ref:`Default Firewall Rules <security-default-firewall-rules>`
:ref:`Modify Firewall Options <security-firewall-options>`
:ref:`IPv4/IPv6 Dual Stack Networking <dual-stack-support-318550fd91b5>`

View File

@ -0,0 +1,277 @@
.. WARNING: Add no lines of text between the label immediately following
.. and the title.
.. _dual-stack-support-318550fd91b5:
============================
IPv4/IPv6 Dual-Stack Network
============================
Dual-stack networking facilitates the simultaneous use of both IPv4 and IPv6
addresses, or continue to use each IP version independently. To accomplish
this, platform networks can be associated with 1 or 2 address pools, one for
each IP version (IPv4 or IPv6). The first pool is linked to the network
upon creation and cannot be subsequently removed. The second pool can be added or
removed to transition the system between dual-stack and single-stack modes.
The |PXE| boot network is an exception, as it currently only supports IPv4. Other
platform networks can be configured as either single-stack or dual-stack based
on specific requirements. For internal management communication among
controllers, workers, and storage nodes, the primary address pool is used as
encryption is currently only available for the primary pool on the management
network.
Once created, a network's primary address pool family cannot be modified.
Reinstalling the system is necessary to change it. While it is possible to
edit address pool addresses for management, |OAM|, and admin networks, all
addresses within a pool must belong to the same address family.
API and Command Line Interface Considerations
=============================================
The following system APIs handle the association between network and address pool:
* network-addrpool-assign: creates the association between a network and address pool
* network-addrpool-remove: removes the association between a network and address pool
* network-addrpool-list: lists all associations
* network-addrpool-show: shows a specific association
The first association is done internally when the network is created using the
command format:
.. code-block::
~(keystone_admin)]$ system network-add <network_name> <network type> <dynamic> <pool_uuid>
The ``addrpool-modify`` command allows to edit all its parameters with the CLI.
Follows the command format:
.. code-block::
~(keystone_admin)]$ system addrpool-modify [--name <name>] [--network <network address>] [--prefix <network prefix>] [--ranges <ranges>] [--order <sequential | random>] [--floating-address <floating address>] [--controller0-address <controller0 address>] [--controller1-address <controller1 address>] [--gateway-address <gateway address>] <address_pool uuid>
Install a System in Dual-Stack
==============================
It is possible to install a system in dual-stack by adding the secondary
subnets into the bootstrap variables with comma separated values as shown in
the following example:
.. code-block::
pxeboot_subnet: 198.51.100.0/24
management_subnet: fd01::/64,198.51.0.0/24
management_start_address: fd01::2,198.51.0.2
management_end_address: fd01::ffff,198.51.0.200
management_gateway_address: fd01::1,198.51.0.1
external_oam_subnet: fd00::/64,10.20.5.0/24
external_oam_gateway_address: fd00::1,10.20.5.1
external_oam_floating_address: fd00::3,10.20.5.3
external_oam_node_0_address: fd00::4,10.20.5.4
external_oam_node_1_address: fd00::5,10.20.5.5
cluster_host_subnet: aefd:100::/64,198.51.100.0/24
cluster_pod_subnet: aefd:206::/64,203.0.113.0/24
cluster_service_subnet: aefd:207::/112,10.96.0.0/12
The order in which networks are listed determines the primary and secondary
address pools. It is important to note that all primary address pools must use
the same address family. For example, if the primary address pool for one
subnet is IPv6, all other subnets primary pools must also be IPv6 (as in the
example above).
Configure DNS Server
====================
It is optional to configure |DNS| servers with both IPv4 and IPv6 addresses.
This can be achieved using the command format:
.. code-block::
~(keystone_admin)]$ system dns-modify nameservers=<IPv6 DNS server>,<IPv4 DNS server>
or
~(keystone_admin)]$ system dns-modify nameservers=<IPv4 DNS server>,<IPv6 DNS server>
If the selected |DNS| servers support both ``A`` and ``AAAA`` records, specifying an
address for each address family is unnecessary.
Distributed Cloud Operations
============================
Subclouds can be installed in a dual-stack configuration, if its version
supports the feature. All operational communication between the system
controller and subclouds uses the primary address pool. While the system
controller and subclouds can operate in different network modes (single-stack
or dual-stack), they must share the same primary address family. Geo redundancy
uses the primary address pools to communicate.
Public Endpoint Considerations
==============================
All available public |OAM| endpoints can be accessed through the secondary
address using the same L4 port. |HA| Proxy is used to map these external
requests to the corresponding internal endpoints.
Modify Network Addresses
========================
**Supported Networks**
Only the |OAM|, Admin, and Management networks can be modified using the ``addrpool-modify`` command during runtime.
**Other Networks**
For other networks, reinstallation is required to make changes.
External API and Command Line Interface Considerations
======================================================
**Deprecated**
The ``external-OAM`` API is marked as deprecated but can be used to modify the |OAM| network primary pool.
**CLI**
The corresponding CLIs are:
.. code-block::
~(keystone_admin)]$ system oam-modify <path=value> [<path=value> ...]
~(keystone_admin)]$ system oam-show
Enable Kubernetes in Dual-stack
===============================
To enable dual-stack functionality in Kubernetes, the |OAM|, cluster-host,
cluster-service, and cluster-pod networks must be configured to support it.
Making these changes at runtime triggers a quick restart for the ``kube-API-server``
and ``kube-controller-manager`` pods.
If converted during runtime, newly created pods will automatically receive
both primary and secondary addresses. Existing pods retain their current
primary addresses but will not acquire a secondary address until they are
restarted. The same happens from a dual-stack to single-stack configuration,
previously existing pods will retain their secondary address until restart.
Runtime Configuration
=====================
To add dual-stack in a running system the following sequence is suggested
(in the example below the system was installed as IPv6). The network's primary
family can be seen with:
.. code-block::
~(keystone_admin)]$ system network-list
+----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+
| id | uuid... | name | type | dynamic | pool_uuid | primary_pool_family |
+----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+
| 4 | 196d...3 | multicast | multicast | False | 7c445f38-067c-4b3c-a511-d8e00da5791c | IPv6 |
| 5 | 43fe...3 | cluster-host | cluster-host | True | 6250edb8-15f5-4204-80f1-8e54b9e28a5a | IPv6 |
| 3 | 9996...9 | oam | oam | False | b46512d7-5404-4daa-a64d-fc510e0c5864 | IPv6 |
| 6 | a374...7 | cluster-pod | cluster-pod | False | f4c9560c-47e5-46bd-aff5-18642831b1da | IPv6 |
| 7 | afc1...d | cluster-service | cluster-service | False | a6366aab-b3c1-4947-97e5-f5171e0e2f3e | IPv6 |
| 1 | b565...9 | mgmt | mgmt | True | 412aebff-9a86-40b1-a379-752f00a0c3a0 | IPv6 |
| 2 | bbb1...2 | pxeboot | pxeboot | True | 05fde56d-f26a-4ea4-8b32-1ebf868743e2 | IPv4 |
+----+-----...--+-----------------+-----------------+---------+--------------------------------------+---------------------+
Configure OAM Network
=====================
Add an address pool for |OAM|:
.. code-block::
~(keystone_admin)]$ system addrpool-add oam-ipv4 171.168.204.0 24 --order random --ranges 171.168.204.1-171.168.204.254 --floating-address 171.168.204.1 --controller0-address 171.168.204.2 --controller1-address 171.168.204.3
Then assign the newly created pool to the |OAM| network:
.. code-block::
~(keystone_admin)]$ system network-addrpool-assign oam oam-ipv4
If the system is AIO-SX, the new configuration is applied immediately,
otherwise it is necessary to lock/unlock both controllers.
Configure Cluster (pod/service/host) Network
============================================
The cluster networks in dual-stack converts kubernetes to dual-stack operation,
in this case, first make sure the |OAM| network is already configured in
dual-stack and then start by adding the correspondent pools:
.. code-block::
~(keystone_admin)]$ system addrpool-add cluster-pod-subnet-ipv4 172.16.0.0 16 --order random --ranges 172.16.0.1-172.16.254.254
~(keystone_admin)]$ system addrpool-add cluster-service-subnet-ipv4 10.96.0.0 12 --order random --ranges 10.96.0.1-10.96.254.254
~(keystone_admin)]$ system addrpool-add cluster-host-subnet-ipv4 192.168.204.0 24 --order random --ranges 192.168.204.1-192.168.204.254 --floating-address 192.168.204.1 --controller0-address 192.168.204.2 --controller1-address 192.168.204.3
Then associate the new pools to each network (there is no preferred order
among the three networks):
.. code-block::
~(keystone_admin)]$ system network-addrpool-assign cluster-service cluster-service-subnet-ipv4
~(keystone_admin)]$ system network-addrpool-assign cluster-pod cluster-pod-subnet-ipv4
~(keystone_admin)]$ system network-addrpool-assign cluster-host cluster-host-subnet-ipv4
After the third cluster network receives dual-stack kubernetes and
calico will be reconfigured with the ``kube-apiserver-controller`` and
``kube-controller-manager-controller`` restarts. The entire operation will
be performed without the need of a node lock/unlock cycle.
Configure Management Network
============================
As stated, the internal communication is done through the primary pool, but it
is possible to add dual-stack configuration by first adding the new pool:
.. code-block::
~(keystone_admin)]$ system addrpool-add management-ipv4 20.20.20.0 24 --order random --ranges 20.20.20.1-20.20.20.254 --floating-address 20.20.20.1 --controller0-address 20.20.20.2 --controller1-address 20.20.20.3
Then create the association:
.. code-block::
~(keystone_admin)]$ system network-addrpool-assign management management-ipv4
A ``Configuration Out-Of-Date`` alarm is raised for the affected nodes and a
node lock/unlock cycle will clean the alarm.
Configure Admin Network
=======================
This network is used by subclouds to communicate with its system-controller
and that is done through the primary pool. To add a dual-stack configuration
start with a new pool:
.. code-block::
~(keystone_admin)]$ system addrpool-add admin-ipv4 30.30.30.0 24 --order random --ranges 30.30.30.1-30.30.30.254 --floating-address 30.30.30.1 --controller0-address 30.30.30.2 --controller1-address 30.30.30.3
Then create the association:
.. code-block::
~(keystone_admin)]$ system network-addrpool-assign admin admin-ipv4
This is done in runtime on the affected controllers, no lock/unlock cycle is
required.
Revert to Single-stack
=======================
By removing the network association with the address pool the single-stack
configuration operates in a similar fashion that was done to configure
dual-stack. If the configuration was done at runtime, or if a node lock/unlock
cycle was required to configure dual-stack, the same happens when configuring
single-stack.
To remove a address pool association with a network use ``network-addrpool-remove``,
for example:
.. code-block::
~(keystone_admin)]$ ADDR_POOL_NAME=”cluster-pod-ipv6"
~(keystone_admin)]$ DEL=$(system network-addrpool-list | awk '$6 == $ADDR_POOL_NAME { print $2 }') && system network-addrpool-remove $DEL

View File

@ -78,6 +78,17 @@ GNSS and SyncE Support
gnss-and-synce-support-62004dc97f3e
-------------------------------
IPv4/IPv6 Dual Stack Networking
-------------------------------
.. toctree::
:maxdepth: 2
dual-stack-support-318550fd91b5
--------------------
OAM IP Configuration
--------------------