x/cloudbase-init
Code Issues Proposed changes

nocloud: support for cloud-init network version 1

Browse Source 
Added support for NoCloud service with Networking Config Version 1:
https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v1.html

Supported configuration types: Physical, Bond, Vlan and Nameserver.
Unsupported configuration types: Bridge, Route.

NoCloud metadata folder should contain a file network-config.
If no file is found, it falls back to using the contents for network-interfaces
key from the metadata file.

Example:

```yaml
version: 1
config:
   - type: physical
     name: interface0
     mac_address: "52:54:00:12:34:00"
     mtu: 1450
     subnets:
        - type: static
          address: 192.168.1.10
          netmask: 255.255.255.0
          dns_nameservers:
            - 192.168.1.11
   - type: bond
     name: bond0
     bond_interfaces:
       - gbe0
       - gbe1
     mac_address: "52:54:00:12:34:00"
     params:
       bond-mode: active-backup
       bond-lacp-rate: false
     mtu: 1450
     subnets:
        - type: static
          address: 192.168.1.10
          netmask: 255.255.255.0
          dns_nameservers:
            - 192.168.1.11
   - type: vlan
     name: vlan0
     vlan_link: eth1
     vlan_id: 150
     mac_address: "52:54:00:12:34:00"
     mtu: 1450
     subnets:
        - type: static
          address: 192.168.1.10
          netmask: 255.255.255.0
          dns_nameservers:
            - 192.168.1.11
   - type: nameserver
     address:
       - 192.168.23.2
       - 8.8.8.8
     search: acme.local
```

Change-Id: Idc413f6f9f001b2327c33a796e9ed494be28ce26
This commit is contained in:
Adrian Vladu 2020-04-28 14:25:55 +03:00
parent 5c3979e1ae
commit 246db50591
4 changed files with 526 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

276
cloudbaseinit/metadata/services/nocloudservice.py
View File

@ -11,13 +11,17 @@
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import netaddr
from oslo_log import log as oslo_logging
from cloudbaseinit import conf as cloudbaseinit_conf
from cloudbaseinit import exception
from cloudbaseinit.metadata.services import base
from cloudbaseinit.metadata.services import baseconfigdrive
from cloudbaseinit.models import network as network_model
from cloudbaseinit.utils import debiface
from cloudbaseinit.utils import network as network_utils
from cloudbaseinit.utils import serialization
@ -25,6 +29,249 @@ CONF = cloudbaseinit_conf.CONF
LOG = oslo_logging.getLogger(__name__)
class NoCloudNetworkConfigV1Parser(object):
NETWORK_LINK_TYPE_PHY = 'physical'
NETWORK_LINK_TYPE_BOND = 'bond'
NETWORK_LINK_TYPE_VLAN = 'vlan'
NETWORK_SERVICE_NAMESERVER = 'nameserver'
SUPPORTED_NETWORK_CONFIG_TYPES = [
NETWORK_LINK_TYPE_PHY,
NETWORK_LINK_TYPE_BOND,
NETWORK_LINK_TYPE_VLAN,
NETWORK_SERVICE_NAMESERVER
]
def _parse_subnets(self, subnets, link_name):
networks = []
if not subnets or not isinstance(subnets, list):
LOG.warning("Subnets '%s' is empty or not a list.",
subnets)
return networks
for subnet in subnets:
if not isinstance(subnet, dict):
LOG.warning("Subnet '%s' is not a dictionary",
subnet)
continue
if subnet.get("type") in ["dhcp", "dhcp6"]:
continue
routes = []
for route_data in subnet.get("routes", []):
route_netmask = route_data.get("netmask")
route_network = route_data.get("network")
route_network_cidr = network_utils.ip_netmask_to_cidr(
route_network, route_netmask)
route_gateway = route_data.get("gateway")
route = network_model.Route(
network_cidr=route_network_cidr,
gateway=route_gateway
)
routes.append(route)
address_cidr = subnet.get("address")
netmask = subnet.get("netmask")
if netmask:
address_cidr = network_utils.ip_netmask_to_cidr(
address_cidr, netmask)
gateway = subnet.get("gateway")
if gateway:
# Map the gateway as a default route, depending on the
# IP family / version (4 or 6)
gateway_net_cidr = "0.0.0.0/0"
if netaddr.valid_ipv6(gateway):
gateway_net_cidr = "::/0"
routes.append(
network_model.Route(
network_cidr=gateway_net_cidr,
gateway=gateway
)
)
networks.append(network_model.Network(
link=link_name,
address_cidr=address_cidr,
dns_nameservers=subnet.get("dns_nameservers"),
routes=routes
))
return networks
def _parse_physical_config_item(self, item):
if not item.get('name'):
LOG.warning("Physical NIC does not have a name.")
return
link = network_model.Link(
id=item.get('name'),
name=item.get('name'),
type=network_model.LINK_TYPE_PHYSICAL,
enabled=True,
mac_address=item.get('mac_address'),
mtu=item.get('mtu'),
bond=None,
vlan_link=None,
vlan_id=None
)
return network_model.NetworkDetailsV2(
links=[link],
networks=self._parse_subnets(item.get("subnets"), link.name),
services=[]
)
def _parse_bond_config_item(self, item):
if not item.get('name'):
LOG.warning("Bond does not have a name.")
return
bond_params = item.get('params')
if not bond_params:
LOG.warning("Bond does not have parameters")
return
bond_mode = bond_params.get('bond-mode')
if bond_mode not in network_model.AVAILABLE_BOND_TYPES:
raise exception.CloudbaseInitException(
"Unsupported bond mode: %s" % bond_mode)
bond_lacp_rate = None
if bond_mode == network_model.BOND_TYPE_8023AD:
bond_lacp_rate = bond_params.get('bond-lacp-rate')
if (bond_lacp_rate and bond_lacp_rate not in
network_model.AVAILABLE_BOND_LACP_RATES):
raise exception.CloudbaseInitException(
"Unsupported bond lacp rate: %s" % bond_lacp_rate)
bond_xmit_hash_policy = bond_params.get('xmit_hash_policy')
if (bond_xmit_hash_policy and bond_xmit_hash_policy not in
network_model.AVAILABLE_BOND_LB_ALGORITHMS):
raise exception.CloudbaseInitException(
"Unsupported bond hash policy: %s" %
bond_xmit_hash_policy)
bond_interfaces = item.get('bond_interfaces')
bond = network_model.Bond(
members=bond_interfaces,
type=bond_mode,
lb_algorithm=bond_xmit_hash_policy,
lacp_rate=bond_lacp_rate,
)
link = network_model.Link(
id=item.get('name'),
name=item.get('name'),
type=network_model.LINK_TYPE_BOND,
enabled=True,
mac_address=item.get('mac_address'),
mtu=item.get('mtu'),
bond=bond,
vlan_link=None,
vlan_id=None
)
return network_model.NetworkDetailsV2(
links=[link],
networks=self._parse_subnets(item.get("subnets"), link.name),
services=[]
)
def _parse_vlan_config_item(self, item):
if not item.get('name'):
LOG.warning("VLAN NIC does not have a name.")
return
link = network_model.Link(
id=item.get('name'),
name=item.get('name'),
type=network_model.LINK_TYPE_VLAN,
enabled=True,
mac_address=item.get('mac_address'),
mtu=item.get('mtu'),
bond=None,
vlan_link=item.get('vlan_link'),
vlan_id=item.get('vlan_id')
)
return network_model.NetworkDetailsV2(
links=[link],
networks=self._parse_subnets(item.get("subnets"), link.name),
services=[]
)
def _parse_nameserver_config_item(self, item):
return network_model.NetworkDetailsV2(
links=[],
networks=[],
services=[network_model.NameServerService(
addresses=item.get('address', []),
search=item.get('search')
)]
)
def _get_network_config_parser(self, parser_type):
parsers = {
self.NETWORK_LINK_TYPE_PHY: self._parse_physical_config_item,
self.NETWORK_LINK_TYPE_BOND: self._parse_bond_config_item,
self.NETWORK_LINK_TYPE_VLAN: self._parse_vlan_config_item,
self.NETWORK_SERVICE_NAMESERVER: self._parse_nameserver_config_item
}
parser = parsers.get(parser_type)
if not parser:
raise exception.CloudbaseInitException(
"Network config parser '%s' does not exist",
parser_type)
return parser
def parse(self, network_config):
links = []
networks = []
services = []
if not network_config:
LOG.warning("Network configuration is empty")
return
if not isinstance(network_config, list):
LOG.warning("Network config '%s' is not a list.",
network_config)
return
for network_config_item in network_config:
if not isinstance(network_config_item, dict):
LOG.warning("Network config item '%s' is not a dictionary",
network_config_item)
continue
net_conf_type = network_config_item.get("type")
if net_conf_type not in self.SUPPORTED_NETWORK_CONFIG_TYPES:
LOG.warning("Network config type '%s' is not supported",
net_conf_type)
continue
net_details = (
self._get_network_config_parser(net_conf_type)
(network_config_item))
if net_details:
links += net_details.links
networks += net_details.networks
services += net_details.services
return network_model.NetworkDetailsV2(
links=links,
networks=networks,
services=services
)
class NoCloudConfigDriveService(baseconfigdrive.BaseConfigDriveService):
def __init__(self):
@ -43,7 +290,7 @@ class NoCloudConfigDriveService(baseconfigdrive.BaseConfigDriveService):
try:
self._meta_data = (
serialization.parse_json_yaml(raw_meta_data))
except base.YamlParserConfigError as ex:
except serialization.YamlParserConfigError as ex:
LOG.error("Metadata could not be parsed")
LOG.exception(ex)
@ -68,3 +315,30 @@ class NoCloudConfigDriveService(baseconfigdrive.BaseConfigDriveService):
return None
return debiface.parse(debian_net_config)
def get_network_details_v2(self):
try:
raw_network_data = self._get_cache_data("network-config",
decode=True)
network_data = serialization.parse_json_yaml(raw_network_data)
if not network_data:
LOG.info("V2 network metadata is empty")
return
if not isinstance(network_data, dict):
LOG.warning("V2 network metadata is not a dictionary")
return
except base.NotExistingMetadataException:
LOG.info("V2 network metadata not found")
return
except serialization.YamlParserConfigError:
LOG.exception("V2 network metadata could not be deserialized")
return
network_data_version = network_data.get("version")
if network_data_version != 1:
LOG.error("Network data version '%s' is not supported",
network_data_version)
return
network_config_parser = NoCloudNetworkConfigV1Parser()
return network_config_parser.parse(network_data.get("config"))

4
cloudbaseinit/models/network.py
View File

@ -54,6 +54,10 @@ AVAILABLE_BOND_LB_ALGORITHMS = [
BOND_LACP_RATE_SLOW = "slow"
BOND_LACP_RATE_FAST = "fast"
AVAILABLE_BOND_LACP_RATES = [
BOND_LACP_RATE_SLOW,
BOND_LACP_RATE_FAST
]
NetworkDetails = collections.namedtuple(
"NetworkDetails",

184
cloudbaseinit/tests/metadata/services/test_nocloudservice.py
View File

@ -12,7 +12,7 @@
# License for the specific language governing permissions and limitations
# under the License.
import ddt
import importlib
import os
import unittest
@ -22,11 +22,169 @@ try:
except ImportError:
import mock
from cloudbaseinit.metadata.services import base
from cloudbaseinit.models import network as nm
from cloudbaseinit.tests import testutils
from cloudbaseinit.utils import serialization
MODULE_PATH = "cloudbaseinit.metadata.services.nocloudservice"
@ddt.ddt
class TestNoCloudNetworkConfigV1Parser(unittest.TestCase):
def setUp(self):
module = importlib.import_module(MODULE_PATH)
self._parser = module.NoCloudNetworkConfigV1Parser()
self.snatcher = testutils.LogSnatcher(MODULE_PATH)
@ddt.data(('', ('Network configuration is empty', None)),
('{t: 1}',
("Network config '{'t': 1}' is not a list", None)),
('["1"]',
("Network config item '1' is not a dictionary",
nm.NetworkDetailsV2(links=[], networks=[], services=[]))),
('[{"type": "router"}]',
("Network config type 'router' is not supported",
nm.NetworkDetailsV2(links=[], networks=[], services=[]))))
@ddt.unpack
def test_parse_empty_result(self, input, expected_result):
with self.snatcher:
result = self._parser.parse(serialization.parse_json_yaml(input))
self.assertEqual(True, expected_result[0] in self.snatcher.output[0])
self.assertEqual(result, expected_result[1])
def test_network_details_v2(self):
expected_bond = nm.Bond(
members=["gbe0", "gbe1"],
type=nm.BOND_TYPE_ACTIVE_BACKUP,
lb_algorithm=None,
lacp_rate=None,
)
expected_link_bond = nm.Link(
id='bond0',
name='bond0',
type=nm.LINK_TYPE_BOND,
enabled=True,
mac_address="52:54:00:12:34:00",
mtu=1450,
bond=expected_bond,
vlan_link=None,
vlan_id=None,
)
expected_link = nm.Link(
id='interface0',
name='interface0',
type=nm.LINK_TYPE_PHYSICAL,
enabled=True,
mac_address="52:54:00:12:34:00",
mtu=1450,
bond=None,
vlan_link=None,
vlan_id=None,
)
expected_link_vlan = nm.Link(
id='vlan0',
name='vlan0',
type=nm.LINK_TYPE_VLAN,
enabled=True,
mac_address="52:54:00:12:34:00",
mtu=1450,
bond=None,
vlan_link='eth1',
vlan_id=150,
)
expected_network = nm.Network(
link='interface0',
address_cidr='192.168.1.10/24',
dns_nameservers=['192.168.1.11'],
routes=[
nm.Route(network_cidr='0.0.0.0/0',
gateway="192.168.1.1")
]
)
expected_network_bond = nm.Network(
link='bond0',
address_cidr='192.168.1.10/24',
dns_nameservers=['192.168.1.11'],
routes=[],
)
expected_network_vlan = nm.Network(
link='vlan0',
address_cidr='192.168.1.10/24',
dns_nameservers=['192.168.1.11'],
routes=[],
)
expected_nameservers = nm.NameServerService(
addresses=['192.168.23.2', '8.8.8.8'],
search='acme.local')
parser_data = """
- type: physical
name: interface0
mac_address: "52:54:00:12:34:00"
mtu: 1450
subnets:
- type: static
address: 192.168.1.10
netmask: 255.255.255.0
gateway: 192.168.1.1
dns_nameservers:
- 192.168.1.11
- type: bond
name: bond0
bond_interfaces:
- gbe0
- gbe1
mac_address: "52:54:00:12:34:00"
params:
bond-mode: active-backup
bond-lacp-rate: false
mtu: 1450
subnets:
- type: static
address: 192.168.1.10
netmask: 255.255.255.0
dns_nameservers:
- 192.168.1.11
- type: vlan
name: vlan0
vlan_link: eth1
vlan_id: 150
mac_address: "52:54:00:12:34:00"
mtu: 1450
subnets:
- type: static
address: 192.168.1.10
netmask: 255.255.255.0
dns_nameservers:
- 192.168.1.11
- type: nameserver
address:
- 192.168.23.2
- 8.8.8.8
search: acme.local
"""
result = self._parser.parse(
serialization.parse_json_yaml(parser_data))
self.assertEqual(result.links[0], expected_link)
self.assertEqual(result.networks[0], expected_network)
self.assertEqual(result.links[1], expected_link_bond)
self.assertEqual(result.networks[1], expected_network_bond)
self.assertEqual(result.links[2], expected_link_vlan)
self.assertEqual(result.networks[2], expected_network_vlan)
self.assertEqual(result.services[0], expected_nameservers)
@ddt.ddt
class TestNoCloudConfigDriveService(unittest.TestCase):
def setUp(self):
@ -88,3 +246,27 @@ class TestNoCloudConfigDriveService(unittest.TestCase):
}
result = self._config_drive.get_public_keys()
self.assertEqual(result, expected_result)
@ddt.data(('', ('V2 network metadata is empty', None)),
('1', ('V2 network metadata is not a dictionary', None)),
('{}', ('V2 network metadata is empty', None)),
('{}}', ('V2 network metadata could not be deserialized', None)),
('{version: 2}', ("Network data version '2' is not supported",
None)),
(base.NotExistingMetadataException('exc'),
('V2 network metadata not found', True)))
@ddt.unpack
@mock.patch(MODULE_PATH + '.NoCloudConfigDriveService._get_cache_data')
def test_network_details_v2_empty_result(self, input, expected_result,
mock_get_cache_data):
if expected_result[1]:
mock_get_cache_data.side_effect = [input]
else:
mock_get_cache_data.return_value = input
with self.snatcher:
result = self._config_drive.get_network_details_v2()
self.assertEqual(True, expected_result[0] in self.snatcher.output[0])
self.assertEqual(result, None)
mock_get_cache_data.assert_called_with(
"network-config", decode=True)

65
doc/source/services.rst
View File

@ -138,7 +138,9 @@ Capabilities:
* instance id
* hostname
* public keys
* static network configuration (Debian format)
* static network configuration (Debian and `network config v1
<https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v1.html>`_
formats)
* user data
Config options for `config_drive` section:
@ -164,6 +166,67 @@ Example metadata:
hwaddress ether 00:11:22:33:44:55
hostname: windowshost1
Cloud-init's `network config v1
<https://cloudinit.readthedocs.io/en/latest/topics/network-config-format-v1.html>`_
format can be used to configure static network configuration.
The configuration file should be named `network-config` and should be present
at the same folder level with the `meta-data` and `user-data` file.
If no `network-config` is found, cloudbase-init will use the `network-interfaces`
value from the metadata (if any).
The following network config types are implemented: physical, bond, vlan and
nameserver.
Unsupported config types: bridge and route.
Example:
.. code-block:: yaml
version: 1
config:
- type: physical
name: interface0
mac_address: "52:54:00:12:34:00"
mtu: 1450
subnets:
- type: static
address: 192.168.1.10
netmask: 255.255.255.0
dns_nameservers:
- 192.168.1.11
- type: bond
name: bond0
bond_interfaces:
- gbe0
- gbe1
mac_address: "52:54:00:12:34:00"
params:
bond-mode: active-backup
bond-lacp-rate: false
mtu: 1450
subnets:
- type: static
address: 192.168.1.10
netmask: 255.255.255.0
dns_nameservers:
- 192.168.1.11
- type: vlan
name: vlan0
vlan_link: eth1
vlan_id: 150
mac_address: "52:54:00:12:34:00"
mtu: 1450
subnets:
- type: static
address: 192.168.1.10
netmask: 255.255.255.0
dns_nameservers:
- 192.168.1.11
- type: nameserver
address:
- 192.168.23.2
- 8.8.8.8
search: acme.local
More information on the NoCloud metadata service specifications can be found
`here <https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html>`_.