zuul/nodepool
Code Issues Proposed changes

Implement an Azure driver

Browse Source 
This change adds an Azure driver.

Supports:
    * Public IPv4 address per VM
    * Private IPv6 address per VM (optional, and not useful yet)
    * Standard Flavors
    * Resource Tagging (for billing / cleanup)

Change-Id: Ief0f8574832df69db472d8704ea3710bc6ca5c59
Co-authored-by: Tristan Cacqueray <tdecacqu@redhat.com>
Co-authored-by: Tobias Henkel <tobias.henkel@bmw.de>
Signed-off-by: Graham Hayes <gr@ham.ie>
This commit is contained in:
Graham Hayes 2019-07-01 19:44:00 +01:00
parent 1845788a95
commit c1a914fa4a
8 changed files with 1215 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

222
doc/source/configuration.rst
View File

@ -472,6 +472,12 @@ Options
static driver, see the separate section static driver, see the separate section
:attr:`providers.[static]` :attr:`providers.[static]`
.. value:: azure
For details on the extra options required and provided by the
Azure driver, see the separate section
:attr:`providers.[azure]`
OpenStack Driver OpenStack Driver
---------------- ----------------
@ -2203,3 +2209,219 @@ section of the configuration.
.. _`Application Default Credentials`: https://cloud.google.com/docs/authentication/production .. _`Application Default Credentials`: https://cloud.google.com/docs/authentication/production
.. _`GCE regions and zones`: https://cloud.google.com/compute/docs/regions-zones/ .. _`GCE regions and zones`: https://cloud.google.com/compute/docs/regions-zones/
.. _`GCE machine types`: https://cloud.google.com/compute/docs/machine-types .. _`GCE machine types`: https://cloud.google.com/compute/docs/machine-types
Azure Compute Driver
--------------------
Selecting the azure driver adds the following options to the :attr:`providers`
section of the configuration.
.. attr-overview::
:prefix: providers.[azure]
:maxdepth: 3
.. attr:: providers.[azure]
:type: list
An Azure provider's resources are partitioned into groups called `pool`,
and within a pool, the node types which are to be made available are listed
.. note:: For documentation purposes the option names are prefixed
``providers.[azure]`` to disambiguate from other
drivers, but ``[azure]`` is not required in the
configuration (e.g. below
``providers.[azure].pools`` refers to the ``pools``
key in the ``providers`` section when the ``azure``
driver is selected).
Example:
.. code-block:: yaml
providers:
- name: azure-central-us
driver: azure
zuul-public-key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAA...
resource-group-location: centralus
location: centralus
resource-group: ZuulCIDev
auth-path: /Users/grhayes/.azure/nodepoolCreds.json
subnet-id: /subscriptions/<subscription-id>/resourceGroups/ZuulCI/providers/Microsoft.Network/virtualNetworks/NodePool/subnets/default
cloud-images:
- name: bionic
username: zuul
image-reference:
sku: 18.04-LTS
publisher: Canonical
version: latest
offer: UbuntuServer
pools:
- name: main
max-servers: 10
labels:
- name: bionic
cloud-image: bionic
hardware-profile:
vm-size: Standard_D1_v2
tags:
department: R&D
purpose: CI/CD
.. attr:: name
:required:
A unique name for this provider configuration.
.. attr:: location
:required:
Name of the Azure region to interact with.
.. attr:: resource-group-location
:required:
Name of the Azure region to where the home Resource Group is or should be created.
.. attr:: auth-path
:required:
Path to the JSON file containing the service principal credentials.
Create with the `Azure CLI`_ and the ``--sdk-auth`` flag
.. attr:: subnet-id
:required:
Subnet to create VMs on
.. attr:: cloud-images
:type: list
Each entry in this section must refer to an entry in the
:attr:`labels` section.
.. code-block:: yaml
cloud-images:
- name: bionic
username: zuul
image-reference:
sku: 18.04-LTS
publisher: Canonical
version: latest
offer: UbuntuServer
- name: windows-server-2016
username: zuul
image-reference:
sku: 2016-Datacenter
publisher: MicrosoftWindowsServer
version: latest
offer: WindowsServer
Each entry is a dictionary with the following keys
.. attr:: name
:type: string
:required:
Identifier to refer this cloud-image from :attr:`labels`
section. Since this name appears elsewhere in the nodepool
configuration file, you may want to use your own descriptive
name here.
.. attr:: username
:type: str
The username that a consumer should use when connecting to the
node.
.. attr:: image-reference
:type: dict
:required:
.. attr:: sku
:type: str
:required:
Image SKU
.. attr:: publisher
:type: str
:required:
Image Publisher
.. attr:: offer
:type: str
:required:
Image offers
.. attr:: version
:type: str
:required:
Image version
.. attr:: pools
:type: list
A pool defines a group of resources from an Azure provider. Each pool has a
maximum number of nodes which can be launched from it, along with a number
of cloud-related attributes used when launching nodes.
.. attr:: name
:required:
A unique name within the provider for this pool of resources.
.. attr:: labels
:type: list
Each entry in a pool's `labels` section indicates that the
corresponding label is available for use in this pool. When creating
nodes for a label, the flavor-related attributes in that label's
section will be used.
.. code-block:: yaml
labels:
- name: bionic
cloud-image: bionic
hardware-profile:
vm-size: Standard_D1_v2
Each entry is a dictionary with the following keys
.. attr:: name
:type: str
:required:
Identifier to refer this label.
.. attr:: cloud-image
:type: str
:required:
Refers to the name of an externally managed image in the
cloud that already exists on the provider. The value of
``cloud-image`` should match the ``name`` of a previously
configured entry from the ``cloud-images`` section of the
provider.
.. attr:: hardware-profile
:required:
.. attr:: vm-size
:required:
:type: str
VM Size of the VMs to use in Azure. See the VM size list on `azure.microsoft.com`_
for the list of sizes availabile in each region.
.. _`Azure CLI`: https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli?view=azure-cli-latest
.. _azure.microsoft.com: https://azure.microsoft.com/en-us/global-infrastructure/services/?products=virtual-machines

27
nodepool/driver/azure/__init__.py Normal file
View File

@ -0,0 +1,27 @@
# Copyright 2018 Red Hat
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
#
# See the License for the specific language governing permissions and
# limitations under the License.
from nodepool.driver import Driver
from nodepool.driver.azure.config import AzureProviderConfig
from nodepool.driver.azure.provider import AzureProvider
class AzureDriver(Driver):
def getProviderConfig(self, provider):
return AzureProviderConfig(self, provider)
def getProvider(self, provider_config):
return AzureProvider(provider_config)

169
nodepool/driver/azure/config.py Normal file
View File

@ -0,0 +1,169 @@
# Copyright 2018 Red Hat
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
#
# See the License for the specific language governing permissions and
# limitations under the License.
import voluptuous as v
import os
from nodepool.driver import ConfigPool
from nodepool.driver import ConfigValue
from nodepool.driver import ProviderConfig
class AzureLabel(ConfigValue):
def __eq__(self, other):
if (other.username != self.username or
other.imageReference != self.imageReference or
other.hardwareProfile != self.hardwareProfile):
return False
return True
class AzurePool(ConfigPool):
def __eq__(self, other):
if other.labels != self.labels:
return False
return True
def __repr__(self):
return "<AzurePool %s>" % self.name
def load(self, pool_config):
pass
class AzureProviderConfig(ProviderConfig):
def __init__(self, driver, provider):
self._pools = {}
self.driver_object = driver
super().__init__(provider)
def __eq__(self, other):
if (other.location != self.location or
other.pools != self.pools):
return False
return True
@property
def pools(self):
return self._pools
@property
def manage_images(self):
return False
@staticmethod
def reset():
pass
def load(self, config):
self.zuul_public_key = self.provider['zuul-public-key']
self.location = self.provider['location']
self.subnet_id = self.provider['subnet-id']
self.ipv6 = self.provider.get('ipv6', False)
self.resource_group = self.provider['resource-group']
self.resource_group_location = self.provider['resource-group-location']
self.auth_path = self.provider.get(
'auth-path', os.getenv('AZURE_AUTH_LOCATION', None))
self.cloud_images = {}
for image in self.provider['cloud-images']:
self.cloud_images[image['name']] = image
for pool in self.provider.get('pools', []):
pp = AzurePool()
pp.name = pool['name']
pp.provider = self
pp.max_servers = pool['max-servers']
self._pools[pp.name] = pp
pp.labels = {}
for label in pool.get('labels', []):
pl = AzureLabel()
pl.name = label['name']
pl.pool = pp
pp.labels[pl.name] = pl
cloud_image_name = label['cloud-image']
if cloud_image_name:
cloud_image = self.cloud_images.get(
cloud_image_name, None)
if not cloud_image:
raise ValueError(
"cloud-image %s does not exist in provider %s"
" but is referenced in label %s" %
(cloud_image_name, self.name, pl.name))
pl.imageReference = cloud_image['image-reference']
pl.username = cloud_image.get('username', 'zuul')
else:
pl.imageReference = None
pl.username = 'zuul'
pl.hardwareProfile = label['hardware-profile']
config.labels[label['name']].pools.append(pp)
pl.tags = label['tags']
def getSchema(self):
azure_image_reference = {
v.Required('sku'): str,
v.Required('publisher'): str,
v.Required('version'): str,
v.Required('offer'): str,
}
azure_hardware_profile = {
v.Required('vm-size'): str,
}
provider_cloud_images = {
v.Required('name'): str,
'username': str,
v.Required('image-reference'): azure_image_reference,
}
azure_label = {
v.Required('name'): str,
v.Required('hardware-profile'): azure_hardware_profile,
v.Required('cloud-image'): str,
v.Optional('tags'): dict,
}
pool = ConfigPool.getCommonSchemaDict()
pool.update({
v.Required('name'): str,
v.Required('labels'): [azure_label],
})
provider = ProviderConfig.getCommonSchemaDict()
provider.update({
v.Required('zuul-public-key'): str,
v.Required('pools'): [pool],
v.Required('location'): str,
v.Required('resource-group'): str,
v.Required('resource-group-location'): str,
v.Required('subnet-id'): str,
v.Required('cloud-images'): [provider_cloud_images],
v.Optional('auth-path'): str,
})
return v.Schema(provider)
def getSupportedLabels(self, pool_name=None):
labels = set()
for pool in self._pools.values():
if not pool_name or (pool.name == pool_name):
labels.update(pool.labels.keys())
return labels

148
nodepool/driver/azure/handler.py Normal file
View File

@ -0,0 +1,148 @@
# Copyright 2018 Red Hat
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
import time
from nodepool import exceptions
from nodepool import zk
from nodepool.driver.utils import NodeLauncher
from nodepool.driver import NodeRequestHandler
from nodepool import nodeutils as utils
class AzureInstanceLauncher(NodeLauncher):
def __init__(
self, handler, node, provider_config,
label, retries=3, boot_timeout=120):
super().__init__(handler, node, provider_config)
self.retries = retries
self.handler = handler
self.label = label
self.boot_timeout = boot_timeout
self.zk = handler.zk
def launch(self):
self.log.debug("Starting %s instance" % self.node.type)
attempts = 1
hostname = '{label.name}-{provider.name}-{node.id}'.format(
label=self.label, provider=self.provider_config, node=self.node
)
while attempts <= self.retries:
try:
instance = self.handler.manager.createInstance(
hostname, self.label, self.node.id,
nodepool_node_label=self.label.name)
break
except Exception:
if attempts <= self.retries:
self.log.exception(
"Launch attempt %d/%d failed for node %s:",
attempts, self.retries, self.node.id)
else:
raise
attempts += 1
time.sleep(1)
self.node.external_id = instance.id
boot_start = time.monotonic()
while time.monotonic() - boot_start < self.boot_timeout:
state = instance.provisioning_state
self.log.debug("Instance %s is %s" % (instance.id, state))
if state == 'Succeeded':
break
time.sleep(0.5)
instance = self.handler.manager.getInstance(instance.id)
if state != 'Succeeded':
raise exceptions.LaunchStatusException(
"Instance %s failed to start: %s" % (instance.id, state))
server_ip = self.handler.manager.getIpaddress(instance)
if self.provider_config.ipv6:
server_v6_ip = self.handler.manager.getv6Ipaddress(instance)
if not server_ip:
raise exceptions.LaunchStatusException(
"Instance %s doesn't have a public ip" % instance.id)
try:
key = utils.nodescan(server_ip, port=22, timeout=180)
except Exception:
raise exceptions.LaunchKeyscanException(
"Can't scan instance %s key" % instance.id)
self.log.info("Instance %s ready" % instance.id)
self.node.state = zk.READY
self.node.external_id = instance.id
self.node.hostname = server_ip
self.node.interface_ip = server_ip
self.node.public_ipv4 = server_ip
if self.provider_config.ipv6:
self.node.public_ipv6 = server_v6_ip
self.node.host_keys = key
self.node.connection_port = 22
self.node.connection_type = "ssh"
self.node.username = self.label.username
self.zk.storeNode(self.node)
self.log.info("Instance %s is ready", instance.id)
class AzureNodeRequestHandler(NodeRequestHandler):
log = logging.getLogger("nodepool.driver.azure."
"AzureNodeRequestHandler")
def __init__(self, pw, request):
super().__init__(pw, request)
self._threads = []
@property
def alive_thread_count(self):
count = 0
for t in self._threads:
if t.is_alive():
count += 1
return count
def launchesComplete(self):
'''
Check if all launch requests have completed.
When all of the Node objects have reached a final state (READY or
FAILED), we'll know all threads have finished the launch process.
'''
if not self._threads:
return True
# Give the NodeLaunch threads time to finish.
if self.alive_thread_count:
return False
node_states = [node.state for node in self.nodeset]
# NOTE: It very important that NodeLauncher always sets one of
# these states, no matter what.
if not all(s in (zk.READY, zk.FAILED) for s in node_states):
return False
return True
def launch(self, node):
label = self.pool.labels[node.type[0]]
thd = AzureInstanceLauncher(self, node, self.provider, label)
thd.start()
self._threads.append(thd)
def imagesAvailable(self):
return True

413
nodepool/driver/azure/provider.py Normal file
View File

@ -0,0 +1,413 @@
# Copyright 2018 Red Hat
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import logging
from azure.common.client_factory import get_client_from_auth_file
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.network import NetworkManagementClient
from azure.mgmt.compute import ComputeManagementClient
from msrestazure.azure_exceptions import CloudError
from nodepool.driver import Provider
from nodepool.driver.azure import handler
from nodepool import zk
class AzureProvider(Provider):
log = logging.getLogger("nodepool.driver.azure.AzureProvider")
API_VERSION_COMPUTE = "2019-12-01"
API_VERSION_DISKS = "2019-11-01"
API_VERSION_NETWORK = "2020-03-01"
API_VERSION_RESOURCE = "2019-10-01"
def __init__(self, provider, *args):
self.provider = provider
self.zuul_public_key = provider.zuul_public_key
self.compute_client = None
self.disks_client = None
self.network_client = None
self.resource_client = None
self.resource_group = provider.resource_group
self.resource_group_location = provider.resource_group_location
self._zk = None
def start(self, zk_conn):
self.log.debug("Starting")
self._zk = zk_conn
self.log.debug(
"Using %s as auth_path for Azure auth" % self.provider.auth_path)
if self.compute_client is None:
self.compute_client = self._get_compute_client()
if self.disks_client is None:
self.disks_client = self._get_disks_client()
if self.network_client is None:
self.network_client = self._get_network_client()
if self.resource_client is None:
self.resource_client = self._get_resource_client()
def _get_compute_client(self):
return get_client_from_auth_file(
ComputeManagementClient,
auth_path=self.provider.auth_path,
api_version=self.API_VERSION_COMPUTE
)
def _get_disks_client(self):
return get_client_from_auth_file(
ComputeManagementClient,
auth_path=self.provider.auth_path,
api_version=self.API_VERSION_DISKS
)
def _get_network_client(self):
return get_client_from_auth_file(
NetworkManagementClient,
auth_path=self.provider.auth_path,
api_version=self.API_VERSION_NETWORK
)
def _get_resource_client(self):
return get_client_from_auth_file(
ResourceManagementClient,
auth_path=self.provider.auth_path,
api_version=self.API_VERSION_RESOURCE
)
def stop(self):
self.log.debug("Stopping")
def listNodes(self):
return self.compute_client.virtual_machines.list(self.resource_group)
def listNICs(self):
return self.network_client.network_interfaces.list(self.resource_group)
def listPIPs(self):
return self.network_client.public_ip_addresses.list(
self.resource_group)
def listDisks(self):
return self.disks_client.disks.list_by_resource_group(
self.resource_group)
def labelReady(self, name):
return True
def join(self):
return True
def getRequestHandler(self, poolworker, request):
return handler.AzureNodeRequestHandler(poolworker, request)
def cleanupLeakedResources(self):
self._cleanupLeakedNodes()
self._cleanupLeakedNICs()
self._cleanupLeakedPIPs()
self._cleanupLeakedDisks()
def _cleanupLeakedDisks(self):
for disk in self.listDisks():
if disk.tags is None:
# Nothing to check ownership against, move on
continue
if 'nodepool_provider_name' not in disk.tags:
continue
if disk.tags['nodepool_provider_name'] != self.provider.name:
# Another launcher, sharing this provider but configured
# with a different name, owns this.
continue
if not self._zk.getNode(disk.tags['nodepool_id']):
self.log.warning(
"Marking for delete leaked Disk %s (%s) in %s "
"(unknown node id %s)",
disk.name, disk.id, self.provider.name,
disk.tags['nodepool_id']
)
try:
self.disks_client.disks.delete(
self.resource_group,
disk.name).wait()
except CloudError as e:
self.log.warning(
"Failed to cleanup Disk %s (%s). Error: %r",
disk.name, disk.id, e
)
def _cleanupLeakedNICs(self):
for nic in self.listNICs():
if nic.tags is None:
# Nothing to check ownership against, move on
continue
if 'nodepool_provider_name' not in nic.tags:
continue
if nic.tags['nodepool_provider_name'] != self.provider.name:
# Another launcher, sharing this provider but configured
# with a different name, owns this.
continue
if not self._zk.getNode(nic.tags['nodepool_id']):
self.log.warning(
"Marking for delete leaked NIC %s (%s) in %s "
"(unknown node id %s)",
nic.name, nic.id, self.provider.name,
nic.tags['nodepool_id']
)
try:
self.network_client.network_interfaces.delete(
self.resource_group,
nic.name).wait()
except CloudError as e:
self.log.warning(
"Failed to cleanup NIC %s (%s). Error: %r",
nic.name, nic.id, e
)
def _cleanupLeakedPIPs(self):
for pip in self.listPIPs():
if pip.tags is None:
# Nothing to check ownership against, move on
continue
if 'nodepool_provider_name' not in pip.tags:
continue
if pip.tags['nodepool_provider_name'] != self.provider.name:
# Another launcher, sharing this provider but configured
# with a different name, owns this.
continue
if not self._zk.getNode(pip.tags['nodepool_id']):
self.log.warning(
"Marking for delete leaked PIP %s (%s) in %s "
"(unknown node id %s)",
pip.name, pip.id, self.provider.name,
pip.tags['nodepool_id']
)
try:
self.network_client.public_ip_addresses.delete(
self.resource_group,
pip.name).wait()
except CloudError as e:
self.log.warning(
"Failed to cleanup IP %s (%s). Error: %r",
pip.name, pip.id, e
)
def _cleanupLeakedNodes(self):
deleting_nodes = {}
for node in self._zk.nodeIterator():
if node.state == zk.DELETING:
if node.provider != self.provider.name:
continue
if node.provider not in deleting_nodes:
deleting_nodes[node.provider] = []
deleting_nodes[node.provider].append(node.external_id)
for n in self.listNodes():
if n.tags is None:
# Nothing to check ownership against, move on
continue
if 'nodepool_provider_name' not in n.tags:
continue
if n.tags['nodepool_provider_name'] != self.provider.name:
# Another launcher, sharing this provider but configured
# with a different name, owns this.
continue
if (self.provider.name in deleting_nodes and
n.id in deleting_nodes[self.provider.name]):
# Already deleting this node
continue
if not self._zk.getNode(n.tags['nodepool_id']):
self.log.warning(
"Marking for delete leaked instance %s (%s) in %s "
"(unknown node id %s)",
n.name, n.id, self.provider.name,
n.tags['nodepool_id']
)
node = zk.Node()
node.external_id = n.id
node.provider = self.provider.name
node.state = zk.DELETING
self._zk.storeNode(node)
def cleanupNode(self, server_id):
self.log.debug('Server ID: %s' % server_id)
try:
vm = self.compute_client.virtual_machines.get(
self.resource_group, server_id.rsplit('/', 1)[1])
except CloudError as e:
if e.status_code == 404:
return
self.log.warning(
"Failed to cleanup node %s. Error: %r",
server_id, e
)
self.compute_client.virtual_machines.delete(
self.resource_group, server_id.rsplit('/', 1)[1]).wait()
nic_deletion = self.network_client.network_interfaces.delete(
self.resource_group, "%s-nic" % server_id.rsplit('/', 1)[1])
nic_deletion.wait()
pip_deletion = self.network_client.public_ip_addresses.delete(
self.resource_group, "%s-nic-pip" % server_id.rsplit('/', 1)[1])
pip_deletion.wait()
if self.provider.ipv6:
pip_deletion = self.network_client.public_ip_addresses.delete(
self.resource_group,
"%s-nic-v6-pip" % server_id.rsplit('/', 1)[1])
pip_deletion.wait()
disk_handle_list = []
for disk in self.listDisks():
if disk.tags is not None and \
disk.tags.get('nodepool_id') == vm.tags['nodepool_id']:
async_disk_delete = self.disks_client.disks.delete(
self.resource_group, disk.name)
disk_handle_list.append(async_disk_delete)
for async_disk_delete in disk_handle_list:
async_disk_delete.wait()
def waitForNodeCleanup(self, server_id):
# All async tasks are handled in cleanupNode
return True
def getInstance(self, server_id):
return self.compute_client.virtual_machines.get(
self.resource_group, server_id, expand='instanceView')
def createInstance(
self, hostname, label, nodepool_id, nodepool_node_label=None):
self.log.debug("Create resouce group")
tags = label.tags or {}
tags['nodepool_provider_name'] = self.provider.name
if nodepool_node_label:
tags['nodepool_node_label'] = nodepool_node_label
self.resource_client.resource_groups.create_or_update(
self.resource_group, {
'location': self.provider.resource_group_location,
'tags': tags
})
tags['nodepool_id'] = nodepool_id
v4_params_create = {
'location': self.provider.location,
'public_ip_allocation_method': 'dynamic',
'tags': tags,
}
v4_pip_poll = self.network_client.public_ip_addresses.create_or_update(
self.resource_group,
"%s-nic-pip" % hostname,
v4_params_create,
)
v4_public_ip = v4_pip_poll.result()
nic_data = {
'location': self.provider.location,
'tags': tags,
'ip_configurations': [{
'name': "zuul-v4-ip-config",
'private_ip_address_version': 'IPv4',
'subnet': {
'id': self.provider.subnet_id
},
'public_ip_address': {
'id': v4_public_ip.id
}
}]
}
if self.provider.ipv6:
nic_data['ip_configurations'].append({
'name': "zuul-v6-ip-config",
'private_ip_address_version': 'IPv6',
'subnet': {
'id': self.provider.subnet_id
}
})
nic_creation = self.network_client.network_interfaces.create_or_update(
self.resource_group,
"%s-nic" % hostname,
nic_data
)
nic = nic_creation.result()
vm_creation = self.compute_client.virtual_machines.create_or_update(
self.resource_group, hostname, {
'location': self.provider.location,
'os_profile': {
'computer_name': hostname,
'admin_username': label.username,
'linux_configuration': {
'ssh': {
'public_keys': [{
'path': "/home/%s/.ssh/authorized_keys" % (
label.username),
'key_data': self.provider.zuul_public_key,
}]
},
"disable_password_authentication": True,
}
},
'hardware_profile': {
'vmSize': label.hardwareProfile["vm-size"]
},
'storage_profile': {'image_reference': label.imageReference},
'network_profile': {
'network_interfaces': [{
'id': nic.id,
'properties': {
'primary': True,
}
}]
},
'tags': tags,
})
return vm_creation.result()
def getIpaddress(self, instance):
# Copied from https://github.com/Azure/azure-sdk-for-python/issues/897
ni_reference = instance.network_profile.network_interfaces[0]
ni_reference = ni_reference.id.split('/')
ni_group = ni_reference[4]
ni_name = ni_reference[8]
net_interface = self.network_client.network_interfaces.get(
ni_group, ni_name)
ip_reference = net_interface.ip_configurations[0].public_ip_address
ip_reference = ip_reference.id.split('/')
ip_group = ip_reference[4]
ip_name = ip_reference[8]
public_ip = self.network_client.public_ip_addresses.get(
ip_group, ip_name)
public_ip = public_ip.ip_address
return public_ip
def getv6Ipaddress(self, instance):
# Copied from https://github.com/Azure/azure-sdk-for-python/issues/897
ni_reference = instance.network_profile.network_interfaces[0]
ni_reference = ni_reference.id.split('/')
ni_group = ni_reference[4]
ni_name = ni_reference[8]
net_interface = self.network_client.network_interfaces.get(
ni_group, ni_name)
return net_interface.ip_configurations[1].private_ip_address

41
nodepool/tests/fixtures/azure.yaml vendored Normal file
View File

@ -0,0 +1,41 @@
webapp:
port: 8005
listen_address: '0.0.0.0'
zookeeper-servers:
- host: 127.0.0.1
port: 2181
labels:
- name: bionic
min-ready: 1
providers:
- name: azure
driver: azure
zuul-public-key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+mplenM+m6pNY9Un3fpO9eqf808Jrfb3d1gXg7BZVawCvtEZ/cDYvLQ3OF1AeL2kcIC0UAIglM5JXae7yO5CJbJRdkbXvv0u1LvpLxYSPM4ATR0r4IseC5YVxkfJQNi4ixSwTqD4ScEkuCXcSqSU9M+hB+KlnwXoR4IcYHf7vD2Z0Mdwm2ikk3SeERmspmMxx/uz0SPn58QxONuoTlNWQKqDWsV6bRyoPa6HWccMrIH1/e7E69Nw/30oioOQpKBgaDCauh+QkDtSkjRpRMOV47ZFh16Q9DqMgLx+FD8z6++9rsHlB65Zas1xyQsiRCFG09s00b7OR7Xz9ukQ5+vXV
resource-group-location: centralus
location: centralus
resource-group: ZuulCI
auth-path: /etc/nodepool/azurecredentials.json
subnet-id: /subscriptions/c35cf7df-ed75-4c85-be00-535409a85120/resourceGroups/ZuulCI/providers/Microsoft.Network/virtualNetworks/NodePool/subnets/default
cloud-images:
- name: bionic
username: zuul
image-reference:
sku: 18.04-LTS
publisher: Canonical
version: latest
offer: UbuntuServer
pools:
- name: main
max-servers: 10
labels:
- name: bionic
cloud-image: bionic
hardware-profile:
vm-size: Standard_D1_v2
tags:
department: R&D
team: DevOps
systemPurpose: CI

192
nodepool/tests/unit/test_driver_azure.py Normal file
View File

@ -0,0 +1,192 @@
# Copyright (C) 2018 Red Hat
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import fixtures
import logging
import os
import tempfile
from unittest.mock import MagicMock
import yaml
from nodepool import tests
from nodepool import zk
from nodepool import nodeutils as utils
from nodepool.driver.azure import provider, AzureProvider
from azure.common.client_factory import get_client_from_json_dict
from azure.mgmt.resource.resources.v2019_10_01.operations import ResourceGroupsOperations # noqa
from azure.mgmt.network.v2020_03_01.operations import PublicIPAddressesOperations # noqa
from azure.mgmt.network.v2020_03_01.operations import NetworkInterfacesOperations # noqa
from azure.mgmt.compute.v2019_12_01.operations import VirtualMachinesOperations
from azure.mgmt.resource import ResourceManagementClient
from azure.mgmt.network import NetworkManagementClient
from azure.mgmt.compute import ComputeManagementClient
auth = {
"clientId": "ad735158-65ca-11e7-ba4d-ecb1d756380e",
"clientSecret": "b70bb224-65ca-11e7-810c-ecb1d756380e",
"subscriptionId": "bfc42d3a-65ca-11e7-95cf-ecb1d756380e",
"tenantId": "c81da1d8-65ca-11e7-b1d1-ecb1d756380e",
"activeDirectoryEndpointUrl": "https://login.microsoftonline.com",
"resourceManagerEndpointUrl": "https://management.azure.com/",
"activeDirectoryGraphResourceId": "https://graph.windows.net/",
"sqlManagementEndpointUrl": "https://management.core.windows.net:8443/",
"galleryEndpointUrl": "https://gallery.azure.com/",
"managementEndpointUrl": "https://management.core.windows.net/",
}
class FakeAzureResource:
def __init__(self, id_, provisioning_state='Unknown'):
self.id = id_
self.provisioning_state = provisioning_state
class FakePIPResult:
@staticmethod
def result():
return FakeAzureResource('fake_pip_id')
class FakeNICResult:
@staticmethod
def result():
return FakeAzureResource('fake_nic_id')
class FakeVMResult:
@staticmethod
def result():
return FakeAzureResource('fake_vm_id', provisioning_state='Succeeded')
class TestDriverAzure(tests.DBTestCase):
log = logging.getLogger("nodepool.TestDriverAzure")
def setUp(self):
super().setUp()
self.useFixture(fixtures.MockPatchObject(
provider.AzureProvider, 'cleanupLeakedResources',
MagicMock()))
self.useFixture(fixtures.MockPatchObject(
provider.AzureProvider, 'cleanupNode',
MagicMock()))
self.useFixture(fixtures.MockPatchObject(
provider.AzureProvider, 'getIpaddress',
MagicMock(return_value="127.0.0.1")))
self.useFixture(fixtures.MockPatchObject(
provider.AzureProvider, '_get_compute_client',
MagicMock(
return_value=get_client_from_json_dict(
ComputeManagementClient, auth, credentials={},
api_version=AzureProvider.API_VERSION_COMPUTE
)
)
))
self.useFixture(fixtures.MockPatchObject(
provider.AzureProvider, '_get_disks_client',
MagicMock(
return_value=get_client_from_json_dict(
ComputeManagementClient, auth, credentials={},
api_version=AzureProvider.API_VERSION_DISKS
)
)
))
self.useFixture(fixtures.MockPatchObject(
utils, 'nodescan',
MagicMock(return_value="FAKE_KEY")))
self.useFixture(fixtures.MockPatchObject(
provider.AzureProvider, '_get_network_client',
MagicMock(
return_value=get_client_from_json_dict(
NetworkManagementClient, auth, credentials={},
api_version=AzureProvider.API_VERSION_NETWORK
)
)
))
self.useFixture(fixtures.MockPatchObject(
provider.AzureProvider, '_get_resource_client',
MagicMock(
return_value=get_client_from_json_dict(
ResourceManagementClient, auth, credentials={},
api_version=AzureProvider.API_VERSION_RESOURCE
)
)
))
self.useFixture(fixtures.MockPatchObject(
ResourceGroupsOperations, 'create_or_update',
MagicMock(
return_value=FakeAzureResource('fake_rg_id'))
))
self.useFixture(fixtures.MockPatchObject(
PublicIPAddressesOperations, 'create_or_update',
MagicMock(return_value=FakePIPResult())
))
self.useFixture(fixtures.MockPatchObject(
NetworkInterfacesOperations, 'create_or_update',
MagicMock(return_value=FakeNICResult())
))
self.useFixture(fixtures.MockPatchObject(
VirtualMachinesOperations, 'create_or_update',
MagicMock(return_value=FakeVMResult())
))
def test_azure_machine(self):
az_template = os.path.join(
os.path.dirname(__file__), '..', 'fixtures', 'azure.yaml')
with open(az_template) as f:
raw_config = yaml.safe_load(f)
raw_config['zookeeper-servers'][0] = {
'host': self.zookeeper_host,
'port': self.zookeeper_port,
'chroot': self.zookeeper_chroot,
}
with tempfile.NamedTemporaryFile() as tf:
tf.write(yaml.safe_dump(
raw_config, default_flow_style=False).encode('utf-8'))
tf.flush()
configfile = self.setup_config(tf.name)
pool = self.useNodepool(configfile, watermark_sleep=1)
pool.start()
req = zk.NodeRequest()
req.state = zk.REQUESTED
req.node_types.append('bionic')
self.zk.storeNodeRequest(req)
req = self.waitForNodeRequest(req)
self.assertEqual(req.state, zk.FULFILLED)
self.assertNotEqual(req.nodes, [])
node = self.zk.getNode(req.nodes[0])
self.assertEqual(node.allocated_to, req.id)
self.assertEqual(node.state, zk.READY)
self.assertIsNotNone(node.launcher)
self.assertEqual(node.connection_type, 'ssh')

3
requirements.txt
View File

@ -18,3 +18,6 @@ WebOb>=1.8.1
openshift<=0.8.9 openshift<=0.8.9
boto3 boto3
google-api-python-client google-api-python-client
azure-mgmt-compute
azure-mgmt-network
azure-mgmt-resource