zuul/nodepool

Zuul ff7dd84aef Merge "Add provider/pool priority support"

2022-05-24 16:02:34 +00:00

22 KiB

Raw Blame History

zuul

IBM VPC Driver

This driver is for use with the Virtual Private Cloud (VPC) service of IBM Cloud.

Preparation

Note the following caveats:

IBM Cloud VPC does not support IPv6 addressing.
IBM Cloud VPC does not supply quota information via its API (but if you know the values, you can supply them in the Nodepool configuration).
Per-instance floating IP addresses are required for in-bound connections from the Internet (but are not required for outbound-only access). As of the time of this writing, floating IP addresses are billed monthly and are not pro-rated. Because Nodepool allocates and deallocates a floating IP address for each instance, this means that every instance will incur at least a full-month's charge for floating IP use even if it is used only briefly.
IBM Cloud does not provide a facility for atomically associating metadata with instances, which Nodepool generally relies upon for detecting and cleaning up leaked resources. To approximate this functionality, the ibmvpc driver will create all instances and associated resources within resource groups that it will expect to control. These groups begin with either the string np_ or npimages_. Do not create any resource groups matching these names, and do not create any resources within these groups. To do so runs the risk of Nodepool mistaking your resource for one that it controls and automatically deleting it.

Before using this driver, attend to the following pre-requisites:

Authentication

Several authentication methods are available. The one which will work in the widest circumstances is using an API key. The console can be used to create an API key.

The API key (or other authentication information) may be provided to Nodepool either through environment variables, or stored in a file. If a file is used, the content is exactly the same as the environment variable definition. To use an API key, the content would be:

VPC_AUTH_TYPE=iam
VPC_APIKEY=api-key-goes-here

Either set these environment variables when running the Nodepool process, or write them to a file and specify the path with the providers.[ibmvpc].credentials-file provider attribute.

Note

Several other authentication types are available; their configuration is outside the scope of this document. While nodepool-launcher should work with any auth type supported by the IBM Cloud Python SDK, nodepool-builder is only compatible with the iam auth type.

Image Building

Custom images in IBM cloud require the use of Cloud Object Storage which is not integrated with the VPC API. To use nodepool-builder, the following additional steps are required:

Create a Cloud Object Storage instance and grant access to the VPC service.

Create a bucket within the Cloud Object Storage instance.

Select an endpoint for use with Nodepool.

Note the name of the Cloud Object Storage instance, the endpoint, and the name of the bucket you created for use when configuring Nodepool.

Configuration

Selecting the ibmvpc driver adds the following options to the providers section of the configuration.

providers.[ibmvpc]

A provider's resources are partitioned into groups called pools, and within a pool, the node types which are to be made available are listed.

Note

For documentation purposes the option names are prefixed providers.[ibmvpc] to disambiguate from other drivers, but [ibmvpc] is not required in the configuration (e.g. below providers.[ibmvpc].pools refers to the pools key in the providers section when the ibmvpc driver is selected).

Example:

providers:
  - name: ibmvpc-us-south
    driver: ibmvpc
    credentials-file: /path/to/creds.env
    vpc: nodepool
    region: us-south
    subnet: sn-nodepool
    cloud-images:
      - name: bionic
        username: zuul
        keys:
          - zuul-ssh-key
        image-name: ibm-ubuntu-20-04-3-minimal-amd64-1
    pools:
      - name: main
        zone: us-south-1
        max-servers: 10
        host-key-checking: false
        labels:
          - name: bionic
            cloud-image: bionic
            profile: cx2-2x4

name

A unique name for this provider configuration. The name may not contain underscores.

vpc

The name of the VPC for this provider.

region

Name of the IBM Cloud region to interact with (e.g., us-south).

zone

Name of the IBM Cloud region zone to interact with (e.g., us-south-1). This field may be omitted if providers.[ibmvpc].pools.zone is supplied.

credentials-file

Path to the file containing the authentication information. Required unless the information is supplied via environment variables.

subnet

The name of the network upon which to create VMs. This field may be omitted if providers.[ibmvpc].pools.subnet is supplied.

public-ipv4

Whether to attach a public IPv4 address to instances.

Warning

As of the time of this writing, floating IP addresses are billed monthly and are not pro-rated. Enabling this option can lead to high usage charges.

use-internal-ip

If a public IP is attached but Nodepool should prefer the private IP, set this to true.

host-key-checking

Whether to validate SSH host keys. When true, this helps ensure that nodes are ready to receive SSH connections before they are supplied to the requestor. When set to false, nodepool-launcher will not attempt to ssh-keyscan nodes after they are booted. Disable this if nodepool-launcher and the nodes it launches are on different networks, where the launcher is unable to reach the nodes directly, or when using Nodepool with non-SSH node platforms. The default value is true.

rate

In seconds, amount to wait between operations on the provider.

boot-timeout

Once an instance is active, how long to try connecting to the image via SSH. If the timeout is exceeded, the node launch is aborted and the instance deleted.

launch-timeout

The time to wait from issuing the command to create a new instance until that instance is reported as "active". If the timeout is exceeded, the node launch is aborted and the instance deleted.

launch-retries

The number of times to retry launching a server before considering the request failed.

post-upload-hook

Filename of an optional script that can be called after an image has been uploaded to a provider but before it is taken into use. This is useful to perform last minute validation tests before an image is really used for build nodes. The script will be called as follows:

<SCRIPT> <PROVIDER> <EXTERNAL_IMAGE_ID> <LOCAL_IMAGE_FILENAME>

If the script returns with result code 0 it is treated as successful otherwise it is treated as failed and the image gets deleted.

object-storage

In order to use nodepool-builder, the information in this attribute must be supplied. The builder will upload images to the specified bucket during the image creation process, and immediately remove them. Long-term storage is not required.

instance-name

The name of the Cloud Object Storage instance.

endpoint

The endpoint URL that Nodepool should use.

bucket-name

The bucket in which to store image files.

quota

The IBM Cloud VPC API does not provide quota information, so Nodepool is unable to estimate how much quota is available before creating instances. This may lead to a high failure rate.

If you know the quota values, you may supply them here.

instances

The number of instances available in this region.

cores

The number of vCPU cores available in this region.

ram

The amount of RAM available in this region (in mebibytes).

cloud-images

Each entry in this section must refer to an entry in the labels section.

cloud-images:
  - name: bionic
    username: zuul
    keys:
      - zuul-ssh-key
    image-name: ibm-ubuntu-20-04-3-minimal-amd64-1
  - name: stretch
    username: zuul
    keys:
      - zuul-ssh-key
    image-filter:
      operating-system:
        family: "Debian GNU/Linux"
        version: "9.x Stretch/Stable - Minimal Install"

Each entry is a dictionary with the following keys

name

Identifier to refer this cloud-image from labels section. Since this name appears elsewhere in the nodepool configuration file, you may want to use your own descriptive name here.

username

The username that should be used when connecting to the node.

keys

A list of SSH keys (specified by name) to install on the node.

connection-type

The connection type that a consumer should use when connecting to the node. For most diskimages this is not necessary. However when creating Windows images this could be winrm to enable access via ansible.

connection-port

The port that a consumer should use when connecting to the node. For most diskimages this is not necessary. This defaults to 22 for ssh and 5986 for winrm.

python-path

The path of the default python interpreter. Used by Zuul to set ansible_python_interpreter. The special value auto will direct Zuul to use inbuilt Ansible logic to select the interpreter on Ansible >=2.8, and default to /usr/bin/python2 for earlier versions.

shell-type

The shell type of the node's default shell executable. Used by Zuul to set ansible_shell_type. This setting should only be used

For a windows image with the experimental connection-type ssh in which case cmd or powershell should be set and reflect the node's DefaultShell configuration.
If the default shell is not Bourne compatible (sh), but instead e.g. csh or fish, and the user is aware that there is a long-standing issue with ansible_shell_type in combination with become

image-filter

Specifies a private image to use via filters. Either this field, providers.[ibmvpc].cloud-images.image-name, providers.[ibmvpc].cloud-images.image-href, or providers.[ibmvpc].cloud-images.image-id must be provided.

If a filter is provided, Nodepool will list all of the images in the provider and reduce the list using the supplied filter. All items specified in the filter must match in order for an image to match. If more than one image matches, the images are sorted by creation time and the last one matches.

The following filters are available:

operating-system

This is a dictionary with any of the following keys:

architecture

The architecture (e.g., amd64).

dedicated-host-only

Whether the image requires a dedicated host.

display-name

The display name (e.g., Debian GNU/Linux 9.x Stretch/Stable - Minimal Install (amd64))

family

The OS family (e.g., Debian GNU/Linux).

href

The URL for the operating system record in IBM Cloud VPC (e.g., https://us-south.iaas.cloud.ibm.com/v1/operating_systems/debian-9-amd64).

name

The name of the operating system record in IBM Cloud VPC (e.g., debian-9-amd64).

vendor

The vendor (e.g., Debian).

version

The version (e.g., 9.x Stretch/Stable - Minimal Install).

owner-type

Whether the image is provided by the cloud (provider) or the user (user).

status

The status of the image (e.g., available).

image-id

Specifies a private image to use by ID. Either this field, providers.[ibmvpc].cloud-images.image-name, providers.[ibmvpc].cloud-images.image-href, or providers.[ibmvpc].cloud-images.image-filter must be provided.

image-href

Specifies a public image to use by href. Either this field, providers.[ibmvpc].cloud-images.image-name, providers.[ibmvpc].cloud-images.image-id, or providers.[ibmvpc].cloud-images.image-filter must be provided.

image-name

Specifies a public image to use by name. Either this field, providers.[ibmvpc].cloud-images.image-href, providers.[ibmvpc].cloud-images.image-id, or providers.[ibmvpc].cloud-images.image-filter must be provided.

diskimages

Each entry in a provider's diskimages section must correspond to an entry in diskimages. Such an entry indicates that the corresponding diskimage should be uploaded for use in this provider. Additionally, any nodes that are created using the uploaded image will have the associated attributes (such as flavor or metadata).

If an image is removed from this section, any previously uploaded images will be deleted from the provider.

Each entry is a dictionary with the following keys

name

Identifier to refer this image from providers.[ibmvpc].pools.labels and diskimages sections.

operating-system

The name of the IBM Cloud VPC operating-system record for this image. IBM Cloud VPC requires that custom images correspond with one of the pre-defined operating systems. If your exact OS isn't available, select the closest technology match. E.g., debian-9-amd64.

pause

When set to True, nodepool-builder will not upload the image to the provider.

key

The SSH public key that should be installed on the node.

connection-type

connection-port

The port that a consumer should use when connecting to the node. For most diskimages this is not necessary. This defaults to 22 for ssh and 5986 for winrm.

python-path

pools

A pool defines a group of resources from a provider. Each pool has a maximum number of nodes which can be launched from it, along with a number of cloud-related attributes used when launching nodes.

name

A unique name within the provider for this pool of resources. The name may not contain underscores.

priority

The priority of this provider pool (a lesser number is a higher priority). Nodepool launchers will yield requests to other provider pools with a higher priority as long as they are not paused. This means that in general, higher priority pools will reach quota first before lower priority pools begin to be used.

This setting may be specified at the provider level in order to apply to all pools within that provider, or it can be overridden here for a specific pool.

zone

Name of the IBM Cloud region zone to interact with (e.g., us-south-1). This field may be omitted if providers.[ibmvpc].zone is supplied. If both are supplied, this will override.

subnet

The name of the network upon which to create VMs. This field may be omitted if providers.[ibmvpc].subnet is supplied. If both are supplied, this will override.

node-attributes

A dictionary of key-value pairs that will be stored with the node data in ZooKeeper. The keys and values can be any arbitrary string.

public-ipv4

Whether to attach a public IPv4 address to instances.

Warning

As of the time of this writing, floating IP addresses are billed monthly and are not pro-rated. Enabling this option can lead to high usage charges.

use-internal-ip

If a public IP is attached but Nodepool should prefer the private IP, set this to true.

host-key-checking

labels

Each entry in a pool's labels section indicates that the corresponding label is available for use in this pool. When creating nodes for a label, the flavor-related attributes in that label's section will be used.

labels:
  - name: bionic
    cloud-image: bionic
    profile: cx2-2x4

Each entry is a dictionary with the following keys:

name

Identifier for this label.

cloud-image

Refers to the name of an externally managed image in the cloud that already exists on the provider. The value of cloud-image should match the name of a previously configured entry from the cloud-images section of the provider.

diskimage

Refers to provider's diskimages, see providers.[ibmvpc].diskimages. Mutually exclusive with providers.[ibmvpc].pools.labels.cloud-image

profile

The name of the IBM Cloud VPC hardware profile for the instance. E.g., cx2-2x4.

user-data

The user data value for newly created VMs.

22 KiB Raw Blame History

IBM VPC Driver

Preparation

Authentication

Image Building

Configuration

22 KiB

Raw Blame History