zuul/nodepool
Code Issues Proposed changes

Add shell-type config

Browse Source 
Ansible needs to know which shell type the node uses to operate
correctly, especially for ssh connections for windows nodes because
otherwise ansible defaults to trying bash.

Change-Id: I71abfefa57aaafd88f199be19ee7caa64efda538
This commit is contained in:
Albin Vass 2021-02-12 13:22:14 +01:00 committed by Daniel Blixt
parent 7a9d83603e
commit 0c84b7fa4e
35 changed files with 242 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
doc/source/aws.rst
View File

@ -189,6 +189,17 @@ section of the configuration.
most diskimages this is not necessary. This defaults to 22 for ssh and
5986 for winrm.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: pools
:type: list

11
doc/source/configuration.rst
View File

@ -401,6 +401,17 @@ Options
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g.
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: dib-cmd
:type: string
:default: disk-image-create

11
doc/source/gce.rst
View File

@ -153,6 +153,17 @@ section of the configuration.
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g.
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: connection-type
:type: str

11
doc/source/kubernetes.rst
View File

@ -130,6 +130,17 @@ Selecting the kubernetes driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g.
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: cpu
:type: int

11
doc/source/openshift-pods.rst
View File

@ -111,6 +111,17 @@ Selecting the openshift pods driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows node with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: env
:type: list
:default: []

11
doc/source/openshift.rst
View File

@ -138,6 +138,17 @@ Selecting the openshift driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: cpu
:type: int

11
doc/source/openstack.rst
View File

@ -347,6 +347,17 @@ Selecting the OpenStack driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows image with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: connection-type
:type: str

11
doc/source/static.rst
View File

@ -145,6 +145,17 @@ Selecting the static driver adds the following options to the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: shell-type
:type: str
:default: sh
The shell type of the node's default shell executable. Used by Zuul
to set ``ansible_shell_type``. This setting should not be used
unless the default shell is a non-Bourne (sh) compatible shell, e.g
``csh`` or ``fish``. For a windows node with the experimental
`connection-type` ``ssh``, ``cmd`` or ``powershell`` should be set
and reflect the node's ``DefaultShell`` configuration.
.. attr:: max-parallel-jobs
:type: int
:default: 1

9
nodepool/builder.py
View File

@ -938,6 +938,7 @@ class BuildWorker(BaseWorker):
build_data.builder = self._hostname
build_data.username = diskimage.username
build_data.python_path = diskimage.python_path
build_data.shell_type = diskimage.shell_type
if self._statsd:
pipeline = self._statsd.pipeline()
@ -1045,7 +1046,7 @@ class UploadWorker(BaseWorker):
self._config = new_config
def _uploadImage(self, build_id, upload_id, image_name, images, provider,
username, python_path):
username, python_path, shell_type):
'''
Upload a local DIB image build to a provider.
@ -1057,6 +1058,7 @@ class UploadWorker(BaseWorker):
:param provider: The provider from the parsed config file.
:param username:
:param python_path:
:param shell_type:
'''
start_time = time.time()
timestamp = int(start_time)
@ -1166,6 +1168,7 @@ class UploadWorker(BaseWorker):
data.format = image.extension
data.username = username
data.python_path = python_path
data.shell_type = shell_type
return data
@ -1264,13 +1267,15 @@ class UploadWorker(BaseWorker):
data.state = zk.UPLOADING
data.username = build.username
data.python_path = build.python_path
data.shell_type = build.shell_type
upnum = self._zk.storeImageUpload(
image.name, build.id, provider.name, data)
data = self._uploadImage(build.id, upnum, image.name,
local_images, provider,
build.username, build.python_path)
build.username, build.python_path,
build.shell_type)
# Set final state
self._zk.storeImageUpload(image.name, build.id,

5
nodepool/config.py
View File

@ -210,6 +210,7 @@ class DiskImage(ConfigValue):
self.image_types = set([])
self.pause = False
self.python_path = 'auto'
self.shell_type = None
self.rebuild_age = self.REBUILD_AGE
self.release = ''
self.username = 'zuul'
@ -249,6 +250,9 @@ class DiskImage(ConfigValue):
python_path = config.get('python-path', None)
if python_path:
self.python_path = python_path
shell_type = config.get('shell-type', None)
if shell_type:
self.shell_type = shell_type
rebuild_age = config.get('rebuild-age', None)
if rebuild_age:
self.rebuild_age = rebuild_age
@ -269,6 +273,7 @@ class DiskImage(ConfigValue):
other.image_types == self.image_types and
other.pause == self.pause and
other.python_path == self.python_path and
other.shell_type == self.shell_type and
other.rebuild_age == self.rebuild_age and
other.release == self.release and
other.username == self.username)

3
nodepool/driver/aws/config.py
View File

@ -35,6 +35,7 @@ class ProviderCloudImage(ConfigValue):
and self.image_id == other.image_id
and self.username == other.username
and self.python_path == other.python_path
and self.shell_type == other.shell_type
and self.connection_type == other.connection_type
and self.connection_port == other.connection_port)
return False
@ -225,6 +226,7 @@ class AwsProviderConfig(ProviderConfig):
i.username = image.get('username', None)
i.python_path = image.get('python-path', 'auto')
i.shell_type = image.get('shell-type', None)
i.connection_type = image.get('connection-type', 'ssh')
i.connection_port = image.get(
'connection-port',
@ -272,6 +274,7 @@ class AwsProviderConfig(ProviderConfig):
'name': str,
'connection-type': str,
'connection-port': int,
'shell-type': str,
'image-id': str,
"image-filters": [image_filters],
'username': str,

1
nodepool/driver/aws/handler.py
View File

@ -111,6 +111,7 @@ class AwsInstanceLauncher(NodeLauncher):
self.node.host_keys = keys
self.node.username = self.label.cloud_image.username
self.node.python_path = self.label.cloud_image.python_path
self.node.shell_type = self.label.cloud_image.shell_type
self.zk.storeNode(self.node)
self.log.info("Instance %s is ready", instance_id)

6
nodepool/driver/gce/config.py
View File

@ -30,6 +30,7 @@ class ProviderCloudImage(ConfigValue):
self.python_path = None
self.connection_type = None
self.connection_port = None
self.shell_type = None
def __eq__(self, other):
if isinstance(other, ProviderCloudImage):
@ -39,7 +40,8 @@ class ProviderCloudImage(ConfigValue):
and self.key == other.key
and self.python_path == other.python_path
and self.connection_type == other.connection_type
and self.connection_port == other.connection_port)
and self.connection_port == other.connection_port
and self.shell_type == other.shell_type)
return False
def __repr__(self):
@ -202,6 +204,7 @@ class GCEProviderConfig(ProviderConfig):
i.connection_port = image.get(
'connection-port',
default_port_mapping.get(i.connection_type, 22))
i.shell_type = image.get('shell-type', None)
self.cloud_images[i.name] = i
for pool in self.provider.get('pools', []):
@ -229,6 +232,7 @@ class GCEProviderConfig(ProviderConfig):
'name': str,
'connection-type': str,
'connection-port': int,
'shell-type': str,
'image-id': str,
'image-project': str,
'image-family': str,

3
nodepool/driver/kubernetes/config.py
View File

@ -28,6 +28,7 @@ class KubernetesLabel(ConfigValue):
other.type == self.type and
other.image_pull == self.image_pull and
other.python_path == self.python_path and
other.shell_type == self.shell_type and
other.image == self.image and
other.cpu == self.cpu and
other.memory == self.memory and
@ -61,6 +62,7 @@ class KubernetesPool(ConfigPool):
pl.image = label.get('image')
pl.image_pull = label.get('image-pull', 'IfNotPresent')
pl.python_path = label.get('python-path', 'auto')
pl.shell_type = label.get('shell-type')
pl.cpu = label.get('cpu')
pl.memory = label.get('memory')
pl.env = label.get('env', [])
@ -112,6 +114,7 @@ class KubernetesProviderConfig(ProviderConfig):
'image': str,
'image-pull': str,
'python-path': str,
'shell-type': str,
'cpu': int,
'memory': int,
'env': [env_var],

1
nodepool/driver/kubernetes/handler.py
View File

@ -38,6 +38,7 @@ class K8SLauncher(NodeLauncher):
self.node.state = zk.READY
self.node.python_path = self.label.python_path
self.node.shell_type = self.label.shell_type
# NOTE: resource access token may be encrypted here
self.node.connection_port = resource
if self.label.type == "namespace":

3
nodepool/driver/openshift/config.py
View File

@ -32,6 +32,7 @@ class OpenshiftLabel(ConfigValue):
other.cpu == self.cpu and
other.memory == self.memory and
other.python_path == self.python_path and
other.shell_type == self.shell_type and
other.env == self.env and
other.node_selector == self.node_selector)
return False
@ -64,6 +65,7 @@ class OpenshiftPool(ConfigPool):
pl.cpu = label.get('cpu')
pl.memory = label.get('memory')
pl.python_path = label.get('python-path', 'auto')
pl.shell_type = label.get('shell-type')
pl.env = label.get('env', [])
pl.node_selector = label.get('node-selector')
pl.pool = self
@ -116,6 +118,7 @@ class OpenshiftProviderConfig(ProviderConfig):
'cpu': int,
'memory': int,
'python-path': str,
'shell-type': str,
'env': [env_var],
'node-selector': dict,
}

1
nodepool/driver/openshift/handler.py
View File

@ -47,6 +47,7 @@ class OpenshiftLauncher(NodeLauncher):
self.node.state = zk.READY
self.node.python_path = self.label.python_path
self.node.shell_type = self.label.shell_type
# NOTE: resource access token may be encrypted here
self.node.connection_port = resource
self.zk.storeNode(self.node)

1
nodepool/driver/openshiftpods/config.py
View File

@ -56,6 +56,7 @@ class OpenshiftPodsProviderConfig(OpenshiftProviderConfig):
'cpu': int,
'memory': int,
'python-path': str,
'shell-type': str,
'env': [env_var],
'node-selector': dict
}

1
nodepool/driver/openshiftpods/handler.py
View File

@ -34,6 +34,7 @@ class OpenshiftPodLauncher(OpenshiftLauncher):
self.node.state = zk.READY
self.node.python_path = self.label.python_path
self.node.shell_type = self.label.shell_type
# NOTE: resource access token may be encrypted here
k8s = self.handler.manager.k8s_client
self.node.connection_port = {

4
nodepool/driver/openstack/config.py
View File

@ -53,6 +53,7 @@ class ProviderCloudImage(ConfigValue):
self.image_name = None
self.username = None
self.python_path = None
self.shell_type = None
self.connection_type = None
self.connection_port = None
@ -64,6 +65,7 @@ class ProviderCloudImage(ConfigValue):
self.image_name == other.image_name and
self.username == other.username and
self.python_path == other.python_path and
self.shell_type == other.shell_type and
self.connection_type == other.connection_type and
self.connection_port == other.connection_port)
return False
@ -336,6 +338,7 @@ class OpenStackProviderConfig(ProviderConfig):
i.image_name = image.get('image-name', None)
i.username = image.get('username', None)
i.python_path = image.get('python-path', 'auto')
i.shell_type = image.get('shell-type', None)
i.connection_type = image.get('connection-type', 'ssh')
i.connection_port = image.get(
'connection-port',
@ -366,6 +369,7 @@ class OpenStackProviderConfig(ProviderConfig):
v.Exclusive('image-name', 'cloud-image-name-or-id'): str,
'username': str,
'python-path': str,
'shell-type': str,
}
pool_label_main = {

3
nodepool/driver/openstack/handler.py
View File

@ -87,6 +87,7 @@ class OpenStackNodeLauncher(NodeLauncher):
image_name = diskimage.name
username = cloud_image.username
python_path = cloud_image.python_path
shell_type = cloud_image.shell_type
connection_type = diskimage.connection_type
connection_port = diskimage.connection_port
@ -105,6 +106,7 @@ class OpenStackNodeLauncher(NodeLauncher):
image_name = self.label.cloud_image.name
username = self.label.cloud_image.username
python_path = self.label.cloud_image.python_path
shell_type = self.label.cloud_image.shell_type
connection_type = self.label.cloud_image.connection_type
connection_port = self.label.cloud_image.connection_port
@ -159,6 +161,7 @@ class OpenStackNodeLauncher(NodeLauncher):
self.node.username = username
self.node.python_path = python_path
self.node.shell_type = shell_type
self.node.connection_type = connection_type
self.node.connection_port = connection_port

1
nodepool/driver/simple.py
View File

@ -153,6 +153,7 @@ class SimpleTaskManagerLauncher(NodeLauncher):
self.node.host_keys = keys
self.node.username = self.label.cloud_image.username
self.node.python_path = self.label.cloud_image.python_path
self.node.shell_type = self.label.cloud_image.shell_type
self.zk.storeNode(self.node)
self.log.info("Instance %s is ready", hostname)

2
nodepool/driver/static/config.py
View File

@ -57,6 +57,7 @@ class StaticPool(ConfigPool):
'connection-port': int(
node.get('connection-port', node.get('ssh-port', 22))),
'connection-type': node.get('connection-type', 'ssh'),
'shell-type': node.get('shell-type', None),
'username': node.get('username', 'zuul'),
'max-parallel-jobs': int(node.get('max-parallel-jobs', 1)),
'python-path': node.get('python-path', 'auto'),
@ -108,6 +109,7 @@ class StaticProviderConfig(ProviderConfig):
'host-key': v.Any(str, [str]),
'connection-port': int,
'connection-type': str,
'shell-type': str,
'max-parallel-jobs': int,
'python-path': str,
}

7
nodepool/driver/static/provider.py
View File

@ -194,6 +194,7 @@ class StaticNodeProvider(Provider):
node.connection_port = static_node["connection-port"]
node.connection_type = static_node["connection-type"]
node.python_path = static_node["python-path"]
node.shell_type = static_node["shell-type"]
nodeutils.set_node_ip(node)
node.host_keys = host_keys
node.attributes = pool.node_attributes
@ -218,14 +219,15 @@ class StaticNodeProvider(Provider):
static_node["username"],
static_node["connection-port"],
static_node["connection-type"],
static_node["shell-type"],
static_node["python-path"],
host_keys,
)
for node in nodes:
original_attrs = (node.type, node.username, node.connection_port,
node.connection_type, node.python_path,
node.host_keys)
node.shell_type, node.connection_type,
node.python_path, node.host_keys)
if original_attrs == new_attrs:
continue
@ -236,6 +238,7 @@ class StaticNodeProvider(Provider):
node.username = static_node["username"]
node.connection_port = static_node["connection-port"]
node.connection_type = static_node["connection-type"]
node.shell_type = static_node["shell-type"]
node.python_path = static_node["python-path"]
nodeutils.set_node_ip(node)
node.host_keys = host_keys

17
nodepool/tests/fixtures/aws.yaml vendored
View File

@ -17,6 +17,7 @@ labels:
- name: ubuntu1404-iam-instance-profile-arn
- name: ubuntu1404-with-tags
- name: ubuntu1404-with-name-tag
- name: ubuntu1404-with-shell-type
providers:
- name: ec2-us-west-2
@ -48,6 +49,10 @@ providers:
values:
- ubuntu*
username: ubuntu
- name: ubuntu1404-with-shell-type
image-id: ami-1e749f67
username: ubuntu
shell-type: csh
pools:
- name: ebs-optimized
max-servers: 1
@ -161,3 +166,15 @@ providers:
key-name: zuul
tags:
Name: different-name
- name: shell-type
max-servers: 1
subnet-id: null
security-group-id: null
node-attributes:
key1: value1
key2: value2
labels:
- name: ubuntu1404-with-shell-type
cloud-image: ubuntu1404-with-shell-type
instance-type: t3.medium
key-name: zuul

12
nodepool/tests/fixtures/config_validate/good.yaml vendored
View File

@ -24,6 +24,9 @@ labels:
- name: openshift-project
- name: openshift-pod
- name: centos-ami
- name: winrm
- name: winssh
providers:
- name: cloud1
@ -124,6 +127,15 @@ providers:
connection-port: 22022
username: zuul
max-parallel-jobs: 1
- name: windows.example.com
labels: winrm
username: zuul
connection-type: winrm
connection-port: 5986
- name: windows-ssh.example.com
labels: winssh
username: zuul
shell-type: cmd
- name: kubespray
driver: kubernetes

1
nodepool/tests/fixtures/node_unmanaged_image.yaml vendored
View File

@ -29,6 +29,7 @@ providers:
cloud-images:
- name: fake-image
python-path: /usr/bin/python3
shell-type: csh
- name: fake-image-windows
username: zuul
connection-type: winrm

1
nodepool/tests/fixtures/openshift.yaml vendored
View File

@ -28,3 +28,4 @@ providers:
type: pod
image: docker.io/fedora:28
python-path: '/usr/bin/python3'
shell-type: csh

26
nodepool/tests/fixtures/static-shell-type.yaml vendored Normal file
View File

@ -0,0 +1,26 @@
zookeeper-servers:
- host: {zookeeper_host}
port: {zookeeper_port}
chroot: {zookeeper_chroot}
zookeeper-tls:
ca: {zookeeper_ca}
cert: {zookeeper_cert}
key: {zookeeper_key}
labels:
- name: fake-label
providers:
- name: static-provider
driver: static
pools:
- name: main
nodes:
- name: fake-host-1
labels: fake-label
host-key: ssh-rsa FAKEKEY
timeout: 13
connection-port: 22022
username: zuul
shell-type: cmd

10
nodepool/tests/unit/test_driver_aws.py
View File

@ -49,7 +49,8 @@ class TestDriverAws(tests.DBTestCase):
host_key_checking=True,
userdata=None,
public_ip=True,
tags=[]):
tags=[],
shell_type=None):
aws_id = 'AK000000000000000000'
aws_key = '0123456789abcdef0123456789abcdef0123456789abcdef'
self.useFixture(
@ -97,6 +98,8 @@ class TestDriverAws(tests.DBTestCase):
raw_config['providers'][0]['pools'][4]['security-group-id'] = sg_id
raw_config['providers'][0]['pools'][5]['subnet-id'] = subnet_id
raw_config['providers'][0]['pools'][5]['security-group-id'] = sg_id
raw_config['providers'][0]['pools'][6]['subnet-id'] = subnet_id
raw_config['providers'][0]['pools'][6]['security-group-id'] = sg_id
with tempfile.NamedTemporaryFile() as tf:
tf.write(yaml.safe_dump(
@ -152,6 +155,7 @@ class TestDriverAws(tests.DBTestCase):
self.assertEqual(node.connection_type, 'ssh')
self.assertEqual(node.attributes,
{'key1': 'value1', 'key2': 'value2'})
self.assertEqual(node.shell_type, shell_type)
if host_key_checking:
nodescan.assert_called_with(
node.interface_ip,
@ -251,3 +255,7 @@ class TestDriverAws(tests.DBTestCase):
tags=[
{"Key": "Name", "Value": "different-name"}
])
def test_ec2_machine_shell_type(self):
self._test_ec2_machine('ubuntu1404-with-shell-type',
shell_type="csh")

1
nodepool/tests/unit/test_driver_azure.py
View File

@ -195,3 +195,4 @@ class TestDriverAzure(tests.DBTestCase):
self.assertEqual(node.state, zk.READY)
self.assertIsNotNone(node.launcher)
self.assertEqual(node.connection_type, 'ssh')
self.assertIsNone(node.shell_type)

3
nodepool/tests/unit/test_driver_openshift.py
View File

@ -155,6 +155,7 @@ class TestDriverOpenshift(tests.DBTestCase):
self.assertEqual(node.connection_type, 'kubectl')
self.assertEqual(node.connection_port.get('token'), 'fake-token')
self.assertEqual(node.python_path, '/usr/bin/python3')
self.assertEqual(node.shell_type, 'csh')
self.assertEqual(node.attributes,
{'key1': 'value1', 'key2': 'value2'})
@ -183,6 +184,8 @@ class TestDriverOpenshift(tests.DBTestCase):
self.assertIsNotNone(node.launcher)
self.assertEqual(node.connection_type, 'project')
self.assertEqual(node.connection_port.get('token'), 'fake-token')
self.assertEqual(node.python_path, 'auto')
self.assertIsNone(node.shell_type)
node.state = zk.DELETING
self.zk.storeNode(node)

21
nodepool/tests/unit/test_driver_static.py
View File

@ -64,6 +64,8 @@ class TestDriverStatic(tests.DBTestCase):
self.assertEqual(nodes[0].host_keys, ['ssh-rsa FAKEKEY'])
self.assertEqual(nodes[0].attributes,
{'key1': 'value1', 'key2': 'value2'})
self.assertEqual(nodes[0].python_path, 'auto')
self.assertIsNone(nodes[0].shell_type)
def test_static_python_path(self):
'''
@ -528,6 +530,25 @@ class TestDriverStatic(tests.DBTestCase):
self.assertEqual(len(nodes), 1)
nodescan_mock.assert_not_called()
def test_static_shell_type(self):
'''
Test that static python-path works.
'''
configfile = self.setup_config('static-shell-type.yaml')
pool = self.useNodepool(configfile, watermark_sleep=1)
pool.start()
self.log.debug("Waiting for node pre-registration")
nodes = self.waitForNodes('fake-label')
self.assertEqual(nodes[0].shell_type, "cmd")
nodes[0].state = zk.USED
self.zk.storeNode(nodes[0])
self.log.debug("Waiting for node to be re-available")
nodes = self.waitForNodes('fake-label')
self.assertEqual(nodes[0].shell_type, "cmd")
def test_missing_static_node(self):
"""Test that a missing static node is added"""
configfile = self.setup_config('static-2-nodes.yaml')

4
nodepool/tests/unit/test_launcher.py
View File

@ -1383,6 +1383,7 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual(len(nodes), 1)
self.assertIsNone(nodes[0].username)
self.assertEqual(nodes[0].python_path, '/usr/bin/python3')
self.assertEqual(nodes[0].shell_type, 'csh')
nodes = self.waitForNodes('fake-label-windows')
self.assertEqual(len(nodes), 1)
@ -1391,6 +1392,7 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual(5986, nodes[0].connection_port)
self.assertEqual(nodes[0].host_keys, [])
self.assertEqual(nodes[0].python_path, 'auto')
self.assertIsNone(nodes[0].shell_type)
nodes = self.waitForNodes('fake-label-arbitrary-port')
self.assertEqual(len(nodes), 1)
@ -1398,6 +1400,8 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual('winrm', nodes[0].connection_type)
self.assertEqual(1234, nodes[0].connection_port)
self.assertEqual(nodes[0].host_keys, [])
self.assertEqual(nodes[0].python_path, 'auto')
self.assertIsNone(nodes[0].shell_type)
def test_unmanaged_image_provider_name(self):
"""

15
nodepool/zk.py
View File

@ -299,6 +299,7 @@ class ImageBuild(BaseModel):
self.builder_id = None # Unique ID
self.username = None
self.python_path = None
self.shell_type = None
def __repr__(self):
d = self.toDict()
@ -332,6 +333,7 @@ class ImageBuild(BaseModel):
d['formats'] = ','.join(self.formats)
d['username'] = self.username
d['python_path'] = self.python_path
d['shell_type'] = self.shell_type
return d
@staticmethod
@ -350,6 +352,7 @@ class ImageBuild(BaseModel):
o.builder_id = d.get('builder_id')
o.username = d.get('username', 'zuul')
o.python_path = d.get('python_path', '/usr/bin/python2')
o.shell_type = d.get('shell_type')
# Only attempt the split on non-empty string
if d.get('formats', ''):
o.formats = d.get('formats', '').split(',')
@ -363,7 +366,8 @@ class ImageUpload(BaseModel):
VALID_STATES = set([UPLOADING, READY, DELETING, FAILED])
def __init__(self, build_id=None, provider_name=None, image_name=None,
upload_id=None, username=None, python_path=None):
upload_id=None, username=None, python_path=None,
shell_type=None):
super(ImageUpload, self).__init__(upload_id)
self.build_id = build_id
self.provider_name = provider_name
@ -371,6 +375,7 @@ class ImageUpload(BaseModel):
self.format = None
self.username = username
self.python_path = python_path
self.shell_type = shell_type
self.external_id = None # Provider ID of the image
self.external_name = None # Provider name of the image
@ -403,6 +408,7 @@ class ImageUpload(BaseModel):
d['format'] = self.format
d['username'] = self.username
d['python_path'] = self.python_path
d['shell_type'] = self.shell_type
return d
@staticmethod
@ -425,6 +431,7 @@ class ImageUpload(BaseModel):
o.format = d.get('format')
o.username = d.get('username', 'zuul')
o.python_path = d.get('python_path', '/usr/bin/python2')
o.shell_type = d.get('shell_type')
return o
@ -551,7 +558,9 @@ class Node(BaseModel):
self.public_ipv6 = None
self.host_id = None
self.interface_ip = None
self.connection_type = None
self.connection_port = 22
self.shell_type = None
self.image_id = None
self.launcher = None
self.created_time = None
@ -560,7 +569,6 @@ class Node(BaseModel):
self.comment = None
self.hold_job = None
self.username = None
self.connection_type = None
self.host_keys = []
self.hold_expiration = None
self.resources = None
@ -600,6 +608,7 @@ class Node(BaseModel):
self.username == other.username and
self.connection_type == other.connection_type and
self.connection_port == other.connection_port and
self.shell_type == other.shell_type and
self.host_keys == other.host_keys and
self.hold_expiration == other.hold_expiration and
self.resources == other.resources and
@ -650,6 +659,7 @@ class Node(BaseModel):
d['username'] = self.username
d['connection_type'] = self.connection_type
d['connection_port'] = self.connection_port
d['shell_type'] = self.shell_type
d['hold_expiration'] = self.hold_expiration
d['resources'] = self.resources
d['attributes'] = self.attributes
@ -718,6 +728,7 @@ class Node(BaseModel):
self.resources = d.get('resources')
self.attributes = d.get('attributes')
self.python_path = d.get('python_path')
self.shell_type = d.get('shell_type')
class ZooKeeper(object):