zuul/nodepool
Code Issues Proposed changes

Set default python-path to "auto"

Browse Source 
The "python-path" configuration option makes its way through to Zuul
where it sets the "ansible_interpreter_path" in the inventory.
Currently this defaults to "/usr/bin/python2" which is wrong for
Python 3-only distributions.

Ansible >=2.8 provides for automated discovery of the interpreter to
avoid runtime errors choosing an invalid interpreter [1].  Using this
should mean that "python-path" doesn't need to be explicitly for any
common case.  As more distributions become Python 3 only, this should
"do the right thing" without further configuration.

This switches the default python-path to "auto".  The dependent change
updates Zuul to accept this and use it when running with Ansible
>=2.8, or default back to "/usr/bin/python2" for earlier Ansible
versions.

Testing and documentation is updated, and a release note added.

[1] https://docs.ansible.com/ansible/2.8/reference_appendices/interpreter_discovery.html

Depends-On: https://review.opendev.org/682275
Change-Id: I02a1a618c8806b150049e91b644ec3c0cb826ba4
This commit is contained in:
Ian Wienand 2019-09-16 15:38:35 +10:00
parent 67bdd85425
commit db87a0845f
9 changed files with 67 additions and 19 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
doc/source/configuration.rst
View File

@ -311,9 +311,13 @@ Options
.. attr:: python-path
:type: string
:default: /usr/bin/python2
:default: auto
The path of the default python interpreter.
The path of the default python interpreter. Used by Zuul to set
``ansible_python_interpreter``. The special value ``auto`` will
direct Zuul to use inbuilt Ansible logic to select the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: dib-cmd
:type: string
@ -711,9 +715,13 @@ Selecting the OpenStack driver adds the following options to the
.. attr:: python-path
:type: str
:default: /usr/bin/python2
:default: auto
The path of the default python interpreter.
The path of the default python interpreter. Used by Zuul to set
``ansible_python_interpreter``. The special value ``auto`` will
direct Zuul to use inbuilt Ansible logic to select the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: connection-type
:type: str
@ -1134,7 +1142,11 @@ Selecting the static driver adds the following options to the
:type: str
:default: /usr/bin/python2
The path of the default python interpreter.
The path of the default python interpreter. Used by Zuul to set
``ansible_python_interpreter``. The special value ``auto`` will
direct Zuul to use inbuilt Ansible logic to select the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: max-parallel-jobs
:type: int
@ -1257,9 +1269,14 @@ Selecting the kubernetes driver adds the following options to the
.. attr:: python-path
:type: str
:default: /usr/bin/python2
:default: auto
The path of the default python interpreter. Used by Zuul to set
``ansible_python_interpreter``. The special value ``auto`` will
direct Zuul to use inbuilt Ansible logic to select the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
The path of the default python interpreter.
Openshift Driver
@ -1384,9 +1401,13 @@ Selecting the openshift driver adds the following options to the
.. attr:: python-path
:type: str
:default: /usr/bin/python2
:default: auto
The path of the default python interpreter.
The path of the default python interpreter. Used by Zuul to set
``ansible_python_interpreter``. The special value ``auto`` will
direct Zuul to use inbuilt Ansible logic to select the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: cpu
:type: int
@ -1629,9 +1650,13 @@ section of the configuration.
.. attr:: python-path
:type: str
:default: /usr/bin/python2
:default: auto
The path of the default python interpreter.
The path of the default python interpreter. Used by Zuul to set
``ansible_python_interpreter``. The special value ``auto`` will
direct Zuul to use inbuilt Ansible logic to select the
interpreter on Ansible >=2.8, and default to
``/usr/bin/python2`` for earlier versions.
.. attr:: connection-type
:type: str

2
nodepool/config.py
View File

@ -119,7 +119,7 @@ class Config(ConfigValue):
d.image_types = set(diskimage.get('formats', []))
d.pause = bool(diskimage.get('pause', False))
d.username = diskimage.get('username', 'zuul')
d.python_path = diskimage.get('python-path', '/usr/bin/python2')
d.python_path = diskimage.get('python-path', 'auto')
d.build_timeout = diskimage.get('build-timeout', (8 * 60 * 60))
self.diskimages[d.name] = d

2
nodepool/driver/aws/config.py
View File

@ -192,7 +192,7 @@ class AwsProviderConfig(ProviderConfig):
i.name = image['name']
i.image_id = image.get('image-id', None)
i.username = image.get('username', None)
i.python_path = image.get('python-path', '/usr/bin/python2')
i.python_path = image.get('python-path', 'auto')
i.connection_type = image.get('connection-type', 'ssh')
i.connection_port = image.get(
'connection-port',

2
nodepool/driver/kubernetes/config.py
View File

@ -56,7 +56,7 @@ class KubernetesPool(ConfigPool):
pl.type = label['type']
pl.image = label.get('image')
pl.image_pull = label.get('image-pull', 'IfNotPresent')
pl.python_path = label.get('python-path', '/usr/bin/python2')
pl.python_path = label.get('python-path', 'auto')
pl.pool = self
self.labels[pl.name] = pl
full_config.labels[label['name']].pools.append(self)

2
nodepool/driver/openshift/config.py
View File

@ -61,7 +61,7 @@ class OpenshiftPool(ConfigPool):
pl.image_pull = label.get('image-pull', 'IfNotPresent')
pl.cpu = label.get('cpu')
pl.memory = label.get('memory')
pl.python_path = label.get('python-path', '/usr/bin/python2')
pl.python_path = label.get('python-path', 'auto')
pl.pool = self
self.labels[pl.name] = pl
full_config.labels[label['name']].pools.append(self)

2
nodepool/driver/openstack/config.py
View File

@ -325,7 +325,7 @@ class OpenStackProviderConfig(ProviderConfig):
i.image_id = image.get('image-id', None)
i.image_name = image.get('image-name', None)
i.username = image.get('username', None)
i.python_path = image.get('python-path', '/usr/bin/python2')
i.python_path = image.get('python-path', 'auto')
i.connection_type = image.get('connection-type', 'ssh')
i.connection_port = image.get(
'connection-port',

2
nodepool/driver/static/config.py
View File

@ -59,7 +59,7 @@ class StaticPool(ConfigPool):
'connection-type': node.get('connection-type', 'ssh'),
'username': node.get('username', 'zuul'),
'max-parallel-jobs': int(node.get('max-parallel-jobs', 1)),
'python-path': node.get('python-path', '/usr/bin/python2'),
'python-path': node.get('python-path', 'auto'),
})
if isinstance(node['labels'], str):
for label in node['labels'].split():

4
nodepool/tests/unit/test_launcher.py
View File

@ -147,7 +147,7 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual(nodes[1].type, ['fake-label1'])
self.assertEqual(nodes[2].type, ['fake-label4'])
self.assertEqual(nodes[3].type, ['fake-label2'])
self.assertEqual(nodes[0].python_path, '/usr/bin/python2')
self.assertEqual(nodes[0].python_path, 'auto')
def _test_node_assignment_at_quota(self,
config,
@ -1331,7 +1331,7 @@ class TestLauncher(tests.DBTestCase):
self.assertEqual('winrm', nodes[0].connection_type)
self.assertEqual(5986, nodes[0].connection_port)
self.assertEqual(nodes[0].host_keys, [])
self.assertEqual(nodes[0].python_path, '/usr/bin/python2')
self.assertEqual(nodes[0].python_path, 'auto')
nodes = self.waitForNodes('fake-label-arbitrary-port')
self.assertEqual(len(nodes), 1)

23
releasenotes/notes/python-path-auto-83881e94d835ff83.yaml Normal file
View File

@ -0,0 +1,23 @@
---
features:
- |
The default ``python-path`` is now ``auto`` (instead of
``/usr/bin/python2``). With this, Zuul 3.11.1 and greater will set the
``ansible_python_interpreter`` to ``auto`` when using Ansible
>=2.8 to use `automated interpreter discovery
<https://docs.ansible.com/ansible/latest/reference_appendices/interpreter_discovery.html>`__.
When using earlier Ansible, it will remain the old default of
``/usr/bin/python2``.
This will remove the need to override python-path explicitly for
Python 3-only distributions, which should be detected correctly
automatically.
upgrade:
- |
This release should only be run against Zuul 3.11.1 or greater.
Earlier Zuul releases will not convert the new default
``python-path`` of ``auto`` to ``/usr/bin/python2`` for Ansible
<2.8, leading to a configuration error. It may be possible to use
earlier Zuul releases if you you are only using Ansible >= 2.8, or
explicitly set ``python-path`` for every image.