Add ensure-virtualenv

This is part of the efforts to remove pip-and-virtualenv from our base
images [1].

There are some users who specifically require the virtualenv command
(perhaps, like dib, they have some code that uses the activate_this.py
mechanisms it provides wich venv does not).

This installs the virtualenv package for the currently running
distribution.

One of the main maintenance issues of pip-and-virtualenv is that tried
to ensure that "virtualenv" created a Python 2 environment always by
default.  Now that we have Python 3 only distributions like current
Fedora, this is not something we can continue to do (even if we wanted
to, which we don't).

What owns virtualenv and what it produces in our heterogeneous
environment is messy, and I think the best we can do is document it as
done here.

[1] https://docs.opendev.org/opendev/infra-specs/latest/specs/cleanup-test-node-python.html

Change-Id: I97d8bfb970ed2b5aaa02a0813899014c94994066

doc/source/python-roles.rst

@@ -10,6 +10,7 @@ Python Roles
 .. zuul:autorole:: ensure-sphinx
 .. zuul:autorole:: ensure-tox
 .. zuul:autorole:: ensure-twine
+.. zuul:autorole:: ensure-virtualenv
 .. zuul:autorole:: fetch-coverage-output
 .. zuul:autorole:: fetch-python-sdist-output
 .. zuul:autorole:: fetch-sphinx-output

roles/ensure-virtualenv/README.rst

@@ -0,0 +1,28 @@
+Ensure virtualenv is available
+
+This role installs the requirements for the ``virtualenv`` command
+on the current distribution.
+
+Users should be aware of some portability issues when using
+``virtualenv``:
+
+* Distributions differ on the interpreter that ``virtualenv`` is
+  provided by, so by calling ``virtualenv`` with no other arguments
+  means that on some platforms you will get a Python 2 environment and
+  others a Python 3 environment.
+* If you wish to call ``virtualenv`` as a module (e.g. ``python -m
+  virtualenv``) you will need to know which interpreter owns the
+  ``virtualenv`` package for your distribution; e.g. on some, such as
+  Bionic, ``virtualenv`` is provided by ``python3-virtualenv`` but
+  ``python`` refers to Python 2, so ``python -m virtualenv`` is not a
+  portable way to call ``virtualenv``.
+* ``virtualenv -p python3`` is likely the most portable way to
+  consistently get a Python 3 environment.  ``virtualenv -p python2``
+  may not work on some platforms without Python 2.
+* If you use Python 3 and do not require the specific features of
+  ``virtualenv``, it is likely easier to use Python's inbuilt
+  ``python3 -m venv`` module to create an isolated environment.  If
+  you are using ``pip:`` in your Ansible roles and require an
+  environment, see the documentation for :zuul:role:`ensure-pip`.
+
+

roles/ensure-virtualenv/tasks/CentOS-7.yaml

@@ -0,0 +1,6 @@
+- name: Install virtualenv
+  package:
+    name:
+      - python-virtualenv
+  become: yes
+

roles/ensure-virtualenv/tasks/Debian.yaml

@@ -0,0 +1,5 @@
+- name: Install virtualenv
+  package:
+    name:
+      - virtualenv
+  become: yes

roles/ensure-virtualenv/tasks/Gentoo.yaml

@@ -0,0 +1,5 @@
+- name: Install virtualenv
+  package:
+    name: dev-python/virtualenv
+  become: yes
+

roles/ensure-virtualenv/tasks/RedHat.yaml

@@ -0,0 +1,6 @@
+- name: Install virtualenv
+  package:
+    name:
+      - python3-virtualenv
+  become: yes
+

roles/ensure-virtualenv/tasks/Suse.yaml

@@ -0,0 +1,4 @@
+- name: Install virtualenv
+  package:
+    name: python3-virtualenv
+  become: yes

roles/ensure-virtualenv/tasks/default.yaml

@@ -0,0 +1,3 @@
+- name: Unsupported platform
+  fail:
+    msg: 'This platform is currently unsupported'

roles/ensure-virtualenv/tasks/main.yaml

@@ -0,0 +1,18 @@
+- name: Check if virtualenv is installed
+  shell: |
+    command -v virtualenv || exit 1
+  args:
+    executable: /bin/bash
+  register: virtualenv_preinstalled
+  failed_when: false
+
+- name: Install virtualenv package
+  include: "{{ item }}"
+  with_first_found:
+    - "{{ ansible_distribution }}-{{ ansible_distribution_major_version }}.yaml"
+    - "{{ ansible_distribution_release }}.yaml"
+    - "{{ ansible_distribution }}.yaml"
+    - "{{ ansible_os_family }}.yaml"
+    - "default.yaml"
+  when:
+    - virtualenv_preinstalled.rc != 0

test-playbooks/ensure-pip.yaml

@@ -1,7 +1,9 @@
 - hosts: all
-  roles:
-    - ensure-pip
   tasks:
+    - name: Include ensure-pip
+      include_role:
+        name: ensure-pip
+
     - name: Sanity check provided virtualenv command works
       shell: |
         tmp_venv=$(mktemp -d -t venv-XXXXXXXXXX)
@@ -11,12 +13,31 @@
       failed_when: false
       register: _venv_sanity
 
-    - name: Assert sanity check
+    - name: Assert pip venv sanity check
       fail:
         msg: 'The virtualenv_command: "{{ ensure_pip_virtualenv_command }}" does not appear to work!'
       when:
         - _venv_sanity.rc != 0
 
+    - name: Include ensure-virtualenv
+      include_role:
+        name: ensure-virtualenv
+
+    - name: Sanity check virtualenv command works
+      shell: |
+        tmp_venv=$(mktemp -d -t venv-XXXXXXXXXX)
+        trap "rm -rf $tmp_venv" EXIT
+        virtualenv $tmp_venv
+        $tmp_venv/bin/pip install tox
+      failed_when: false
+      register: _virtualenv_sanity
+
+    - name: Assert sanity check
+      fail:
+        msg: 'The virtualenv command does not appear to work!'
+      when:
+        - _virtualenv_sanity.rc != 0
+
 # NOTE(ianw) : this does not play nicely with pip-and-virtualenv which
 # has already installed from source.  We might be able to test this
 # once it's gone...

zuul-tests.d/python-jobs.yaml

@@ -3,6 +3,7 @@
     description: Test the ensure-pip role
     files:
       - roles/ensure-pip/.*
+      - roles/ensure-virtualenv/.*
     run: test-playbooks/ensure-pip.yaml
     tags: all-platforms