Add run-bindep role and add it to unittests pre

Before we revoke sudo, we need to run bindep in the project. Add
the run-bindep role.

Copy the content of the jenkins script for now. This way we can iterate
on the content in smaller chunks.

Change-Id: Ib893b06d05039af078e6eea22e882f6f1efae226

playbooks/unittests/pre.yaml

@@ -1,4 +1,5 @@
 hosts: all
 roles:
   - extra-test-setup
+  - run-bindep
   - revoke-sudo

roles/extra-test-setup/README.rst

@@ -1,9 +1,9 @@
 Perform project test setup tasks.
 
 This role assumes that Zuul has checked out a change for a project at
-``{{ zuul_workspace_root }}/src/{{ zuul.project.canonical_name }}``
-and looks for a file named ``tools/test-setup.sh``.  If that file
-exists and is executable, it will be run.
+``src/{{ zuul.project.canonical_name }}`` and looks for a file named
+``tools/test-setup.sh``.  If that file exists and is executable, it will
+be run.
 
 This allows projects to specify test-setup steps (such as creating or
 initializing a database) in a form that can be easily run by both an

roles/extra-test-setup/tasks/main.yaml

@@ -1,12 +1,12 @@
-- name: Check if projects tools/test-setup.sh exists.
+- name: Check if project's tools/test-setup.sh exists
   stat:
-    path: "{{ zuul_workspace_root }}/src/{{ zuul.project.canonical_name }}/tools/test-setup.sh"
+    path: "src/{{ zuul.project.canonical_name }}/tools/test-setup.sh"
   register: p
 
-- name: Run tools/test-setup.sh.
+- name: Run tools/test-setup.sh
   shell: tools/test-setup.sh
   args:
-    chdir: "{{ zuul_workspace_root }}/src/{{ zuul.project.canonical_name }}"
+    chdir: "src/{{ zuul.project.canonical_name }}"
   when:
     - p.stat.exists
     - p.stat.executable

roles/run-bindep/README.rst

@@ -0,0 +1,6 @@
+Installs distro packages using bindep tool
+
+Looks for a ``bindep.txt`` in a project's source directory, or failing
+that a ``other-requirements.txt``. If one exists, run ``bindep`` on the
+file to produce a list of required distro packages that do not exist and
+then install the missing packages.

roles/run-bindep/tasks/main.yaml

@@ -0,0 +1,69 @@
+---
+- name: Install distro packages from bindep
+  args:
+    chdir: "src/{{ zuul.project.canonical_name }}"
+  shell: |
+    #!/bin/bash -xe
+
+    # set a default path to the preinstalled bindep entrypoint
+    export BINDEP=${BINDEP:-/usr/bindep-env/bin/bindep}
+
+    function is_fedora {
+        [ -f /usr/bin/yum ] && cat /etc/*release | grep -q -e "Fedora"
+    }
+
+    YUM=yum
+    if is_fedora; then
+        YUM=dnf
+    fi
+
+    # figure out which bindep list to use
+    if [ -n "$PACKAGES" ] ; then
+        # already set in the calling environment
+        :
+    elif [ -e bindep.txt ] ; then
+        # project has its own bindep list
+        export PACKAGES=bindep.txt
+    elif [ -e other-requirements.txt ] ; then
+        # project has its own bindep list
+        export PACKAGES=other-requirements.txt
+    else
+        # use the bindep fallback list preinstalled on the worker
+        export PACKAGES=/usr/local/jenkins/common_data/bindep-fallback.txt
+    fi
+
+    # an install loop, retrying to check that all requested packages are
+    # obtained
+    try=0
+    # Install test profile using bindep
+    until $BINDEP -b -f $PACKAGES test; do
+        if [ $try -gt 2 ] ; then
+            set +x
+            echo -e "\nERROR: These requested packages were not installed:\n" \
+                "\n`$BINDEP -b -f $PACKAGES test`\n" 1>&2
+            set -x
+            exit 1
+        fi
+
+        # don't abort inside the loop, we check for the desired outcome
+        set +e
+        if apt-get -v >/dev/null 2>&1 ; then
+            sudo apt-get -qq update
+            sudo PATH=/usr/sbin:/sbin:$PATH DEBIAN_FRONTEND=noninteractive \
+                apt-get -q --option "Dpkg::Options::=--force-confold" \
+                --assume-yes install `$BINDEP -b -f $PACKAGES test`
+        elif emerge --version >/dev/null 2>&1 ; then
+            sudo emerge -uDNq --jobs=4 @world
+            sudo PATH=/usr/sbin:/sbin:$PATH emerge -q --jobs=4 \
+                `$BINDEP -b -f $PACKAGES test`
+        elif zypper --version >/dev/null 2>&1 ; then
+            sudo PATH=/usr/sbin:/sbin:$PATH zypper --non-interactive install \
+                `$BINDEP -b -f $PACKAGES test`
+        else
+            sudo PATH=/usr/sbin:/sbin:$PATH $YUM install -y \
+                `$BINDEP -b -f $PACKAGES test`
+        fi
+        set -e
+
+        try=$(( $try+1 ))
+    done