Make validate-host read from site-variables

Change the default parameters to the role to be zuul site variables.
Because of variable precedence, having these not be site variables means
someone could override them in a job. Since one of the actions is to
read and log the contents of a file, we likely don't want to give people
the ability to do that with an arbitrary file.

The traceroute host isn't as important to be a site variable, but it's
still not actually something that jobs should override - it's a feature
of the deployment.

Both variables work if they are not set, so deployers should still be
able to use this role without defining site-variables. But it should be
made clear to them that if they want those features they really should
define the locations in a site-variable and not in a normal job
variable.

configure-mirror similarly allows in-job override, but maybe that's ok
for now and leaving the site-variable value as a default is fine?

Finally, add a new zuul_site_image_manifest_files list, so that we can
specify more than one file to read. Set the defaults of it to be the
files that the dib nodepool elements emit. We'll also look in to pushing
those manifest files up a level into dib so that expecting nodepool
nodes to have them is even more reasonable.

Change-Id: I632a32fdfac4bfe57eb269ac8e183fb8df34d48f

roles/configure-mirrors/README.rst

@@ -3,6 +3,7 @@ An ansible role to configure services to use mirrors.
 **Role Variables**
 
 .. zuul:rolevar:: mirror_fqdn
+   :default: {{ zuul_site_mirror_fqdn }}
 
    The base host for mirror servers.
 

roles/configure-mirrors/defaults/main.yaml

@@ -1 +1,2 @@
+mirror_fqdn: "{{ zuul_site_mirror_fqdn|default(omit) }}"
 pypi_mirror: "http://{{ mirror_fqdn }}/pypi/simple"

roles/validate-host/README.rst

@@ -1 +1,32 @@
-Log information about the remote build host
+Log information about the build node
+
+**Role Variables**
+
+.. zuul:rolevar:: zuul_traceroute_host
+   :default: {{ zuul_site_traceroute_host }}
+
+   If defined, a host to run a traceroute against to verify build node
+   network connectivity.
+   **DEPRECATED** being replaced by zuul_site versions.
+
+.. zuul:rolevar:: zuul_image_manifest
+   :default: {{ zuul_site_image_manifest|default('/etc/dib-builddate.txt') }}
+
+   A file expected to be on the filesystem of the build node to read if it
+   exists and log. The default value comes from a site-variable called
+   ``zuul_site_image_manifest``, but if that is not set
+   ``/etc/dib-builddate.txt`` is used, which is written to nodes by
+   diskimage-builder in the ``nodepool-base`` element.
+   **DEPRECATED** being replaced by zuul_site versions.
+
+.. zuul:rolevar:: zuul_site_traceroute_host
+
+   If defined, a host to run a traceroute against to verify build node
+   network connectivity.
+
+.. zuul:rolevar:: zuul_site_image_manifest_files
+   :default: ['/etc/dib-builddate.txt', '/etc/image-hostname.txt']
+
+   A list of files to read from the filesystem of the build node and
+   whose contents will be logged. The default files are files written
+   to nodes by diskimage-builder.

roles/validate-host/defaults/main.yaml

@@ -0,0 +1,3 @@
+zuul_site_traceroute_host: "{{ zuul_traceroute_host|default(omit) }}"
+zuul_site_image_manifest: "{{ validate_host_default_image_manifest_files }}"
+zuul_site_image_manifest_files: "{{ zuul_site_image_manifest_files|default(validate_host_default_image_manifest_files) }}"

roles/validate-host/library/zuul_debug_info.py

@@ -42,16 +42,26 @@ def main():
     module = AnsibleModule(
         argument_spec=dict(
             image_manifest=dict(required=False, type='str'),
+            image_manifest_files=dict(required=False, type='list'),
             traceroute_host=dict(required=False, type='str'),
         )
     )
 
     image_manifest = module.params['image_manifest']
     traceroute_host = module.params['traceroute_host']
-    ret = {'image_manifest': None, 'traceroute': None}
+    image_manifest_files = module.params['image_manifest_files']
+    if not image_manifest_files and image_manifest:
+        image_manifest_files = [image_manifest]
+    ret = {'image_manifest_files': [], 'traceroute': None}
 
-    if image_manifest and os.path.exists(image_manifest):
-        ret['image_manifest'] = open(image_manifest, 'r').read()
+    for image_manifest in image_manifest_files:
+        if image_manifest and os.path.exists(image_manifest):
+            ret['image_manifest_files'].append({
+                'filename': image_manifest,
+                # Do this in python cause it's easier than in jinja2
+                'underline': len(image_manifest) * '-',
+                'content': open(image_manifest, 'r').read(),
+            })
     if traceroute_host:
         passed = False
         try:

roles/validate-host/tasks/main.yaml

@@ -21,8 +21,9 @@
 
 - name: Collect information about zuul worker
   zuul_debug_info:
-    image_manifest: "{{ zuul_image_manifest|default(omit) }}"
-    traceroute_host: "{{ zuul_traceroute_host|default(omit) }}"
+    image_manifest: "{{ zuul_site_image_manifest|default(omit) }}"
+    image_manifest_files: "{{ zuul_site_image_manifest_files|default(omit) }}"
+    traceroute_host: "{{ zuul_site_traceroute_host|default(omit) }}"
   register: zdi
 
 - name: Write out all zuul information for each host

roles/validate-host/templates/zuul-info.j2

@@ -1,46 +1,51 @@
-{% if 'image_manifest' in zdi %}
+{% if zdi.image_manifest_files %}
 Image Information
 =================
-{{ zdi.image_manifest }}
-{% endif %}
 
+{% for item in zdi.image_manifest_files %}
+{{ item.filename }}
+{{ item.underline }}
+{{ item.content }}
+{% endfor %}
+
+{% endif %}
 {% if 'uname' in zdi %}
 Host & kernel
 =============
 {{ zdi.uname }}
-{% endif %}
 
+{% endif %}
 {% if 'network_interfaces' in zdi %}
 Network interface addresses
 ===========================
 {{ zdi.network_interfaces }}
-{% endif %}
 
+{% endif %}
 {% if 'network_routing_v4' in zdi %}
 Network routing tables v4
 =========================
 {{ zdi.network_routing_v4 }}
-{% endif %}
 
+{% endif %}
 {% if 'network_routing_v6' in zdi %}
 Network routing tables v6
 =========================
 {{ zdi.network_routing_v6 }}
-{% endif %}
 
+{% endif %}
 {% if 'network_neighbors' in zdi %}
 Network neighbors
 =================
 {{ zdi.network_neighbors }}
-{% endif %}
 
+{% endif %}
 {% if 'traceroute_v4' in zdi %}
 Route to Known Host v4
 ======================
 Known Host: {{ zuul_traceroute_host }}
 {{ zdi.traceroute_v4 }}
-{% endif %}
 
+{% endif %}
 {% if 'traceroute_v6' in zdi %}
 Route to Known Host v6
 ======================

roles/validate-host/vars/main.yaml

@@ -0,0 +1,3 @@
+validate_host_default_image_manifest_files:
+  - /etc/dib-builddate.txt
+  - /etc/image-hostname.txt