Fix partial subunit stream logging

In a previous change I attempted to log in flight subunit streams that
may be left behind if a job times out. This unfortunately didn't work
because in the cases we leave those files behind we also assume that we
don't have a valid .stestr state due to the contents of this directly.

Address this by modifying the find-stestr.sh script to indicate via its
return code if .stestr or .testrepository are present separate from
which commands are valid to run. Then in ansible we can do steps that
always apply if (s)testr were used (like copying partial logs) and
separately do subunit parsing if the contents of the (s)testr db
directories are valid.

Change-Id: I1c8f2405d74484631f633065baf9764dbd0209ee

roles/fetch-subunit-output/files/find-testr.sh

@@ -28,17 +28,28 @@ zuul_work_dir=$1
 cd $HOME
 cd $zuul_work_dir
 
+# A non zero exit code means we didn't find any stestr or testr database
+# content. Additionally if the database content is parseable we emit on
+# stdout the commands which can be used to retrieve that parsed content.
+# This distiction is necessary as we want to take different actions based
+# on whether or not the database content is parseable.
+rc=1
 commands=""
+
 if [[ -d .testrepository ]] ; then
+    rc=0
     commands="testr ${commands}"
 fi
 
-# NOTE(mordred) Check for the failing file in the .stestr directory
-# nstead of just the directory. A stestr run that fails due to python
-# parsing errors will leave a directory but with no test results, which
-# will result in an error in the subunit generation phase.
-if [[ -f .stestr/failing ]] ; then
-    commands="stestr ${commands}"
+if [[ -d .stestr ]] ; then
+    rc=0
+    # NOTE(mordred) Check for the failing file in the .stestr directory
+    # instead of just the directory. A stestr run that fails due to python
+    # parsing errors will leave a directory but with no test results, which
+    # will result in an error in the subunit generation phase.
+    if [[ -f .stestr/failing ]] ; then
+        commands="stestr ${commands}"
+    fi
 fi
 
 # Add all the tox envs to the path so that type will work. Prefer tox
@@ -57,3 +68,4 @@ for command in $commands; do
         break
     fi
 done
+exit $rc

roles/fetch-subunit-output/tasks/main.yaml

@@ -4,15 +4,49 @@
 - name: Find stestr or testr executable
   script: "find-testr.sh {{ zuul_work_dir }}"
   register: testr_command
+  failed_when: false
 
 - when:
     - testr_command.rc == 0
-    - testr_command.stdout_lines
+  # Here we run steps that should apply whether or not there is a valid
+  # subunit stream present.
   block:
     - name: Get the list of directories with subunit files
       set_fact:
         all_subunit_dirs: "{{ [ zuul_work_dir ] + fetch_subunit_output_additional_dirs }}"
 
+    # If (s)testr was stopped early (possibly due to a timeout) it will "leak"
+    # a tmp file of the inflight subunit stream. Collect this as it is useful
+    # for debugging in these situations. Because it isn't a complete file
+    # we don't process it further.
+    - name: Find any inflight partial subunit files
+      find:
+        paths:
+          - "{{ zj_item }}/.testrepository"
+          - "{{ zj_item }}/.stestr"
+        patterns:
+          - 'tmp*'
+      register: partial_subunit_files
+      loop: "{{ all_subunit_dirs }}"
+      loop_control:
+        loop_var: zj_item
+
+    - name: Copy any inflight subunit files
+      copy:
+        dest: "{{ zuul_output_dir }}/logs/"
+        src: "{{ zj_item.path }}"
+        remote_src: true
+      with_items: "{{ partial_subunit_files.files }}"
+      loop_control:
+        loop_var: zj_item
+      when: partial_subunit_files.files is defined
+
+- when:
+    - testr_command.rc == 0
+    - testr_command.stdout_lines
+  # Here we run steps that only apply when there is a valid subunity stream.
+  # This is indicated through testr_command.stdout_lines content.
+  block:
     # The usage an independent target file instead of sending the output
     # to zuul_work_dir prevents issues related to zuul_work_dir being
     # a relative path, which may happen despite what the documentation
@@ -43,31 +77,5 @@
         state: absent
       failed_when: false
 
-    # If (s)testr was stopped early (possibly due to a timeout) it will "leak"
-    # a tmp file of the inflight subunit stream. Collect this as it is useful
-    # for debugging in these situations. Because it isn't a complete file
-    # we don't process it further.
-    - name: Find any inflight partial subunit files
-      find:
-        paths:
-          - "{{ zj_item }}/.testrepository"
-          - "{{ zj_item }}/.stestr"
-        patterns:
-          - 'tmp*'
-      register: partial_subunit_files
-      loop: "{{ all_subunit_dirs }}"
-      loop_control:
-        loop_var: zj_item
-
-    - name: Copy any inflight subunit files
-      copy:
-        dest: "{{ zuul_output_dir }}/logs/"
-        src: "{{ zj_item.path }}"
-        remote_src: true
-      with_items: "{{ partial_subunit_files.files }}"
-      loop_control:
-        loop_var: zj_item
-      when: partial_subunit_files.files is defined
-
     - name: Process and fetch subunit results
       include_tasks: process.yaml