Simplify Uploads by only sending Pass results

Change-Id: Iffa95584bd3417391704707811cc9ef21527dc60
storyboard: https://storyboard.openstack.org/#!/story/108

specs/approved/simplify-uploads-by-only-sending-pass-results.rst

@@ -0,0 +1,129 @@
+=============================================
+Simplify Uploads by only sending Pass results
+=============================================
+
+Blueprint: https://blueprints.launchpad.net/refstack/+spec/pass-only-uploads
+Storyboard: https://storyboard.openstack.org/#!/story/108
+
+As part of helping the community accept the publication of results, refstack
+test uploads should default to only PASS results. We are NOT uploading skips
+or errors. This aligns with the interop objective because we want have a
+positive assertion.
+
+Problem description
+===================
+
+Because fail results often include sensitive information it has been a pain
+point for some of our early adopters to upload results to a foundation
+controlled database. Since refstack really only cares about the things that
+pass, we'll just parse the results and leave the fails out.
+
+Proposed change
+===============
+
+This would involve using the subunit parsing code inside the tester. We would
+run the parser on the subunit before uploading the results to the api.
+
+We'll want to have a non-default option to send all data. Because I am sure
+that some folks will want to use refstack internally for real debugging and
+test regression.
+
+If a user tries to submit results to the public api server (Regardless of what
+the non-defaulting option is set to.). It will either produce an error (which I
+don't think is as useful) or it will just scrub the results anyways.
+
+Alternatives
+------------
+
+We could take the full upload and do the processing server side to accomplish
+the DefCore objective; however, that does not avoid the data leak issue.
+
+Data model impact
+-----------------
+
+None
+
+REST API impact
+---------------
+
+None
+
+Security impact
+---------------
+
+This change addresses security concerns. as far as I can tell it will not
+create any new ones.
+
+Notifications impact
+--------------------
+
+It should mention that results are being scrubbed in the on screen log messages.
+so that it's clear on every action.
+
+Other end user impact
+---------------------
+
+None
+
+Performance Impact
+------------------
+
+This will slow down the tester but not by a lot. However it will also speed up
+the upload of the results since they will be trimmed.
+
+Other deployer impact
+---------------------
+
+None
+
+Developer impact
+----------------
+
+This could potentially be a maintance problem moving forward as we move the
+subunit parsing utils into tempest then move the tester into its own repo.
+
+It will require that someone stays on top of things durring those changes to avoid
+duplication of code.
+
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+
+Primary assignee:
+  dlenwell
+
+Other contributors:
+  catherine wrote the parsing utils. So some support might be needed.
+
+Work Items
+----------
+
+The tester already has a stubbed out function for this code. Just needs
+to be filled in.
+
+Dependencies
+============
+
+None
+
+Testing
+=======
+
+None
+
+Documentation Impact
+====================
+
+It should be written down that we aren't uploading fails. So its known to
+operators and they don't have to be concerned about security leaks.
+
+But outside of that I don't see the need for documentation.
+
+
+References
+==========
+
+N/A