Create a preliminary flag guidelines list

This was discussed at the Flag.2 meeting [1] where the
included initial suggestions were made.  The intent is to
start a discussion using this patch about the flagging process
where we can establish clear and transparent criteria.

[1] https://etherpad.openstack.org/p/DefCoreFlag.2

Co-authored-by: Rob Hirschfeld <rob@zehicle.com>
Co-authored-by: Mark T. Voelker <mvoelker@vmware.com>
Co-authored-by: Jim Meyer <jim@geekdaily.org>

Change-Id: Ieacf8f8b4307658392a21f57cb1a2031e3573383

HACKING.rst

@@ -3,7 +3,12 @@ DefCore Style Commandments
 
 - Step 1: Read the OpenStack Style Commandments
   http://docs.openstack.org/developer/hacking/
-- Step 2: Read on
+- Step 2: Read the following DefCore process documents in the following recommended order:
+
+  - `Core Definition <doc/source/process/CoreDefinition.rst>`_
+  - `OpenStack DefCore Process 2015A <doc/source/process/2015A.rst>`_
+
+- Step 3: Read on
 
 DefCore Specific Commandments
 -----------------------------
@@ -18,18 +23,18 @@ DefCore Specific Commandments
   in an approved capability list, add a "flagged" block to the test.  Do
   _not_ remove it from the "tests" section.
 - [D302] If a Capability is found to not meet the `Core Criteria
-  <./process/CoreCriteria.rst>`_ after the Board has approved a Guideline,
+  <doc/source/process/CoreCriteria.rst>`_ after the Board has approved
-  the corresponding tests should have a "flagged" block added to the
+  a Guideline, the corresponding tests should have a "flagged" block added
-  the relevant tests in the "tests" section of the relevant Board-approved
+  to the the relevant tests in the "tests" section of the relevant
-  Guidelines.  The tests should also be removed from the "tests" section of
+  Board-approved Guidelines.
-  the .next.json file.
+
-    - See [D307] for details about the flagged block
+    - See [D307] and [D309] for details about format requirements.
+    - See [D308] for conditions on also adding to the .next.json.
+
 - [D303] Tests that are found to inadequately test the underlying
   Capability due to bugs or design flaws, should have a "flagged"
   block added to the section for the test in the "tests" section of
   the most recent Board-approved Guideline.
-    - See [D307] for details about the flagged block
-    - See [D308] for conditions on also adding to the .next.json.
 - [D304] Before the Board approves the capabilities listed in the
   .next.json file, a committee member will submit a patch that copies
   the .next.json file to an appropriately named new file, updates the
@@ -54,3 +59,46 @@ DefCore Specific Commandments
   to be resolved before the next Guideline is submitted to tbe Board
   for approval, then matching entries should also be made in the
   .next.json Guideline.
+- [D309] The "reason" field of the "flagged" section must begin with the
+  flag type. For example:
+
+  ``"reason" : "[D400] The Foo test doesn't meet DefCore criteria because ..."``
+
+- [D310] If you believe a test needs to be flagged but the reason for doing
+  so doesn't appear in the list below, you must do the following:
+
+  #. Submit the test for flagging using [D404] as the flag type. Please also
+     provide the reason you believe this test needs to be flagged; see [D309]
+     above for details. [D404] indicates that you are uncertain about which
+     flag to use or believe that a new reason for considering flags needs to be
+     discussed. *[D404] is only a placeholder to facilitate this discussion.*
+  #. In that same proposed change, also submit a patch against the HACKING
+     file adding your proposed new flag to the list below.
+  #. If at all possible, please include a link to code and/or test runs which
+     demonstrate the reason a new flag type is needed.
+  #. The DefCore committee will discuss and consider the flagging proposal as
+     well as the proposed new reason. They may accept or decline either proposal.
+
+
+DefCore Test Flagging Guidelines
+--------------------------------
+
+The DefCore Committee may "flag" tests to mark them as not required for a
+given Guideline. There are different flag types; each flag type indicates a
+fairly narrow category of reasons for flagging a given test.
+
+Valid reasons for flagging a test are limited to the following:
+
+- [D400] The test is for a Capability that fails to meet DefCore Criteria
+  as set out in the
+  `Core Criteria document <doc/source/process/CoreCriteria.rst>`_.
+- [D401] The test fails or is skipped due to a bug in the test and the bug is
+  accepted by the OpenStack project which maintains the test.
+- [D402] The test fails or is skipped due to a bug in the code that provides
+  the Capability and the bug is accepted by the OpenStack project which
+  maintains the Capability.
+- [D403] The test fails because other non-DefCore Capabilities are also
+  tested.
+- [D404] Flag Not Found - Use this flag if none of the others fit.
+- [D405] The test reflects an implementation choice that is not widely
+  deployed even if the Capability is widely deployed.