openinfra/interop
Code Issues Proposed changes

Merge "Change doc references to DefCore"

Browse Source
This commit is contained in:
Jenkins 2017-01-24 04:07:23 +00:00 committed by Gerrit Code Review
commit 4a057117da
20 changed files with 503 additions and 137 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

48
HACKING.rst
View File

@ -1,22 +1,23 @@
DefCore Style Commandments
==========================
Interop Working Group Commandments
===================================
- Step 1: Read the OpenStack Style Commandments
http://docs.openstack.org/developer/hacking/
- Step 2: Read the following DefCore process documents in the following recommended order:
- Step 2: Read the following Interop Working Group process
documents in the following recommended order:
- `Core Definition <doc/source/process/CoreDefinition.rst>`_
- `OpenStack DefCore Process 2015A <doc/source/process/2015A.rst>`_
- `OpenStack Interop WG Process 2016A <doc/source/process/2016A.rst>`_
- Step 3: Read on
DefCore Specific Commandments
-----------------------------
Interop Working Group Specific Commandments
--------------------------------------------
- [D300] When adding tests to "flagged" lists, generally only the most
current Board-approved .json file and the .next.json file should be
modified. There is no need to modify older guidelines unless the most
current Board-approved guideline doesn't cover the OpenStack release
modified. There is no need to modify older Guidelines unless the most
current Board-approved Guideline doesn't cover the OpenStack release
you are concerned with.
- [D301] The "tests" lists in the .json capabilities lists are immutable
once approved by the Board. Therefore if you desire to flag a test,
@ -42,9 +43,9 @@ DefCore Specific Commandments
field within newly required capabilities. The patch should include the
matching generated RST version of the JSON file. This patch should be
marked as -1 workflow until after approval.
- [D305] DefCore guidelines generally cover the most recent three
releases of OpenStack, though the DefCore Committee has the power to
determine otherwise. The "releases" section of the .next.json file
- [D305] Interop Guidelines generally cover the most recent three
releases of OpenStack, though the Interop Working Group has the
power to determine otherwise. The "releases" section of the .next.json file
should generally be updated shortly after the Board approves a release
so that contributors can see what releases the proposed Guideline
targets.
@ -62,7 +63,7 @@ DefCore Specific Commandments
- [D309] The "reason" field of the "flagged" section must begin with the
flag type. For example:
``"reason" : "[D400] The Foo test doesn't meet DefCore criteria because ..."``
``"reason" : "[D400] The Foo test doesn't meet Core Criteria because ..."``
- [D310] If you believe a test needs to be flagged but the reason for doing
so doesn't appear in the list below, you must do the following:
@ -76,22 +77,23 @@ DefCore Specific Commandments
file adding your proposed new flag to the list below.
#. If at all possible, please include a link to code and/or test runs which
demonstrate the reason a new flag type is needed.
#. The DefCore committee will discuss and consider the flagging proposal as
well as the proposed new reason. They may accept or decline either proposal.
#. The Interop Working Group will discuss and consider the flagging
proposal as well as the proposed new reason. They may accept or decline
either proposal.
- [D311] Once a test has been flagged, it will remain flagged for that Guideline.
- [D312] When a new guideline is proposed for Board approval, no flagged tests
will be included in the guideline. Flags will be added in subsequent patches.
- [D312] When a new Guideline is proposed for Board approval, no flagged tests
will be included in the Guideline. Flags will be added in subsequent patches.
DefCore Test Flagging Guidelines
--------------------------------
Interop Working Group Test Flagging Guidelines
-----------------------------------------------
The DefCore Committee may "flag" tests to mark them as not required for a
given Guideline. There are different flag types; each flag type indicates a
fairly narrow category of reasons for flagging a given test.
The Interoper Working Group may "flag" tests to mark them as not
required for a given Guideline. There are different flag types; each flag
type indicates a fairly narrow category of reasons for flagging a given test.
Valid reasons for flagging a test are limited to the following:
- [D400] The test is for a Capability that fails to meet DefCore Criteria
- [D400] The test is for a Capability that fails to meet the Criteria
as set out in the
`Core Criteria document <doc/source/process/CoreCriteria.rst>`_.
- [D401] The test fails or is skipped due to a bug in the test and the bug is
@ -99,7 +101,7 @@ Valid reasons for flagging a test are limited to the following:
- [D402] The test fails or is skipped due to a bug in the code that provides
the Capability and the bug is accepted by the OpenStack project which
maintains the Capability.
- [D403] The test fails because other non-DefCore Capabilities are also
- [D403] The test fails because other non-required Capabilities are also
tested.
- [D404] Flag Not Found - Use this flag if none of the others fit.
- [D405] The test reflects an implementation choice that is not widely

27
README.rst
View File

@ -1,32 +1,35 @@
=================================================
Understanding the DefCore Guidelines
Understanding the Interoperability Guidelines
=================================================
This repository contains DefCore committee managed files that provide guidance
for the OpenStack community.
This repository contains files managed by the Interop Working
Group that provide guidance for the OpenStack community.
NOTE: Changes to file requires approval of the DefCore committee chair(s).
NOTE: Changes to file requires approval of the Interop Working
Group chair(s).
DefCore Process Documentation
=============================
Interop Working Group Process Documentation
============================================
Want to certify as a vendor? Consult `OpenStack Interop
Are you a vendor who wants to get a license to use the OpenStack trademark
and logo? Consult `OpenStack Interop
<http://www.openstack.org/brand/interop/>`_.
The /doc/source/process directory contains details about the DefCore process.
The /doc/source/process directory contains details about the
Interop Working Group process.
The /doc/source/schema directory contains details about the JSON schema
versions used to express Guidelines.
The /doc/source/guidelines directory contains RST versions of the DefCore
Guidelines approved by the OpenStack Board of Directors.
The /doc/source/guidelines directory contains RST versions of the
Interop Guidelines approved by the OpenStack Board of Directors.
:Core Definition: doc/source/process/CoreDefinition.rst
:Process Goverance: doc/source/process/2015A.rst (please check for latest)
:Designated Sections: doc/source/process/DesignatedSections.rst
:Core Criteria: doc/source/process/CoreCriteria.rst
:DefCore Governance: doc/source/process/GovernanceProcess.rst
:Interop WG Governance: doc/source/process/GovernanceProcess.rst
:Platform and Components: doc/source/process/PlatformCap.rst
:DefCore Cycles: doc/source/process/ProcessCycles.rst
:Interop WG Cycles: doc/source/process/ProcessCycles.rst
:Terminology: doc/source/process/Lexicon.rst

2
doc/source/conf.py
View File

@ -184,7 +184,7 @@ latex_elements = {
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'defcore.tex', u'OpenStack DefCore Documentation',
('index', 'defcore.tex', u'OpenStack Interop WG Documentation',
u'OpenStack Foundation', 'manual'),
]

10
doc/source/guidelines/next.rst
View File

@ -1,9 +1,9 @@
======================
OpenStack DefCore next
======================
=========================================
OpenStack Interoperability Guideline next
=========================================
:Status: draft
:Replaces: 2016.01
:Replaces: 2016.08
:JSON Master: http://git.openstack.org/cgit/openstack/defcore/tree/next.json
This document outlines the mandatory capabilities and designated
@ -14,7 +14,7 @@ This document was generated from the `master JSON version <next.json>`_.
Releases Covered
==============================
Applies to Kilo, Liberty, Mitaka, Newton
Applies to Liberty, Mitaka, Newton, Ocata
Platform Components
==============================

314
doc/source/process/2017A.rst Normal file
View File

@ -0,0 +1,314 @@
OpenStack Interop Working Group Process 2017A
==============================================
:Status: Draft
:Replaces: 2016A
This document describes the Interop Working Group's working
process required by the OpenStack bylaws and approved by the OpenStack
Technical Committee and Board.
Expected Time line:
---------------------------------------
+------------+-----------+--------------------------------------+-----------+
| Time Frame | Milestone | Activities | Lead By |
+============+===========+======================================+===========+
| -3 months | S-3 | Draft status | Interop WG|
+------------+-----------+--------------------------------------+-----------+
| -2 months | S-2 | ID new Capabilities | Community |
+------------+-----------+--------------------------------------+-----------+
| -1 month | S-1 | Score Capabilities | Interop WG|
+------------+-----------+--------------------------------------+-----------+
| Summit | S | Review status | Community |
+ + +--------------------------------------+-----------+
| | | Advisory/Deprecated items selected | Interop WG|
+------------+-----------+--------------------------------------+-----------+
| +1 month | S+1 | Self-testing | Vendors |
+------------+-----------+--------------------------------------+-----------+
| +2 months | S+2 | Test Flagging | Interop WG|
+------------+-----------+--------------------------------------+-----------+
| +3 months | S+3 | Approve Guidance | Board |
+------------+-----------+--------------------------------------+-----------+
Note: The Interop Working Group may accelerate the process to correct errors
and omissions.
Process Definition
--------------------------------------
The Guideline process has two primary phases: Draft and Review.
During the Draft phase (A), the Interop Working Group is working
with community leaders to update and score the components of the Guideline.
During the Review phase (B), the general community and vendors have an
opportunity to provide input and check the Guidelines (C) against actual
implementations. The Review phase ends with Board approval of the draft
Guideline (D).
This section provides specific rules and structure for each phase.
NOTE: To ensure continuity of discussion, process components defined below
must _not_ reuse numbers in future revisions. The numbering pattern
follows draft, section and sub-item numbering, e.g.: 2015A.B2.2. This
requirement may create numbering gaps in future iterations that will help
indicate changes.
Guidelines Draft Phase (A)
^^^^^^^^^^^^^^^^^^^^^^^^^^
Starting: S-3
A1. New Guidelines Start From Previous Guidelines
1. New Guidelines start from the previous Board approved document.
2. New Guidelines are given the preliminary name of "next.json".
A2. Community Groups Tests into Capabilities
1. The Interop Working Group coordinates community
activities with the Technical Leadership to revise the capabilities
based on current technical needs and functionality.
2. Capabilities must correspond to projects which are part of the
"TC-approved release" as designated by the TC (see bylaws of the
Foundation, section 4.1(b)(iii)).
3. Groupings may change between iterations.
4. Tests must have unique identifiers that are durable across releases
and changes in grouping.
5. Tests must be under OpenStack Technical Committee governance.
6. The Interop Working Group will provide the test groupings
in JSON format for scoring.
7. The Interop Working Group will provide a human-readable summary
of the Guideline generated from the JSON version.
A3. Interop Working Group Collects Recommendations for
Designated Sections
1. Designated Sections will not be removed without being deprecated in the
previous Guideline.
2. Designated Sections will not be added without being advisory in the
previous Guideline.
3. Designated Sections will not be added or be made advisory unless the
corresponding code base is designated as part of the "TC-approved release"
by the Technical Committee (see bylaws of the Foundation, section
4.1(b)(iii)).
4. Technical leadership may, but is not required to, assist the
Interop Working Group with defining advisory Designated
Sections for projects that have advisory or required Capabilities.
5. Designated Sections may be sufficiently defined for Guidelines using
general descriptions.
6. The Interop Working Group will present A3.4 descriptions to
the Board for approval.
7. Technical leadership may, but is not required to, provide more specific
details describing the Designated Sections for a project.
8. Designated Sections will be included in the JSON Guideline.
A4. Interop Working Group identifies required capabilities
1. The Interop Working Group uses Board approved scoring
criteria to evaluate Capabilities.
2. The Interop Working Group needs Board approval to change
scoring criteria.
3. Scoring criteria factors or weights cannot change after Draft is
published.
4. The Interop Working Group identifies a cut-off score for
determining that a Capability is required.
5. Capabilities will not be removed without being deprecated in the
previous Guideline.
6. Capabilities will not be added without being advisory in the previous
Guideline.
7. For the "widely deployed" adoption criteria, the size of the
pool being considered will match the scope of the community being
considered. Capabilities will be evaluated based on their use in their
Component. Components will be evaluated based on their use in the
Platform.
A5. Foundation Staff recommends OpenStack Components and OpenStack Platform
Scope
1. Foundation Staff recommends Capabilities to include in each OpenStack
Component.
2. Foundation Staff recommends which Components are required for
the OpenStack Powered Platform.
3. To support the Foundation recommendation, the Interop
Working Group will apply the approved scoring criteria to
evaluate if a component should be included in the
Platform (see A4).
A6. Additional Capabilities and Tests
1. The Interop Working Group will work with the community to
define new Capabilities.
2. Test grouping for new Capabilities will be included in the
Interop Working Group documents.
3. The Interop Working Group will publish a list of missing
Capabilities and Capabilities with inadequate test coverage.
A7. The Interop Working Group creates recommendation for Draft.
1. The Interoper Working Group coordinates activities to create
the Guideline draft.
2. The Interop Working Group may choose to ignore recommendations
with documented justification.
Guidelines Review Phase (B)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Starting: Summit
B1. All Reference Artifacts are reviewed via Gerrit
1. Draft Guideline
2. Designated sections
3. Test-Capability groupings
4. Flagged Test List
5. Capability Scoring criteria and weights
6. May not be in Gerrit: Working materials (spreadsheets, etc)
B2. Presentation of Draft Guidelines for Review
1. The Interop Working Group will present Draft Guidelines to
the Board for review.
2. The Interop Working Group will distribute Draft Guidelines
to the community for review.
3. Foundation Staff will provide Draft Guidelines to vendors for review.
4. A link to the Gerrit document must be provided with the review materials.
B3. Changes to Guideline made by Gerrit Review Process
1. Community discussion including vendors must go through Gerrit.
2. All changes to draft must go through Gerrit process.
3. The Interop Working Group will proxy for users who do not
participate in the Gerrit process with attribution.
B4. For Gerrit reviews, Interop Working Group Co-Chairs act as
Joint PTLs
1. Interop Working Group Co-Chairs serve as "core" reviewers (+2).
2. Requests for changes must be submitted as patches by the requesting
party.
3. Interop Working Group members may proxy change requests as
long as the requesting party is explicitly acknowledged.
4. One Interop Working Group Co-Chair must be Board member.
5. One Interop Working Group Co-Chair will be elected by the
Interop Working Group. Election quorum is composed of
attendees present during the election meeting.
6. Additional core reviewers (+2) can be appointed by Co-Chairs.
7. Election meetings must be posted at least one meeting prior.
Community Review & Vendor Self-Test (C)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Starting: S and continues past S+3
C1. Vendor Self-Tests
1. Vendors are responsible for executing tests identified by the
Interop Working Group.
2. The Foundation may, but is not required to, provide tooling for
running tests.
3. The Foundation may, but is not required to, define a required
reporting format.
4. Self-test results may be published by Vendors in advance of Foundation
review, but must be clearly labeled as "Unofficial Results - Not Yet
Accepted By The OpenStack Foundation".
5. Vendors who publish self-tests MUST provide them in the same format that
would be submitted to the OpenStack Foundation but MAY provide additional
formats if they choose to do so.
6. Self-test results cannot be used as proof of compliance.
C2. Vendor submits results to Foundation for review
1. The Foundation determines the acceptable format for submissions.
2. The Foundation has final authority to determine if Vendor meets
criteria.
3. The Foundation will provide a review of the results within 30 days.
C3. Vendor Grievance Process
1. Vendors may raise concerns with specific tests to the Interop
Working Group.
2. The Interop Working Group may choose to remove tests from
a Guideline (known as flagging).
3. The Interop Working Group will acknowledge vendor requests
to flag tests within 30 days.
C4. Results of Vendor Self-Tests will be open
1. The Foundation will make the final results of approved vendors
available to the community.
2. The Foundation will not publish incomplete or unapproved results.
3. Only "pass" results will be reported. Skipped and failed results will
be omitted from the reports.
4. Reports will include individual test results, not just Capability
scoring.
5. Vendors are required to submit a description of the system and
configuration used to achieve the results.
6. The Foundation may require vendors to submit specific details of the
configuration and may also require use of a specific format for
reporting.
C5. API Usage Data Report
1. The Foundation will provide Interop Working Group with an
open report about API usage based on self-tests.
2. To the extent the data is available, Capabilities beyond the
Interoperability Guideline list will be included in the report.
C6. Only Two Approved Guidelines at a time:
1. Vendors seeking Foundation validation are limited to using the two
latest approved Guidelines.
2. Since past validations are respected, older Guidelines will be
maintained as superseded for historical reference.
3. Guideline status progresses as follows:
:draft: initial work, pre-summit (S-3) discussion material
:review: as presented at summit (S) for community review
:approved: board approved, one of the two official guidelines
:superseded: board approved, now superseded by two latest guidelines
Guideline Approval (D)
^^^^^^^^^^^^^^^^^^^^^^
Starting: S+3
D1. Board will review and approve Interoperability Guidelines from draft
1. Guidelines are set at the Platform, Component and Capability level
only.
2. The Interop Working Group will submit the human-readable
summary of Capabilities (see section A2[6]) to the Board for approval.
3. By voting to approve the summary, the Board delegates responsibility
for maintaining test groupings to the Interop Working Group
subject to the limitations described in section D2.
4. Guidelines only apply to the identified releases (a.k.a. release
tags).
D2. Interop Working Group has authority on test categorization
1. The Interop Working Group can add flagged tests before and
after Guideline approval.
2. The Interop Working Group cannot add additional Tests to
Capability mappings after approval.
3. The Interop Working Group maintains the test to Capability
mappings in the JSON representation.
D3. Designated Sections only enforced for projects with required Capabilities
1. Designated Sections may be defined for any project.
2. Designated Sections apply to the releases (a.k.a. release tags)
identified in the Guideline.
3. Designated Sections will be included in the JSON Capabilities file
to ensure a single source of identification.
D4. Guidelines are named based on the date of Board approval
1. Naming pattern will be: 4-digit year, dot (period), and 2-digit month.
Functional Information
----------------------
:Format: RestructuredText
:Layout: 1.0

23
doc/source/process/CoreCriteria.rst
View File

@ -19,8 +19,8 @@ categories:
:System: the capability integrates with other parts of OpenStack
These categories summarize critical values that we want in OpenStack and so
make sense to be the primary factors used when we select DefCore capabilities.
While we strive to make the DefCore process objective and quantitive, we
make sense to be the primary factors used when we select required Capabilities.
While we strive to make the process objective and quantitive, we
must recognize that these choices drive community behavior.
.. figure:: ../images/Defcore_capabilities_criteria.png
@ -60,11 +60,11 @@ ALIGNS WITH TECHNICAL DIRECTION
capabilities.
* **"Stable"** Capabilities are required to be stable for >2 releases because
we do not want DefCore capabilities that do not have dependable APIs.
we do not want required Capabilities that do not have dependable APIs.
* **"Complete"** Where the code being tested has a designated area of alternate
implementation (extension framework) as per the DefCore Principles, there
should be parity in capability tested across extension implementations.
implementation (extension framework) as per the Interoperability Principles,
there should be parity in capability tested across extension implementations.
This also implies that the capability test is not configuration specific
or locked to non-open technology.
@ -78,9 +78,9 @@ PLAYS WELL WITH OTHERS
This can be a very subjective measure and we expect to refine this
definition over time.
* **"DefCore in Last Release"** A required capability should stay a required
capability. This makes DefCore capabilities sticky per release. Leaving
DefCore is disruptive to the ecosystem.
* **"Required in Last Release"** A required capability should stay a required
capability. This makes Capabilities sticky per release. Leaving
the Interoperability Guidelines is disruptive to the ecosystem.
TAKES A SYSTEM VIEW
-------------------
@ -92,11 +92,12 @@ TAKES A SYSTEM VIEW
required capabilities.
* **"Proximity"** (sometimes called a Capability Cluster) selects for
capabilities that are related to DefCore capabilities. This helps ensure
that related capabilities are managed together.
capabilities that are related to other required capabilities. This helps
ensure that related capabilities are managed together.
NON-ADMIN
---------
The original 13th "non-admin" criteria has been removed because Admin
APIs cannot be used for interoperability and are not considered DefCore.
APIs cannot be used for interoperability and are not considered for end
users.

4
doc/source/process/CoreDefinition.rst
View File

@ -19,8 +19,8 @@ suggest changes to the by-laws to clarify the definition of core.
Implementation
==============
* The `Governance/DefCoreCommittee
<https://wiki.openstack.org/wiki/Governance/DefCoreCommittee/>`_ is
* The `Governance/InteropWG
<https://wiki.openstack.org/wiki/Governance/InteropWG/>`_ is
working to manage this.
* Meetings and agendas will be posted to that page, hosted on Google
Hangouts and (generally) open to the community.

10
doc/source/process/DesignatedSections.rst
View File

@ -16,9 +16,9 @@ Designated Sections Selection Guidance
_Approved 2014 Dec 2_
There DefCore committee identified 10 selection criteria. The first seven are
technical from the TC and last three allow the Board to resolve issues without
needed a technical judgement.
The Interop Working Group identified 10 selection criteria. The first
seven are technical from the TC and last three allow the Board to resolve
issues without needing a technical judgement.
1. Designated if the code provides the project external REST API
@ -64,13 +64,13 @@ needed a technical judgement.
2. The goal is guidance on where we want upstream contributions not a
code inspection police state. Guidance will be revised per release
as part of the DefCore process.
as part of the Interop Working Group process.
Designated Sections
===================
Effective April 2015, approved Designated Sections are maintained
in the Board approved DefCore Guidelines. The 2015.03 Guideline
in the Board approved Interoperability Guidelines. The 2015.03 Guideline
was set to match the Board action of 2014 December 2.
Please see the current Guidelines to determine which Designated

8
doc/source/process/GovernanceProcess.rst
View File

@ -4,13 +4,13 @@ Governance Process
* Meetings will be open to all community members. Information about how
and when to join meetings will be posted on the
defcore-committee@lists.openstack.org mailing list and/or the
`DefCore Committee wiki page
<https://wiki.openstack.org/wiki/Governance/DefCoreCommittee>`_.
`Interop WG wiki page
<https://wiki.openstack.org/wiki/Governance/InteropWG>`_.
* Members are expected to to their homework. We will not be rehashing
due to time limits. Minutes from prior meetings will be posted to the
`DefCore Committee wiki page
<https://wiki.openstack.org/wiki/Governance/DefCoreCommittee>`_.
`Interop WG wiki page
<https://wiki.openstack.org/wiki/Governance/InteropWG>`_.
* Process documents and other "rules of the road" will be maintained and
voted on in `Gerrit

55
doc/source/process/Lexicon.rst
View File

@ -1,9 +1,9 @@
OpenStack DefCore Lexicon
=========================================
OpenStack Interop Working Group Lexicon
========================================
Licensing the OpenStack commercial-use marks requires passing tests of
DefCore required capabilities, and including designated sections of code.
required capabilities, and including designated sections of code.
There are multiple marks available for vendors depending on which
capability groupings are passed.
@ -11,17 +11,16 @@ TERMS:
----------------------------------------
Advisory
Capabilities that have been suggested for the next guideline.
Capabilities that have been suggested for the next Guideline.
Capability
The functionality ensured by a set of tests collected into
a group as defined by the DefCore committee Required - capabilities that
are required to meet the guideline.
a group as defined by the Interop Working Group.
Certify or Accredit
DefCore does not do any of these things for OpenStack clouds. These
actions would fall under the governance of the Foundation trademark
policy.
The Interop Working Group does not do any of these things
for OpenStack clouds. These actions would fall under the
governance of the Foundation trademark policy.
Community
The universe of people and companies that are involved in the OpenStack
@ -41,25 +40,23 @@ Core
uses.
DefCore
The OpenStack board committee that manages commercial definition
of OpenStack for trademark purposes.
DefCore Process
The process used by DefCore to score capabilities and
select criteria.
The original name for the OpenStack board committee that managed
commercial definition of OpenStack for trademark purposes. As the
group evolved, it's name was eventually changed to the Interop
Working Group.
Deprecated
Capabilities that will be removed in the next guideline.
Capabilities that will be removed in the next Guideline.
Designated Sections
Portions of the OpenStack codebase that must be used to provide
required capabilities in a product wishing to use the OpenStack
trademark. Designated sections fulfill one or more of the following
trademark. Designated sections fulfill one or more of the following
criteria: they provide the project-external REST API, or are shared
and provide common functionality for all options, or implement logic
that is critical for cross-platform operation. Designated sections
that is critical for cross-platform operation. Designated sections
must exist in the OpenStack gerrit namespace and have corresponding
tests. Code that meets the following criteria will not be considered
tests. Code that meets the following criteria will not be considered
designated: provides vendor-specific functionality, are explicitly
intended by the project maintainers to be replaceable, extend the
project REST API in a new or different way, or code that is being
@ -70,9 +67,13 @@ Flagged Test
field and it not required for vendor self-test.
Guidelines
Output of the DefCore process detailing which sections and
capabilities are required. Guidelines will be approved on a regular
cadence and identified by the date of approval.
Output of the Interop Working Group process detailing which
Designated Sections and Capabilities are required. Guidelines will be
approved on a regular cadence and identified by the date of approval.
Interop Working Group Process
The process used by the Interop Working Group to score Capabilities
and select Criteria.
OpenStack Mark
Right granted by the OpenStack Foundation to use the name and logo of
@ -82,8 +83,9 @@ Participant
The subset of the Community that actively engages in creating
components of OpenStack including, but not limited to, the code,
documentation, training, product management and other materials.
For DefCore purposes, Participant is not limited to the community
members identified as "ATC" as per http://git.openstack.org/cgit/openstack/governance/tree/reference/charter.rst#n132
For Interop Working Group purposes, Participant is not limited to
the community members identified as "ATC" as per
http://git.openstack.org/cgit/openstack/governance/tree/reference/charter.rst#n132
(see also Technical Leadership)
Platform
@ -91,7 +93,10 @@ Platform
Removed
Capabilities that are no longer required and are not included in the
current guideline.
current Guideline.
Required - Capabilities that are required to be exposed to end users to
satisfy the requirements of an Interoperability Guideline.
Self-test
Process by which a vendor runs tests against their product or service

4
doc/source/process/PlatformCap.rst
View File

@ -9,13 +9,13 @@ Platform and Program Capabilities
The following was approved by the OpenStack Board.
Extend the DefCore principles to allow for multiple levels: programs and
Extend the Interop WG principles to allow for multiple levels: programs and
platforms. Programs represent subsections of the overall platform. In some
cases, it is acceptable for a program identified without being included in
the platform. New programs are added at Foundation recommendation via board
approval. Programs are added to the platform via board approval.
The initial programs will be Compute & Object. The DefCore platform will
The initial programs will be Compute & Object. The Platform will
require the Compute program, Object program and additional capabilities.
Havana Approved: The Compute Program will consist of the following

8
doc/source/process/ProcessCycles.rst
View File

@ -5,9 +5,9 @@ Defining OpenStack Core is a long term process and we are doing the work
in progressive cycles. For reference, we have named the cycles. This
helps describe concrete deliverables for a cycle while allowing
discussion of the broader long term issues. For example, we may say that
"item X is important to DefCore but out of scope for Elephant." We have
found that this approach to breaking down the problem is necessary to
maintain community consensus because we are taking smaller bites of the
"item X is important to interoperability but out of scope for Elephant."
We have found that this approach to breaking down the problem is necessary
to maintain community consensus because we are taking smaller bites of the
larger challenge (aka eating the elephant).
Spider (Fall 2013)
@ -167,7 +167,7 @@ Meetings
~~~~~~~~
Meeting information for the Flag cycle will be archived here once the cycle
is complete. During the cycle, please refer to the
`DefCore Committee wiki page <https://wiki.openstack.org/wiki/Governance/DefCoreCommittee/>`_.
`Interop WG wiki page <https://wiki.openstack.org/wiki/Governance/InteropWG/>`_.
Future
------

4
doc/source/schema/1.6.rst
View File

@ -1,5 +1,5 @@
DefCore Schema v1.6 Change Log
==============================
Interop WG Schema v1.6 Change Log
==================================
Changes from v1.5

4
next.json
View File

@ -3270,12 +3270,12 @@
"weight": 8
},
"stable": {
"Description": "Test is required stable for >2 releases because we don't want DefCore capabilities that do not have dependable APIs",
"Description": "Test is required stable for >2 releases because we don't want Capabilities that do not have dependable APIs",
"name": "Stable",
"weight": 9
},
"sticky": {
"Description": "A test that is a must-pass test should stay a must-pass test. This makes DefCore capabilities sticky release per release. Leaving Core is disruptive to the ecosystem",
"Description": "A test that is a must-pass test should stay a must-pass test. This makes Capabilities sticky release per release. Leaving Core is disruptive to the ecosystem",
"name": "Core in Last Release",
"weight": 9
},

6
setup.cfg
View File

@ -1,11 +1,11 @@
[metadata]
name = defcore
summary = OpenStack DefCore Process
summary = OpenStack Interop Working Group Process
description-file =
README.rst
author = OpenStack
author-email = defcore-committee@lists.openstack.org
home-page = https://wiki.openstack.org/wiki/Governance/DefCoreCommittee
author-email = interop-wg@lists.openstack.org
home-page = https://wiki.openstack.org/wiki/Governance/InteropWG
classifier =
Environment :: OpenStack
Intended Audience :: Information Technology

4
tools/jsonToRst.py
View File

@ -64,7 +64,7 @@ with open(outFileName, "w") as outFile:
print 'Make sure there is a valid id'
sys.exit(1)
line01 = "OpenStack DefCore %s" % data["id"]
line01 = "OpenStack Interoperability Guideline %s" % data["id"]
outFile.write('=' * len(line01) + '\n')
outFile.write(line01 + '\n')
@ -78,7 +78,7 @@ with open(outFileName, "w") as outFile:
# Correct Source
if data.get('source') != \
'http://git.openstack.org/cgit/openstack/defcore/':
print "The expected DefCore source not found"
print "The expected interoperability guideline source not found"
sys.exit(1)
outFile.write("""

34
working_materials/additional_properties_waiver.rst
View File

@ -15,14 +15,13 @@ Nova calls as part of tempest-lib.
Prior to this change, clouds running the Nova 2.0 API could take
advantage of a mechanism to add extensions to the Nova endpoint, and
some also sent additional data back in their Nova responses. These clouds
passed interoperability testing when the DefCore interoperability testing
program was launched for the OpenStack Powered program. After strict
response checking was added to Tempest, these clouds failed interop
testing.
passed interoperability testing when the OpenStack Powered program was
launched. After strict response checking was added to Tempest, these
clouds failed interop testing.
To address this issue, and the challenges vendors have in updating their
products to match upstream API changes, this proposal offers a means for
vendors to pass the DefCore interoperability tests while proving
vendors to pass the interoperability tests while proving
compatibility for required capabilities.
There is a natural tension between the forward motion of upstream
@ -41,10 +40,10 @@ products include, but are not limited to:
#. Remaining on the Nova v/2.0 API, which has been removed from the
Newton release of OpenStack
This waiver program will cover the 2015.07, 2016.01, and 2016.08 DefCore
guidelines, and give downstream vendors a year to work internally
and within the ecosystem to update their products before re-verifying
their products.
This waiver program will cover the 2015.07, 2016.01, and 2016.08
interoperability guidelines, and give downstream vendors a year
to work internally and within the ecosystem to update their products
before re-verifying their products.
It's important to note that the Nova team has for two years been
broadcasting their intentions[1][2][3], offering microversions as an
@ -61,21 +60,22 @@ Details of Waiver
#. Products appyling for the OpenStack Powered Trademark in 2016 may
request the waiver by submitting subunit data from their Tempest run
that can be analyzed by the `find_additional_properties.py` script
from the DefCore repository. This script will identify tests that
failed because of additional properties. The vendor will then need
to modify tempest-lib[4] to remove additional checks on the impacted
APIs. Development is beginning within the refstack-client project[5]
to automate generation of a patch for tempest-lib.
from the Interop WG's git repository. This script will identify
tests that failed because of additional properties. The vendor will
then need to modify tempest-lib[4] to remove additional checks on
the impacted APIs. Development is beginning within the
refstack-client project[5] to automate generation of a patch for
tempest-lib.
#. Products that use additional properties in Nova API responses will be
clearly identified in the OpenStack Marketplace, with the product
listing showing which APIs have included additional response data.
Products using additional data will be restricted to the Nova 2.0 API.
#. Beginning with the 2017.01 release of the DefCore guidelines, this
waiver program will no longer be in force, unless 'additional
#. Beginning with the 2017.01 release of the Interoperability Guideline,
this waiver program will no longer be in force, unless 'additional
properties' is listed as an acceptable implementation using the Nova
2.0 API in the forthcoming DefCore capabilities. All other new
2.0 API in the forthcoming capabilities list. All other new
products must pass upstream testing.
#. Aside from additional properties, no products may change the json API

7
working_materials/DefCoreSpec.rst → working_materials/interop_test_spec.rst
View File

@ -3,8 +3,9 @@ Feature Description
======================
This test specification describes interoperability testing as it is understood
by the DefCore Committee. An important goal of interoperability is that end users
of OpenStack should be able to expect certain APIs to be reliable across clouds.
by the Interop Working Group. An important goal of interoperability is that
end users of OpenStack should be able to expect certain APIs to be reliable
across clouds.
The spec aims to define how interoperability should be tested for OpenStack.
It provides guidelines for evaluating current tests and the need for new test
@ -76,4 +77,4 @@ Guidelines
#. Test cases must setup any data they need to run and clean up after
themselves. It is not acceptable to corrupt the user's environment
or leave any test data or test configuration behind.
or leave any test data or test configuration behind.

66
working_materials/scoring.txt
View File

@ -1,17 +1,17 @@
DefCore Scoring Worksheet
=========================
Interop Working Group Scoring Worksheet
========================================
This file is used to maintain capabilities scoring for DefCore.
This file is used to maintain capabilities scoring.
It is considered "working materials" under the 2016A process and
is not an authoritative, Board-approved source of information
as to which capabilities are required for a particular Guideline.
Rather, it is a point-in-time listing used by DefCore Committee
members while determining which capabilities may be included in
a future Guideline. Each line takes the following form:
Rather, it is a point-in-time listing used by Interop
Working Group members while determining which capabilities may be
included in a future Guideline. Each line takes the following form:
Capability Name: [Widely Deployed,Used By Tools,Used by Clients],
[TC Future Direction, Complete, Stable],
[Discoverable, Doc'd, Core in Last Release],
[Discoverable, Doc'd, Required in Last Release],
[Foundational, Atomic, Proximity],
[Non-Admin] [Total]
@ -74,9 +74,10 @@ Notes:
For example, there are several tests for network creation, but only one or
few for deletion, listing, and updating. Combining these into a single
"CRUD" group reduces the chance the chance of running into problems with
DefCore's 2015A process (specifically section C3(4)) in cases where there
is only a single test for a capability and the test needs to be flagged on
account of bugs or other valid reasons. The combined groups are:
the Interop Working Group's 2015A process (specifically section
C3(4)) in cases where there is only a single test for a capability and the
test needs to be flagged on account of bugs or other valid reasons. The
combined groups are:
* networks-floating-ips-CRUD-and-associate
* networks-floating-ips-associate
* networks-floating-ips-create
@ -112,7 +113,7 @@ Notes:
pools advisory now, they could become required in 2017.08--in which
the earliest release in the Guideline will be Mitaka. Toolkits like
jClouds, Fog, and libcloud do not yet support subnet pools that I see.
Subnet pools will be used by Neutron's upcoming get-me-a-network
Subnet pools will be used by Neutron's upcoming get-me-a-network
capability and Kuryr appears to have plans to use them as part of
Kubernets integration.
* Extended port attribute tests were not considered because the capability
@ -129,6 +130,46 @@ compute-servers-suspend-resume: [1,1,1] [1,1,1] [1,1,0] [0,1,1] [1] [82]*
compute-flavors-list: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
compute-availability-zones-list: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
Notes:
* There has been some discussion that this capability is not widely
deployed [1]. However it appears that in most cases the API either:
* Is present and works, but only if a non-SSL endpoint is exposed.
SSL endpoints may have issues with redirects (which may actually
merely be configuration issues).
* Is accidentally blocked due to load balancer or API gateway
misconfiguration which is being remedied by affected providers.
Taking those situations into account, it is believed that it's actually
safe to call this "widely deployed".
* I'm having trouble finding evidence that tools actually support this
API (refer to [2] for information about what constitutes "tools").
Unlike most of the CRUD-like API's, this one lists information about
the API versions available instead of on end user actionable resources
(such as instances or security groups), and the need to do so is newish
with the advent of microversioning. Hence, I think many tools may not
actually support this yet. Please feel free to provide evidence to the
contrary; I'd love to be proven wrong.
* Similarly, I'm having trouble finding evidence that clients [3] like
jClouds or Fog support this API (yet?) as they tend to focus on actions.
Please feel free to provide evidence to the contrary; I'd love to be
proven wrong.
* I'm tentatively scoring this API as Proximate [4]. Generally this applies
to CRUD operations (e.g. the delete API is considered proximate to the
create API). However I'm providing a 1 here on the grounds that due
to the shift to microversioning in the Nova API, this will be somewhat
proximate to everything going forward.
* The Interop Working Group is considering this capability for inclusion
in it's projected 2016.01 Guideline even though the test for it wasn't
developed until (narrowly) outside the normal window for identifying
new Capabilities on account of strong indications from the community
that such a Capability is important for API microversioning going
forward [5].
[1] https://bugs.launchpad.net/python-novaclient/+bug/1491579
[2] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n44
[3] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n48
[4] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n90
[5] http://eavesdrop.openstack.org/meetings/defcore_flag_14/2015/defcore_flag_14.2015-09-09-15.03.log.html#l-13
Image
-----
images-v1-delete: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
@ -155,7 +196,7 @@ Notes:
* Scoring for v1 remains in place, but v2 is preferred interop
standard (as reflected in worksheet).
* images-v2-share inherently requires more than one credential,
and currently out of scope for DefCore capabilities.
and currently out of scope for required capabilities.
* images-v2-remove has no test as of Auguest, 2016.
Volume
@ -272,4 +313,3 @@ https://gist.github.com/notmyname/102e4aba7084598638f47cee47f62bb1#file-defcore_
objectstore-container-metadata used in Fog:
https://github.com/fog/fog-openstack/blob/master/docs/storage.md#additional-parameters
Also in jClouds: https://jclouds.apache.org/guides/openstack/#swift

2
working_materials/tabulate_scores.py
View File

@ -31,7 +31,7 @@ parser = argparse.ArgumentParser(
description=textwrap.dedent("""\
Tabulate capability scores and write them to files.
This utility script tabulates scores from a DefCore scoring
This utility script tabulates scores from an Interop WG scoring
worksheet based on the weights from a given Guideline JSON file.
It writes the scores in three formats: