From 33160b11c8b8b15ae38a1021bf336ab400f27ea6 Mon Sep 17 00:00:00 2001 From: ArkadyKanevsky Date: Thu, 22 Apr 2021 14:41:31 -0500 Subject: [PATCH] Created new process 2021A to replace 2017A one https://opendev.org/osf/interop/src/branch/master/doc/source/process/2017A.rst Transition guidelines approval process from tha OIF board to committee consisting of Interop WG, Refstack, TC and Foundation Marketplace manager. Add-on programs are formally included. Updated index file. Change-Id: Ic55f40dd5f61eefed12f458593f39bc883c41bd9 --- doc/source/index.rst | 4 +- doc/source/process/2021A.rst | 417 +++++++++++++++++++++++++++++++++++ 2 files changed, 419 insertions(+), 2 deletions(-) create mode 100644 doc/source/process/2021A.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index 8596de3e..81120bf0 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -9,7 +9,7 @@ Process Documentation .. toctree:: :maxdepth: 1 - process/2016A.rst + process/2021A.rst ===================== Schema Documentation @@ -18,7 +18,7 @@ Schema Documentation .. toctree:: :maxdepth: 1 - schema/1.5.rst + schema/2.0.rst ================= Active Guidelines diff --git a/doc/source/process/2021A.rst b/doc/source/process/2021A.rst new file mode 100644 index 00000000..3f081762 --- /dev/null +++ b/doc/source/process/2021A.rst @@ -0,0 +1,417 @@ +OpenStack Interop Working Group Process 2021A +============================================== + +:Status: Draft +:Replaces: 2017A + +This document describes the Interop Working Group's working +process as required by the OpenStack bylaws +(see ``_) +and approved by the OpenStack Technical Committee and +Open Infrastructure Foundation Board. + +Expected Time line: +--------------------------------------- + ++------------+--------------------------------------+---------------------+ +| Time Frame | Activities | Lead By | ++============+======================================+=====================+ +| -1 month | Draft of next guidelines. | Interop WG | ++------------+-----------+--------------------------+---------------------+ +| | Review next guidelines | Community | ++ PTG +--------------------------------------+---------------------+ +| | Review status | Interop WG | ++------------+--------------------------------------+---------------------+ +| +1 month | Update draft of guidelines | Interop WG | ++------------+--------------------------------------+---------------------+ +| +2 months | Testing draft guidelines | Refstack, Vendors | ++------------+--------------------------------------+---------------------+ +| +3 months | Approve Guidance | approval_committee_ | ++------------+--------------------------------------+---------------------+ + +Note: Time line is aligned with releases schedules. +See ``_ as example of +Xena schedule. Draft is done during the release for which guideline is +being developed, while its completion is done during next release +schedule. +The Interop Working Group may accelerate the process to correct errors +and omissions. + +Process Definition +-------------------------------------- + +The Guideline process has four primary phases: Draft, Review, Validation +and Approval. +- During the Draft phase (A), the Interop Working Group +creates new draft of guideline with input from the community +with updated coverage, scores of the components, and other details. +- During the Review phase (B), guidelines for each project are reviewed +during the PTG with that project, input from refstack community, +and general feedback from the community and vendors. +- During the Validation phase (C) Guidelines go through and implementation +and testing cycle. +- Finally, Approval by approval_committee_ + +The Open Infrastructure Board of Directors is not involved +in the guideline process and review, and approve only changes to the +Interop WG process. + +This section provides specific rules and structure for each phase. + +Guidelines Draft Phase (A) +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Starting: S-1 + +A1. New Guidelines Start From Previous Guidelines + + 1. New Guidelines start from the previously approved Guidelines document. + 2. New Guidelines are given the preliminary name of "next.json". + 3. For each of the new Guidelines add-on component there is + a separate "next.json" document. + +A2. Community Groups Tests for Capabilities + + 1. The Interop Working Group coordinates community + activities with the Technical Leadership to revise the capabilities + based on current technical needs and functionality. + 2. Capabilities must be a subset of the OpenStack Technical Committee + Approved Release as determined by the Board of Directors + (see bylaws of the Foundation, section 4.1(b)(iii)). + 3. Groupings may change between iterations. + 4. Tests must have unique identifiers that are durable across releases + and changes in grouping. + 5. Tests must be under OpenStack Technical Committee governance. + 6. The Interop Working Group will provide the test groupings + in JSON format for scoring. + 7. The Interop Working Group will provide a human-readable summary + of the Guideline generated from the JSON version. + [AI: Update tooling for it] + +A3. Interop Working Group Collects Recommendations for + Designated Sections + + 1. Designated Sections will not be removed without being deprecated in the + previous Guideline. + 2. Designated Sections will not be added without being advisory in the + previous Guideline. + 3. Designated Sections will not be added or be made advisory unless the + corresponding code base is a subset of the OpenStack Technical Committee + Approved Release as determined by the Board of Directors + (see bylaws of the Foundation, section 4.1(b)(iii)). + 4. TC & PTLs may, but is not required to, assist the + Interop Working Group with defining advisory Designated + Sections for projects that have advisory or required Capabilities. + 5. Designated Sections may be sufficiently defined for Guidelines using + general descriptions. + 6. The Interop Working Group will present A3.4 descriptions to + approval_committee_ for approval. + 7. TC & PTLs may, but is not required to, provide more specific + details describing the Designated Sections for a project. + 8. Designated Sections will be included in the JSON Guideline. + +A4. Interop Working Group identifies required capabilities + + 1. The Interop Working Group uses Board approved scoring criteria + `scoring criteria `_ + to evaluate Capabilities. + 2. The Interop Working Group needs Board approval to change + scoring criteria. + 3. Scoring criteria factors or weights cannot change after Draft is + published. + 4. The Interop Working Group identifies a cut-off score for + determining that a Capability is required. + 5. Capabilities will not be removed without being deprecated in the + previous Guideline. + 6. Capabilities will not be added without being advisory in the previous + Guideline. + 7. For the "widely deployed" adoption criteria, the size of the + pool being considered will match the scope of the community being + considered. Capabilities will be evaluated based on their use in their + Component. Components will be evaluated based on their use in the + Platform. + +A5. The Interop Working Group recommends OpenStack Components and OpenStack Platform + Scope + + 1. The Interop Working Group recommends Capabilities to include + in each OpenStack Component. + 2. The Interop Working Group recommends which Components are required for + the OpenStack Powered Platform. + 3. The Interop Working Group will apply the approved scoring criteria to + evaluate if a component should be included in the + Platform (see A4). + +A6. Additional Capabilities and Tests + + 1. The Interop Working Group will work with the OpenStack community to + define new Capabilities. + 2. Test grouping for new Capabilities will be included in the + Interop Working Group documents. + 3. The Interop Working Group will publish a list of missing + Capabilities and Capabilities with inadequate test coverage. + 4. Interop Working Group in conjuction with TC and Foundation Marketplace + product owner can propose additional Add-on component + guidelines. + 5. Each Add-on is provided as a seperate additional guideline, + that follows the same process as "OpenStack powered" ones + as defined by this document. + +A7. The Interop Working Group creates recommendation for Draft. + + 1. The Interop Working Group coordinates activities to create + the Guideline draft. + 2. The Interop Working Group may choose to ignore recommendations + with documented justification. + +Guidelines Review Phase (B) +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Starting: PTG + +B1. All Reference Artifacts are reviewed via Gerrit + + 1. Draft Guideline + 2. Designated sections + 3. Test-Capability groupings + 4. Flagged Test List + 5. Capability Scoring criteria and weights + 6. May not be in Gerrit: Working materials (spreadsheets, etc) + +B2. Presentation of Draft Guidelines for Review + + 1. The Interop Working Group will distribute Draft Guidelines + to the OpenStack community for review. + 2. A link to the Gerrit document must be provided with the review materials. + 3. The draft Guidelines will consist of single document for + "OpenStack-Powered" Logo and separate document for each + of the add-ons components. + +B3. Changes to Guideline made by Gerrit Review Process + + 1. Guidelines proposal and review must go through Gerrit. + 2. The Interop Working Group will proxy for users who do not + participate in the Gerrit process with attribution. + +B4. For Gerrit reviews, Interop Working Group Co-Chairs act as + core reviewers. + + 1. Interop Working Group Co-Chairs serve as "core" reviewers (+2). + 2. Requests for changes must be submitted as patches by the requesting + party. + 3. Interop Working Group members may proxy change requests as + long as the requesting party is explicitly acknowledged. + 4. One Interop Working Group Co-Chair must be an Open Infrastructure + Foundation Board member. + 5. One Co-Chair of Interop Working Group is nominated by + the Open Infrastructure Board of Directors. + 6. One Interop Working Group Co-Chair will be elected by the + Interop Working Group. Election quorum is composed of + attendees present during the election meeting. + 7. Additional core reviewers (+2) can be appointed by Co-Chairs. + 8. Election meetings must be posted at least one meeting prior. + +B5. Projects Interlocks + 1. For the PTG, the Interop Working Group requests a meeting + with each project under the "OpenStack Powered" guideline and + each of the add-on guidelines. + 2. Project meeting covers a review of the current guidelines + for the projects, any changes/addition/deprecation/removal of + APIs. Tests must be available in repositories under + OpenStack TC governance. The tests cannot be in repositories + outside of list in the openstack governance repository: + ``_ + 3. Project meeting covers any test result for specific + drivers of that project to ensure consistent functionality + coverage for all required and designated interoperability + functionality of the project. + +B6. Draft Guidelines update + 1. Following the PTG meeting the Interop Working Group updates + draft Guidelines using feedback from each of the projects. + 2. Ensure any new or modified guidelines must have the corresponding + test(s) in Tempest or Tempest plugins, and Refstack wrapper + include it. + 3. For any new add-on program Refstack need to be able to collect + submissions for marketplace. + 4. For any new add-on programs Foundation Marketplace product + owner need to prepare ability to issue, record and administer + new add-on Logo under Marketplace program. + +Validation (C) +^^^^^^^^^^^^^^ + +Starting: S and continues until S+2 + +C0. RefStack validation + 1. Refstack makes necessary changes to `Page `_ + 2. Refstack makes necessary changes to handle new guidelines. + 3. Refstack representative share test results of new guidelines + on default platform with Interop Working Group. + 4. Refstack flags any tests that do not pass on the default platform. + +C1. Vendor Self-Tests + + 1. Vendors are responsible for executing tests identified by the + Interop Working Group. + 2. The Interop Working Group may, but is not required to, provide tooling for + running tests through Refstack. + 3. The Interop Working Group may, but is not required to, define a required + reporting format. + 4. Self-test results may be published by Vendors in advance of Open + Infrastructure Foundation Marketplace manager review, + but must be clearly labeled as "Unofficial Results - Not Yet + Accepted By The Open Infrastructure Foundation". + 5. Vendors who publish self-tests MUST provide them in the same format that + would be submitted to the Open Infrastructure Foundation + but MAY provide additional formats if they choose to do so. + 6. Self-test results cannot be used as proof of compliance. + +C2. Vendor submits results to Foundation for review + + 1. The Open Infrastructure Foundation Marketplace manager + determines the acceptable format for submissions. + 2. The Open Infrastructure Foundation Marketplace manager + has final authority to determine if Vendor meets criteria. + 3. The Open Infrastructure Foundation Marketplace manager + will provide a review of the results within 30 days. + 4. Vendors can submit results for "OpenStack Powered" Logo + and any of the add-on programs together or separately. + 5. The Open Infrastructure Foundation Marketplace manager + can provide review of the results for all vendor + submissions together or separately for each Logo. + +C3. Vendor Grievance Process + + 1. Vendors may raise concerns with specific tests to the Interop + Working Group. + 2. The Interop Working Group may choose to remove tests from + a Guideline (known as flagging). + 3. The Interop Working Group will acknowledge vendor requests + to flag tests within 30 days. + +C4. Results of Vendor Self-Tests will be open + + 1. The Open Infrastructure Foundation Marketplace manager + will make the final results of approved vendors + available to the community. + 2. The Open Infrastructure Foundation Marketplace manager + will not publish incomplete or unapproved results. + 3. Only "pass" results will be reported. Skipped and failed results will + be omitted from the reports. + 4. Reports will include individual test results, not just Capability + scoring. + 5. Vendors are required to submit a description of the system and + configuration used to achieve the results. + +C5. API Usage Data Report + + 1. The Open Infrastructure Foundation Marketplace manager + will provide the Interop Working Group with an + open report about API usage based on self-tests based on + Refstack submitted results. + 2. To the extent the data is available, Capabilities beyond the + Interoperability Guideline list will be included in the report. + +C6. Only Two Approved Guidelines at a time: + + 1. Vendors seeking validation are limited to using the two + latest approved Guidelines. + + 2. Since past validations are respected, older Guidelines will be + maintained as superseded for historical reference. + + 3. Guideline status progresses as follows: + + :draft: initial work, pre-PTG discussion material + :review: as presented at the PTG (S) for community review + :approved: Committee approved, one of the two official guidelines + :superseded: Committee approved, now superseded by two latest guidelines + +Guideline Approval (D) +^^^^^^^^^^^^^^^^^^^^^^ + +Starting: S+3 or earlier + +.. _approval_committee: + +D0. Approval Committee + + 1. Approval Committee consists of representatives from 4 bodies + + a. Interop Working Group Co-chairs are approval authority of + this committee. + b. Refstack core member or presentative(s) as advisory member + c. Open Infrastructure Foundation Marketplace product owner as + advisory member + d. OpenStack TC representative(s) as advisory member + +D1. Committee will review and approve Interoperability Guidelines from draft + + 1. Interop Working Group must approve the proposed guidelines. + 2. Foundation staff member(s) and OpenStack TC member(s) vote in + advisory capacity but do not have veto power. + 3. The approval is done through formal vote and its results recorded + with the date of Approval. + 4. Upon approval guideline document is marked as approved with the + date of approval. + 5. approval_committee_ members may request delay of the formal vote till + foundation staff and/or Refstack is capable of handling new guidelines. + 6. Delay of the vote can be resolved by at most 1 month prior to + next PTG meeting, which is the start of work on the new + guidelines. + 7. Guidelines are set at the Platform, Component and + Capability level only. + 8. The Interop Working Group will submit the human-readable + summary of Capabilities (see section A2[6]) to the Committee for approval. + 9. By voting to approve the summary, the Committee delegates responsibility + for maintaining test groupings to the Interop Working Group + subject to the limitations described in section D2. + 10. Guidelines only apply to the identified releases (a.k.a. release + tags). + 11. The Add-on Guidelines are set for OpenStack projects in addition + to "OpenStack Powered" ones. + 12. All guidelines follow the same review and approval process + irrespective if they are "OpenStack Powered" or "OpenStack Add-on" + guidelines. + +D2. Interop Working Group has authority on test categorization + + 1. The Interop Working Group can add flagged tests before and + after Guideline approval. + 2. The Interop Working Group cannot add additional Tests to + Capability mappings after approval. + 3. The Interop Working Group maintains the test to Capability + mappings in the JSON representation. + +D3. Designated Sections only enforced for projects with required Capabilities + + 1. Designated Sections may be defined for any project. + 2. Designated Sections apply to the releases (a.k.a. release tags) + identified in the Guideline. + 3. Designated Sections will be included in the JSON Capabilities file + to ensure a single source of identification. + +D4. Guidelines are named based on the date of Board approval + + 1. Naming pattern will be: 4-digit year, dot (period), and 2-digit month. + +Process Change +-------------- + +E1. Process Draft + 1. Any process change follows the process of draft, review and approval. + 2. Any process changes are handled thru gerrit process. + 3. Proposed changes submitted to Gerrit for review by + Interop Working Group as a draft document. + 4. Interop Working Group adds all Committee members for review of the draft. + 5. Once that draft is approved, Interop Working Group co-chairs present it to + OpenStack Board for approval. + 6. Once Board approved the changes the new process is marked as + approved and is linked from `Interop WG wiki page + `_. + +Functional Information +---------------------- +:Format: RestructuredText +:Layout: 1.0