interop/doc/source/process/CoreCriteria.rst
Mark T. Voelker f0065421c5 Adjust definitions of clients and tools
Originally, the definitions we have for the "used by clients" and "used
by tools" Criteria were such that both dealt primarily with software
outside of the OpenStack community.  For example, the definition of
"clients" included mention of jClouds and Fog.  The definition of "tool"
mentions RightScale, Scalr, and CloudForms.  In practice what we've
found over the first few cycles in which adherance to DefCore Guidelines
is required is that we often find issues with Capabilities because
there is poor client support for a major API version within the
OpenStack community itself.  For example, the Keystone v3 API was
originally introduced in Grizzly [1] but python-neutronclient didn't
support v3 until version 2.3.7 in late 2014 (in the Juno timeframe). [2]
Nova still uses the Glance v1 API as of Kilo.  We therefore have an
interest in making it explicitly clear that acceptance of an API by
other OpenStack projects is a consideration for a Capability to become
part of a Guideline.  If an API hasn't achieved acceptance by other
OpenStack projects that will likely be used in the same deployment but
an older version of the API has, we should favor requiring the older
version.

We've also seen general sentiment that our audience tends to see the
distinction between our current definitions of "tools" and "clients" as
not terribly useful in that both definitions essentially state that the
Capability in question must be accepted by the world outside of
OpenStack itself.  Drawing a distinction between outside tools and
outside clients gives somewhat of a "double weight" to whether a
Capability has gained acceptance in the broader world, and these scores
almost inevitably end up being the same anyway.  A more useful
distinciton appears to be drawing the line at what's accepted by the
rest of the OpenStack projects and what's accepted by the world outside
of OpenStack.

This patch updates the two defintions to essentially do that.  It alters
the definition of "tools" to include software outside of OpenStack's
control, and the definiton of "clients" to include other OpenStack
projects (and add a note about API versioning for clarity).  These
changes essentially provide separate weighting for acceptance of a
capability inside of OpenStack and by the outside world.

These changes will be used starting with scoring activities for the
2016.07 Guideline.

[1]
https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly#OpenStack_Identity_.28Keystone.29
[2]
https://blueprints.launchpad.net/python-neutronclient/+spec/keystone-api-v3-support
[3]

Change-Id: I7aa9077be458e688cf4c417233fb7b7611e84cd9
2015-12-02 13:51:46 -08:00

3.7 KiB

Core Criteria

General

This is a subcomponent of the larger Core Definition documentation.

Core Capability Selection uses 12 criteria grouped into four primary categories:

Usage

the capability is widely used (Refstack will collect data)

Direction

the capability advances OpenStack technically

Community

the capability builds the OpenStack community experience

System

the capability integrates with other parts of OpenStack

These categories summarize critical values that we want in OpenStack and so make sense to be the primary factors used when we select DefCore capabilities. While we strive to make the DefCore process objective and quantitive, we must recognize that these choices drive community behavior.

Graphic showing Capabilities Seleciton Criteria

Details

With this perspective, here are the selection criteria.

To make cross reference easier, criteria are given a shortened name.

SHOWS PROVEN USAGE

  • "Widely Deployed" Candidates are widely deployed capabilities. We favor capabilities that are supported by multiple public cloud providers and private cloud products.
  • "Used by Tools" Candidates are widely used capabilities: Should be included if supported by common tools outside of the OpenStack community (RightScale, Scalr, CloudForms, jClouds, Fog, etc...)
  • "Used by Clients" Candidates are widely used capabilities: Should be included if called by common OpenStack clients (openstackclient, neutronclient, novaclient, etc) if necessary. This Criteria pertains mostly to API versioning. For example: if v2 of a given API is not used by other OpenStack clients but v1 is, then v2 doesn't achieve the "used by clients" Criteria.

ALIGNS WITH TECHNICAL DIRECTION

  • "Future Direction" Should reflect future technical direction (from the project technical teams and the TC) and help manage deprecated capabilities.
  • "Stable" Capabilities are required to be stable for >2 releases because we do not want DefCore capabilities that do not have dependable APIs.
  • "Complete" Where the code being tested has a designated area of alternate implementation (extension framework) as per the DefCore Principles, there should be parity in capability tested across extension implementations. This also implies that the capability test is not configuration specific or locked to non-open technology.

PLAYS WELL WITH OTHERS

  • "Discoverable" Capability being tested is Service Discoverable (can be found in Keystone and via service introspection)
  • "Doc'd" Should be well documented, particularly the expected behavior. This can be a very subjective measure and we expect to refine this definition over time.
  • "DefCore in Last Release" A required capability should stay a required capability. This make makes DefCore capabilities sticky release per release. Leaving DefCore is disruptive to the ecosystem

TAKES A SYSTEM VIEW

  • "Foundation" Capabilities that are required by other required capabilities and/or depended on by many other capabilities
  • "Atomic" Capability is unique and cannot be built out of other required capabilities
  • "Proximity" (sometimes called a Capability Cluster) selects for capabilities that are related to DefCore capabilities. This helps ensure that related capabilities are managed together.

NON-ADMIN

The original 13th "non-admin" criteria has been removed because Admin APIs cannot be used for interoperability and are not considered DefCore.