openinfra/interop
Code Issues Proposed changes
5e0ac0a22c
interop/working_materials/scoring.txt
Chris Hoge cf0e8f3323 Added new identity capabilities and scoring matrix 
Added two new capabilities for identity, api discovery
for v3. Included template scoring matrix. Included
new non-admin identity tests.

Deprecated Keystone v2 capabilities.

Change-Id: Ice92d9c7d8af845db6e3bc51762e576ee54f012c
2015-10-15 18:05:47 -07:00

188 lines
10 KiB
Plaintext
Raw Blame History

DefCore Scoring Worksheet
=========================
This file is used to maintain capabilities scoring for DefCore.
It is considered "working materials" under the 2015B process and
is not an authoritative, Board-approved source of information
as to which capabilities are required for a particular Guideline.
Rather, it is a point-in-time listing used by DefCore Committee
members while determining which capabilities may be included in
a future Guideline. Each line takes the following form:
Capability Name: [Widely Deployed,Used By Tools,Used by Clients],
[TC Future Direction, Complete, Stable],
[Discoverable, Doc'd, Core in Last Release],
[Foundational, Atomic, Proximity],
[Non-Admin]
Possible scores: 0, 1, ? (not scored yet or needs discussion)
Networking
----------
network-floating-ips-associate: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [75]
network-floating-ips-create: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [75]
network-floating-ips-update: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [75]
network-floating-ips-delete: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [75]
network-l2-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l2-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l2-external: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l2-list: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l2-port: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l2-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l2-update: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l3-add: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l3-convert: [0,0,0] [1,0,0] [0,0,0] [0,0,0] [1] [9]
network-l3-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l3-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l3-router: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l3-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-l3-update: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-security-groups-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-security-groups-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-security-groups-list: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-security-groups-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
network-extensions-provider-add: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
network-extensions-provider-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
network-extensions-provider-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
network-extensions-provider-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
Notes:
* DVR-related testcases (if any) will not be added until DVR gains wider
acceptance and isn't driver-specific.
* IPv6 is currently not widely deployed, so v6 versions of these tests need
not be added at this time. Vendor feedback on v6 support is requested,
though it is understood that IPv6 capabilities may be driver-dependent.
* A few folks have asked about provider network extensions as they are
frequently used for high performance networks and a simplified networking
model that provides for use cases similar to nova-net. However with default
policy settings, using provider networks requires admin access and therefore
is disqualified. I've listed them here anyway just to show that they were
considered but can't be added to the Guideline.
* Based on feedback from earlier patchsets, capabilities will be grouped
together where they had previously been listed separately but are adjacent.
For example, there are several tests for network creation, but only one or
few for deletion, listing, and updating. Combining these into a single
"CRUD" group reduces the chance the chance of running into problems with
DefCore's 2015A process (specifically section C3(4)) in cases where there
is only a single test for a capability and the test needs to be flagged on
account of bugs or other valid reasons. The combined groups are:
* networks-floating-ips-CRUD-and-associate
* networks-floating-ips-associate
* networks-floating-ips-create
* networks-floating-ips-update
* networks-floating-ips-delete
* networks-l2-CRUD
* networks-l2-create
* networks-l2-delete
* networks-l2-external
* networks-l2-list
* networks-l2-port
* networks-l2-show
* networks-l2-update
* networks-l3-CRUD
* networks-l3-add
* networks-l3-create
* networks-l3-delete
* networks-l3-show
* networks-l3-update
* networks-l3-router
* networks-security-groups-CRUD
* networks-security-groups-create
* networks-security-groups-delete
* networks-security-groups-router
* networks-security-groups-show
* networks-security-groups-update
Compute
-------
compute-list-api-versions: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [74]
Notes:
* There has been some discussion that this capability is not widely
deployed [1]. However it appears that in most cases the API either:
* Is present and works, but only if a non-SSL endpoint is exposed.
SSL endpoints may have issues with redirects (which may actually
merely be configuration issues).
* Is accidentally blocked due to load balancer or API gateway
misconfiguration which is being remedied by affected providers.
Taking those situations into account, it is believed that it's actually
safe to call this "widely deployed".
* I'm having trouble finding evidence that tools actually support this
API (refer to [2] for information about what constitutes "tools").
Unlike most of the CRUD-like API's, this one lists information about
the API versions available instead of on end user actionable resources
(such as instances or security groups), and the need to do so is newish
with the advent of microversioning. Hence, I think many tools may not
actually support this yet. Please feel free to provide evidence to the
contrary; I'd love to be proven wrong.
* Similarly, I'm having trouble finding evidence that clients [3] like
jClouds or Fog support this API (yet?) as they tend to focus on actions.
Please feel free to provide evidence to the contrary; I'd love to be
proven wrong.
* I'm tentatively scoring this API as Proximate [4]. Generally this applies
to CRUD operations (e.g. the delete API is considered proximate to the
create API). However I'm providing a 1 here on the grounds that due
to the shift to microversioning in the Nova API, this will be somewhat
proximate to everything going forward.
* The DefCore Committee is considering this capability for inclusion
in it's projected 2016.01 Guideline even though the test for it wasn't
developed until (narrowly) outside the normal window for identifying
new Capabilities on account of strong indications from the community
that such a Capability is important for API microversioning going
forward [5].
[1] https://bugs.launchpad.net/python-novaclient/+bug/1491579
[2] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n44
[3] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n48
[4] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n90
[5] http://eavesdrop.openstack.org/meetings/defcore_flag_14/2015/defcore_flag_14.2015-09-09-15.03.log.html#l-13
Image
-----
images-v1-delete: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-index: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-list: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-register: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-shared-images-add: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-shared-images-delete: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-shared-images-get: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-shared-images-remove: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v1-update: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
images-v2-remove: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [74]
images-v2-update: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [74]
images-v2-share: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [74]
images-v2-import: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [74]
images-v2-list: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [74]
images-v2-delete: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [74]
images-v2-get: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [74]
Notes:
* Image creation is captured under the register operation.
* No public image tests have explicit support for task API.
* Scoring for v1 remains in place, but v2 is preferred interop
standard (as reflected in worksheet).
Volume
----------
volumes-v2-create-delete: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [74]
volumes-v2-attach-detach: [1,0,0] [1,1,1] [1,1,0] [1,0,1] [1] [66]
volumes-v2-snapshot-create-delete: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [74]
volumes-v2-get: [1,0,0] [1,1,1] [1,1,0] [1,0,1] [1] [66]
volumes-v2-list: [1,0,0] [1,1,1] [1,1,0] [1,0,1] [1] [66]
volumes-v2-update: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [74]
volumes-v2-copy-image-to-volume: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [74]
volumes-v2-copy-volume-to-image: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [74]
volumes-v2-clone: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [74]
volumes-v2-qos: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [66]
volumes-v2-availability-zones: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [66]
volumes-v2-extensions: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [66]
volumes-v2-metadata: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [66]
volumes-v2-transfer: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [66]
volumes-v2-reserve: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [66]
volumes-v2-readonly: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [66]
Identity
--------
identity-v3-api-discovery: [0,0,1] [1,1,1] [1,1,0] [1,1,1] [1] [74]
View Git Blame Copy Permalink