140139ab01
Scored an existing advisory capability (identity-v3-api-discovery) and made it required in next.json. Scored 3 new capabilities for which I couldn't find tests, adding them however for discussion. One of the new capabilities, identity-v3-catalog refers to the catalog that is returned when calling the current required capability identity-v3-tokens-create. Users may be relying on the catalog to be there, so I think it is worth discussing. identity-v3-list-projects and identity-v3-list-groups are worth discussing and consider for addition of new tests. The rationale for this scoring as well as input from keystone's PTL can be found in working_materials/keystone_capabilities_info.csv. Change-Id: Id444f5e982f2e81f140e285c305e9c322f5b9f42
206 lines
11 KiB
Plaintext
206 lines
11 KiB
Plaintext
DefCore Scoring Worksheet
|
|
=========================
|
|
|
|
This file is used to maintain capabilities scoring for DefCore.
|
|
It is considered "working materials" under the 2015B process and
|
|
is not an authoritative, Board-approved source of information
|
|
as to which capabilities are required for a particular Guideline.
|
|
Rather, it is a point-in-time listing used by DefCore Committee
|
|
members while determining which capabilities may be included in
|
|
a future Guideline. Each line takes the following form:
|
|
|
|
Capability Name: [Widely Deployed,Used By Tools,Used by Clients],
|
|
[TC Future Direction, Complete, Stable],
|
|
[Discoverable, Doc'd, Core in Last Release],
|
|
[Foundational, Atomic, Proximity],
|
|
[Non-Admin]
|
|
|
|
Possible scores: 0, 1, ? (not scored yet or needs discussion)
|
|
|
|
Networking
|
|
----------
|
|
network-floating-ips-associate: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [74]
|
|
network-floating-ips-create: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [74]
|
|
network-floating-ips-update: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [74]
|
|
network-floating-ips-delete: [1,1,1] [1,1,1] [0,1,0] [0,1,1] [1] [74]
|
|
network-l2-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l2-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l2-external: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l2-list: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l2-port: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l2-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l2-update: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l3-add: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l3-convert: [0,0,0] [1,0,0] [0,0,0] [0,0,0] [1] [11]
|
|
network-l3-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l3-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l3-router: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l3-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-l3-update: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-security-groups-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-security-groups-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-security-groups-list: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-security-groups-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [1] [91]
|
|
network-extensions-provider-add: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
|
|
network-extensions-provider-create: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
|
|
network-extensions-provider-delete: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
|
|
network-extensions-provider-show: [1,1,1] [1,1,1] [1,1,0] [1,1,1] [0] [0]
|
|
|
|
Notes:
|
|
* DVR-related testcases (if any) will not be added until DVR gains wider
|
|
acceptance and isn't driver-specific.
|
|
* IPv6 is currently not widely deployed, so v6 versions of these tests need
|
|
not be added at this time. Vendor feedback on v6 support is requested,
|
|
though it is understood that IPv6 capabilities may be driver-dependent.
|
|
* A few folks have asked about provider network extensions as they are
|
|
frequently used for high performance networks and a simplified networking
|
|
model that provides for use cases similar to nova-net. However with default
|
|
policy settings, using provider networks requires admin access and therefore
|
|
is disqualified. I've listed them here anyway just to show that they were
|
|
considered but can't be added to the Guideline.
|
|
* Based on feedback from earlier patchsets, capabilities will be grouped
|
|
together where they had previously been listed separately but are adjacent.
|
|
For example, there are several tests for network creation, but only one or
|
|
few for deletion, listing, and updating. Combining these into a single
|
|
"CRUD" group reduces the chance the chance of running into problems with
|
|
DefCore's 2015A process (specifically section C3(4)) in cases where there
|
|
is only a single test for a capability and the test needs to be flagged on
|
|
account of bugs or other valid reasons. The combined groups are:
|
|
* networks-floating-ips-CRUD-and-associate
|
|
* networks-floating-ips-associate
|
|
* networks-floating-ips-create
|
|
* networks-floating-ips-update
|
|
* networks-floating-ips-delete
|
|
* networks-l2-CRUD
|
|
* networks-l2-create
|
|
* networks-l2-delete
|
|
* networks-l2-external
|
|
* networks-l2-list
|
|
* networks-l2-port
|
|
* networks-l2-show
|
|
* networks-l2-update
|
|
* networks-l3-CRUD
|
|
* networks-l3-add
|
|
* networks-l3-create
|
|
* networks-l3-delete
|
|
* networks-l3-show
|
|
* networks-l3-update
|
|
* networks-l3-router
|
|
* networks-security-groups-CRUD
|
|
* networks-security-groups-create
|
|
* networks-security-groups-delete
|
|
* networks-security-groups-router
|
|
* networks-security-groups-show
|
|
* networks-security-groups-update
|
|
|
|
Compute
|
|
-------
|
|
compute-list-api-versions: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [75]
|
|
|
|
Notes:
|
|
* There has been some discussion that this capability is not widely
|
|
deployed [1]. However it appears that in most cases the API either:
|
|
* Is present and works, but only if a non-SSL endpoint is exposed.
|
|
SSL endpoints may have issues with redirects (which may actually
|
|
merely be configuration issues).
|
|
* Is accidentally blocked due to load balancer or API gateway
|
|
misconfiguration which is being remedied by affected providers.
|
|
Taking those situations into account, it is believed that it's actually
|
|
safe to call this "widely deployed".
|
|
* I'm having trouble finding evidence that tools actually support this
|
|
API (refer to [2] for information about what constitutes "tools").
|
|
Unlike most of the CRUD-like API's, this one lists information about
|
|
the API versions available instead of on end user actionable resources
|
|
(such as instances or security groups), and the need to do so is newish
|
|
with the advent of microversioning. Hence, I think many tools may not
|
|
actually support this yet. Please feel free to provide evidence to the
|
|
contrary; I'd love to be proven wrong.
|
|
* Similarly, I'm having trouble finding evidence that clients [3] like
|
|
jClouds or Fog support this API (yet?) as they tend to focus on actions.
|
|
Please feel free to provide evidence to the contrary; I'd love to be
|
|
proven wrong.
|
|
* I'm tentatively scoring this API as Proximate [4]. Generally this applies
|
|
to CRUD operations (e.g. the delete API is considered proximate to the
|
|
create API). However I'm providing a 1 here on the grounds that due
|
|
to the shift to microversioning in the Nova API, this will be somewhat
|
|
proximate to everything going forward.
|
|
* The DefCore Committee is considering this capability for inclusion
|
|
in it's projected 2016.01 Guideline even though the test for it wasn't
|
|
developed until (narrowly) outside the normal window for identifying
|
|
new Capabilities on account of strong indications from the community
|
|
that such a Capability is important for API microversioning going
|
|
forward [5].
|
|
|
|
[1] https://bugs.launchpad.net/python-novaclient/+bug/1491579
|
|
[2] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n44
|
|
[3] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n48
|
|
[4] http://git.openstack.org/cgit/openstack/defcore/tree/doc/source/process/CoreCriteria.rst#n90
|
|
[5] http://eavesdrop.openstack.org/meetings/defcore_flag_14/2015/defcore_flag_14.2015-09-09-15.03.log.html#l-13
|
|
|
|
Image
|
|
-----
|
|
images-v1-delete: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-index: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-list: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-register: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-shared-images-add: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-shared-images-delete: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-shared-images-get: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-shared-images-remove: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
images-v1-update: [0,1,1] [0,1,1] [0,1,0] [1,0,1] [1] [58]
|
|
|
|
images-v2-remove: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [79]
|
|
images-v2-update: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [79]
|
|
images-v2-share: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [79]
|
|
images-v2-import: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [79]
|
|
images-v2-list: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [79]
|
|
images-v2-delete: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [79]
|
|
images-v2-get: [1,0,1] [1,1,1] [1,1,0] [1,0,1] [1] [79]
|
|
|
|
Notes:
|
|
* Image creation is captured under the register operation.
|
|
* No public image tests have explicit support for task API.
|
|
* Scoring for v1 remains in place, but v2 is preferred interop
|
|
standard (as reflected in worksheet).
|
|
|
|
Volume
|
|
----------
|
|
volumes-v2-create-delete: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [75]
|
|
volumes-v2-attach-detach: [1,0,0] [1,1,1] [1,1,0] [1,0,1] [1] [69]
|
|
volumes-v2-snapshot-create-delete: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [75]
|
|
volumes-v2-get: [1,0,0] [1,1,1] [1,1,0] [1,0,1] [1] [69]
|
|
volumes-v2-list: [1,0,0] [1,1,1] [1,1,0] [1,0,1] [1] [69]
|
|
volumes-v2-update: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [75]
|
|
volumes-v2-copy-image-to-volume: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [75]
|
|
volumes-v2-copy-volume-to-image: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [75]
|
|
volumes-v2-clone: [1,0,0] [1,1,1] [1,1,0] [1,1,1] [1] [75]
|
|
volumes-v2-qos: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [67]
|
|
volumes-v2-availability-zones: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [67]
|
|
volumes-v2-extensions: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [67]
|
|
volumes-v2-metadata: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [67]
|
|
volumes-v2-transfer: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [67]
|
|
volumes-v2-reserve: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [67]
|
|
volumes-v2-readonly: [1,0,0] [1,1,1] [1,1,0] [1,1,0] [1] [67]
|
|
|
|
Identity
|
|
--------
|
|
identity-v3-api-discovery: [1,0,1] [1,1,1] [1,1,0] [1,1,1] [1] [85]
|
|
identity-v3-catalog: [1,0,1] [1,1,1] [1,1,0] [1,1,1] [1] [85]
|
|
identity-v3-list-projects: [1,0,1] [1,1,1] [1,1,0] [0,1,0] [1] [68]
|
|
identity-v3-list-groups: [1,0,1] [1,1,1] [1,1,0] [0,1,0] [1] [68]
|
|
|
|
Notes:
|
|
* identity-v3-catalog is returned when the api for
|
|
identity-v3-tokens-create is called (GET /v3/auth/tokens). It is
|
|
important to consider it because end users may be relying on this
|
|
catalog for their apps (even though there are other API calls that
|
|
also show the catalog such as GET /v3/auth/catalog). There is one test
|
|
available for this capability but it is in the admin part of the test
|
|
suite, so not yet tested for non-admin users. Even though it scores enough
|
|
to be included as advisory, we cannot do this due to lack of non-admin
|
|
test case.
|
|
* identity-v3-list-projects and identity-v3-list-groups are here because
|
|
they deserve some visibility and some explicit test cases, which at the
|
|
moment they are lacking. It seems important for users to be able to
|
|
discriminate between projects and groups when running their apps.
|