openinfra/interop
Code Issues Proposed changes

Merge "Keystone scoring for 2018.01 guideline"

Browse Source
This commit is contained in:
Zuul 2017-10-24 21:33:21 +00:00 committed by Gerrit Code Review
commit 9980d2ec82
3 changed files with 11 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
working_materials/keystone_capabilities_info.csv
View File

@ -1,7 +1,7 @@
Capability,Program,Status,Method,Endpoint,Test available?,interop relevant?,PTL Comments,From Defcore Discussion,Scorer Comments,
identity-v3-tokens-create,platform/compute/object,required,POST,/v3/auth/tokens,1,yes,The returned token value is in the X-Auth-Token header,,tempest.api.identity.v3.test_tokens{test_create_token}, This TC refers to API https://developer.openstack.org/api-ref/identity/v3/#password-authentication-with-unscoped-authorization. Should we add other test cases to tempest in order to validate API for: password-authentication-with-scoped-authorization and password-authentication-with-explicit-unscoped-authorization?,
identity-v3-api-discovery,platform/compute,required,,,3,yes,,make required,"tempest.api.identity.v3.test_api_discovery{test_api_version_resources, test_api_media_types, test_api_version_statuses}",
identity-v3-list-projects,platform/compute,advisory,GET,/v3/users/{user_id}/projects,1,yes,,,,
identity-v3-tokens-create,platform/compute/object,required,POST,/v3/auth/tokens,1,yes,The returned token value is in the X-Auth-Token header,,tempest.api.identity.v3.test_tokens{test_create_token}, This TC refers to API https://developer.openstack.org/api-ref/identity/v3/#password-authentication-with-unscoped-authorization. Should we add other test cases to tempest in order to validate API for: password-authentication-with-scoped-authorization and password-authentication-with-explicit-unscoped-authorization?
identity-v3-api-discovery,platform/compute,required,GET,/v3,3,yes,,make required,"tempest.api.identity.v3.test_api_discovery{test_api_version_resources, test_api_media_types, test_api_version_statuses}",
identity-v3-list-projects,platform/compute,required,GET,/v3/users/{user_id}/projects,1,yes,,,Flagged since require 2 set of user credentials.,
,,,,,,,,,,
identity-v3-create-ec2-credentials,,,POST,/v3/credentials,1,yes,,Should we make ec2 compatibility required? unclear,,
identity-v3-list-ec2-credentials,,,GET,/v3/credentials,1,yes,,Should we make ec2 compatibility required? unclear,,
@ -9,8 +9,7 @@ identity-v3-show-ec2-credentials,,,GET,/v3/credentials/{credential_id},1,yes,,Sh
identity-v3-delete-ec2-credentials,,,DELETE,/v3/credentials/{credential_id},1,yes,,Should we make ec2 compatibility required? unclear,,
identity-v3-update-ec2-credentials,,,PATCH,/v3/credentials/{credential_id},,,,Should we make ec2 compatibility required? unclear,,
identity-v3-catalog,(make sure it works on all supported releases),,,,,,returned with the token,,,
identity-v3-password-update,,,POST,/v3/users/{user_id}/password,1,yes,,"
Untestable without changing user's password, security risk. Also password policies are very particular to different companies, making a test that would pass on all is near impossible.",tempest.api.identity.v3.test_users{test_update_own_password},
identity-v3-password-update,,,POST,/v3/users/{user_id}/password,1,yes,,"Untestable without changing user's password, security risk. Also password policies are very particular to different companies, making a test that would pass on all is near impossible.",tempest.api.identity.v3.test_users{test_update_own_password},
,,,,,,,,,,
identity-v3-list-groups,platform/compute,,GET,/v3/users/{user_id}/groups,0,yes,,,no test available for this feature,
identity-v3-get-project,platform/compute,,GET,/v3/projects/{project_id},0,yes,,,admin required,
@ -19,9 +18,9 @@ identity-v3-get-role,platform/compute,,GET,/v3/roles/{role_id},,no,,,admin requi
identity-v3-list-domains,platform/compute,,GET,/v3/domains,,no,,,admin required,
identity-v3-get-domain,platform/compute,,GET,/v3/domains/{domain_id},,no,,,admin required,
,,,,,,,,,,
identity-v3-tokens-validate,platform/compute,,GET,/v3/auth/tokens,,yes,Token to be validated is passed in the X-Subject-Token header,,,"This sounds backwards to me, need to check with steve, shouldn't it be POST for validating and GET for getting a token?"
identity-v3-tokens-validate,platform/compute,advisory,GET,/v3/auth/tokens,,yes,Token to be validated is passed in the X-Subject-Token header,,,
identity-v3-revoke-token,platform/compute,,DELETE,/v3/auth/tokens,1,yes,Token to be revoked is passed in the X-Subject-Token header,keystone.keystone.tests.unit.test_revoke{test_revoke_by_user},,
identity-v3-get-catalog,platform/compute/object,,GET,/v3/auth/catalog,0,yes,,,"couldn't find a test specific for this, there are some tests related in keystone.tests.unit.test_v3_auth.py",
identity-v3-get-catalog,platform/compute/object,advisory,GET,/v3/auth/catalog,0,yes,,,"TC added in Tempest",
identity-v3-get-auth-projects,platform/compute,,GET,/v3/auth/projects,0,yes,,,"equivalent as far as I can tell to identity-v3-list-projects. couldn't find a test specific for this, there are some tests related in keystone.tests.unit.test_v3_auth.py",
,,,,,,,,,,
identity-v2-list-versions,,,GET,/,1,yes,,,Deprecated,

Can't render this file because it has a wrong number of fields in line 2.

9
working_materials/scoring.txt
View File

@ -294,20 +294,19 @@ identity-v3-api-discovery: [1,0,1] [1,1,1] [1,1,1] [1,1,1] [1] [94]*
identity-v3-catalog: [1,0,1] [1,1,1] [1,1,0] [1,1,1] [1] [85]*
identity-v3-list-projects: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
identity-v3-list-groups: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
identity-v3-tokens-create: [1,1,1] [1,1,1] [1,1,1] [1,1,0] [1] [92]*
identity-v3-tokens-validate: [1,1,1] [1,1,1] [1,1,0] [0,1,0] [1] [74]*
Notes:
* identity-v3-catalog is returned when the api for
identity-v3-tokens-create is called (GET /v3/auth/tokens). It is
important to consider it because end users may be relying on this
catalog for their apps (even though there are other API calls that
also show the catalog such as GET /v3/auth/catalog).
* identity-v3-list-projects and identity-v3-list-groups didn't have usable
tests in the past, but one was added for identity-v3-list-projects last year.
Providers like Fog.io
now actually use the /v3/users/{user_id}/[projects|groups] API's:
https://git.io/vX9S6
https://git.io/vX9SP
Capability became required 2017.09 but the only TC available was flagged
since it requires two sets of credentials. Capability needs additional TCs
or existing test should be changed to require only one set of credentials.
* identity-v3-change-password was considered here but it's applicability is
a bit hard to gauge: many systems using third-party authentication (such as
an LDAP/AD server, an external oauth system, etc) require password changes

1
working_materials/tabulated_scores.csv
View File

@ -106,6 +106,7 @@ identity-v3-api-discovery,1,0,1,1,1,1,1,1,1,1,1,1,1,94*
identity-v3-catalog,1,0,1,1,1,1,1,1,0,1,1,1,1,85*
identity-v3-list-projects,1,1,1,1,1,1,1,1,0,0,1,0,1,74*
identity-v3-list-groups,1,1,1,1,1,1,1,1,0,0,1,0,1,74*
identity-v3-tokens-create,1,1,1,1,1,1,1,1,1,1,1,0,1,92*
identity-v3-tokens-validate,1,1,1,1,1,1,1,1,0,0,1,0,1,74*
objectstore-object-copy,1,1,1,1,1,1,1,1,1,1,1,1,1,100*
objectstore-object-create,1,1,1,1,1,1,1,1,1,1,1,1,1,100*

Can't render this file because it has a wrong number of fields in line 25.