refstack/specs/ocata/implemented/associate-test-result-to-product.rst

Test Results to Vendor Products Association

Launchpad blueprint:

• https://blueprints.launchpad.net/refstack/+spec/result-listing-page
• https://blueprints.launchpad.net/refstack/+spec/marketplace-product-model

This specification defines the processes and mechanisms to associate test results to vendor products.

Problem description

So far, community test results are shared to the public anonymously, with no linkage to users, vendors or products. DefCore, OpenStack Foundation (OSF) and community user feedback revealed strong interest in enabling RefStack to show the results with linkage to vendors, products and test statuses. This is especially important for those test results that were used for official OpenStack Powered Logo applications.

With the recent implementation of vendor and product registration process, it is now possible to associate a particular test result record to a vendor's product of a particular version.

Note

Test results are not associated directly to vendors. Test results are associated to products which belong to vendors. A product may have many versions.

RefStack interprets OSF/DefCore requirements as following:

• Vocabulary:
  • product version: a product version is a version of a product.
  • product vendor: a product vendor is a vendor who owns the product.
• Model:
  • A product version can be used to deploy many clouds.
  • A product version can have many tests.
  • Test results will be associated to product versions (not products).
• Use cases:
  • Only the user who uploads the result and is also an admin of the product vendor or an interop admin can associate a test result to a product version.
  • Once a test is associated to a product version, only the admins of the product vendor or interop admins can perform CRUD operations on the test result.

Proposed change

• Add a new columns to the "test" table in the database:

  • product_version_id: this field stores the product version ID that the test run is associated to.

• Associate a test result to a product

  • The association must be initiated by a user who creates the test result record and is an admin of the vendor which owns the product. The association is done by updating the product_version_id field of the test result with the ID of the product version from the "version" table.
  • Once a test result is associated to a product version, the test result can not be deleted until it is disassociate from a product.
  • Once a test result is associated to a product version, only interop (i.e. RefStack site) admins or vendor admins can manage the test result (e.g. making decision of sharing the test result record).
  • A product version can't be deleted if there are tests associated to it.

  Note

  Currently, RefStack has not implemented "user role" to differentiate the roles of users in a "user group". As such, at this time, all users in a "user group" are admin users.

Alternatives

An alternative method to associate a test record to a product is by matching values of the "cpid" field (in the "version" table) and the "cpid" field (in the "test" table). The major concern and shortcomings of this option are as follows:

• The "cpid" field is a user input parameter, therefore it is not guarantee to be unique.
• The cpid fields may be identical for test results run by different users against the same public cloud instance.
• A product may have many cloud instances which are identified by different cpids.

Data model impact

Add a new column named "product_version_id" to the existing "test" table. This field can be null.

Column	Type
product_version_id	varchar(36)	FK

Note: The user input product_version_id must exist in the "version" table.

REST API impact

The following REST APIs will be modified.

Update result

• Description:

  This API will be used to make update to a test entry of the "test" table. To begin with, only the owner user who uploaded the test result, can make update to the product_version_id filed. Once a test is associated to a product version, only interop or vendor admins can make updates to a test result.

• Method type: PUT

• URI: v1/results/{result_id}/

• Normal Response Codes:

  • OK (200)

• Error Response Codes:

  • Bad Request (400)
  • Unauthorized (401)
  • Forbidden (403)
  • Not found (404)

• Request parameters:

  Parameter	Style	Type	Description
  result_id	URI	csapi:UUID	Test result ID for marking.

• JSON schema definition for the body data:

  { { "verification_status" : 1, "product_version_id": "85346866-307f-4052-ba31-ff6270635e14", "required": [] } }

  Note

  • Although the verification_status column is listed here for completeness of the API body schema, this field can only be updated by interop admin as described in spec https://review.opendev.org/#/c/343954/ .
  • Update request including the "verification_status" field will return "Forbidden(403)" if the requester is not an interop admin.

• JSON schema definition for the response data: N/A

List results

• Description: (No update)

• Method type: GET (No update)

• URI: v1/results/ (No update)

• Normal Response Codes: (No update)

• Error Response Codes: (No update)

• Request parameters: (No update)

  Add the following parameter to the existing ones:

  Parameter	Style	Type	Description
  product_version_id (optional)	query	xsd:string	Only return the test records belonging to this product_version_id.

• JSON schema definition for the body data: N/A

• JSON schema definition for the response data:

  Update to add product_version_id to the response body.

  { pagination: { current_page: 6, total_pages: 37 }, results: [ { url: "https://refstack.openstack.org/#/results/7943e04a-2b95-453c-b627-8a24b2c6faa0", created_at: "2016-07-25 02:24:34", meta: { }, id: "7943e04a-2b95-453c-b627-8a24b2c6faa0", duration_seconds: 0, verification_status : 0, product_version_id: "" }, { url: "https://refstack.openstack.org/#/results/91ae10c5-ecf5-4823-81d4-09836dc212cf", created_at: "2016-07-13 18:37:53", meta: { shared: ""true"", target: "compute", guideline: "2016.01.json" }, id: "91ae10c5-ecf5-4823-81d4-09836dc212cf", duration_seconds: 6037, verification_status : 1, product_version_id: "68668534-307f-4052-ba31-ff6270635e14" }, ........ ] }

Show result details

• Description: (No update)

• Method type: GET (No update)

• URI: v1/results/{result_id} (No update)

• Normal Response Codes: (No update)

• Error Response Codes: (No update)

• Request parameters: (No update)

• JSON schema definition for the body data: N/A

• JSON schema definition for the response data:

  Update to add product_version_id to the response body.

  { user_role: "user", created_at: "2016-07-13 18:37:53", meta: { shared: ""true"", target: "compute", guideline: "2016.01.json" }, id: "91ae10c5-ecf5-4823-81d4-09836dc212cf", duration_seconds: 6037, verification_status : 1, product_version_id; "68668534-307f-4052-ba31-ff6270635e14", results: [ "tempest.api.compute.certificates.test_certificates.CertificatesV2TestJSON.test_create_root_certificate", "tempest.api.compute.certificates.test_certificates.CertificatesV2TestJSON.test_get_root_certificate", ...... ] }

Security impact

None.

Notifications impact

None.

Other end user impact

None

Performance Impact

None

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

Paul Van Eck Andrey Pavlov

Other contributors:

TBD

Work Items

• Add the defined additional field to the "test" table.
• Develop business and UI code to enable association of a test result to a product.

Dependencies

None

Testing

• Add unit tests to test the newly added code.

Documentation Impact

None

References

None