c5b1920556
This commit bakes several cleanups into one patch. First it clearly separates devstack and tempest specs by creating a separate tempest subdir for specs. The next step is moving all implemented specs/bps into the implemented dir to indicate this. The last is deleting specs which were outdated and never implemented. Change-Id: I86331ac279b6a9cdb94ed4c43e5b0c41697e6a7d
2.7 KiB
Fuzzy test framework
https://blueprints.launchpad.net/tempest/+spec/fuzzy-test
The negative testing framework tests single aspects of an API server in an automatic manner based on json schema's. Using this functionality fuzzy tests can be created with the same process but with a different focus.
Problem description
Tempest does not have any coverage of security aspects. Using such a framework to detect security vulnerabilities will be an important new testing area for Tempest.
Proposed change
Focus of this framework is vulnerabilities identification and denial of service (DoS) attacks. It can use the api schema definitions as input to produce flawed requests.
Denial of service
DoS patterns are easy to validate and are considered as first step. A certain service produces a portion of flawed requests together with valid requests like authentication. To validate if a DoS attack was successful a set of usual Tempest API test can be used for this purpose. To produce the needed load the stress test framework of Tempest can be used to produce a higher load rate.
Vulnerabilities identification
Identification of security issues can be very complex and automatic detection can be only done very limited. To identify issues that may are vulnerabilities the following data needs to be analyzed:
- Result of a request: Success codes or internal server errors are potential threats that need be analyzed and logged by the framework.
- System availability: A check if all the OpenStack components are available be used as validation.
- Request logging: Tempest rest client loges all requests. This is needed to identify request or scenarios that causes a threat.
Data generation
The data generation should support different sources and this could be a possible interface to 3PP fuzzy testing products. With the multibackend functionality of the negative testing framework (see https://review.openstack.org/#/c/73982/) different test generators can be used. These generator must stick to the interface define in the base class (tempest.common.negative.base).
Alternatives
Use third party product for fuzzy test generation and don't integrate it in Tempest.
Implementation
Assignee(s)
- Primary assignees:
-
Marc Koderer (mkoderer)
Milestones
- Target Milestone for completion:
-
Juno-3
Work Items
- Will be tracked in:
Dependencies
None.