rally-openstack/doc/source/verification/verifiers.rst

Verifiers

What is it?

Verifier Plugin is a compatibility layer between Rally and the specific tool (such as Tempest) which runs tests. It implements features like installation, configuration, upgrades, running, etc in terms of the tool. It is a driver in other words. It is a pluggable entity, which means that you can easily add support for whatever tool you want (see howto-add-support-for-new-tool page for more information). Even more, you can deliver such plugin separately from Rally itself, but we firmly recommend to push a change to Rally upstream (see contribute guide), so Rally core-team will able to review it and help to improve.

Verifier is an instance of the Verifier Plugin. It is an installed tool. For example, "Tempest" is a set of functional tests, it is Verifier Plugin (we have a plugin for it). Installed Tempest 12.0 from https://github.com/openstack/tempest in a virtual environment is the verifier.

Verifier is not aligned to any particular deployment like it was in the past, you can use one verifier for testing unlimited number of deployments (each deployment will have separate configuration files for the tool).

Verifier & Verifier Plugin are the main entities which Verification component operates with. Another one is the verifications results.

Verifier statuses

All verifiers can be in next statuses:

• init - Initial state. It appears while you call rally verify create-verifier command and installation step is not yet started.
• installing - Installation of the verifier is not a quick task. It is about cloning tool, checking packages or installing virtual environments with all required packages. This state indicates that this step is in the process.
• installed - It should be one of your favourite states. It means that everything is ok and you can start verifying your cloud.
• updating - This state identifies the process of updating verifier (version, source, packages, etc.).
• extending - The process of extending a verifier by its plugins.
• failed - Something went wrong while installation.

Verification statuses

• init - Initial state. It appears instantly after calling rally verify start command before the actual run of verifier's tool.
• running - Identifies the process of execution tool.
• finished- Verification is finished without errors and failures.
• failed - Verification is finished, but there are some failed tests.
• crashed - Unexpected error had happened while running verification.

Known verifier types

Out of the box

You can execute command rally verify list-plugins locally to check available verifiers in your environment.

Cut down from Global plugin-reference page:

Third-party

Nothing here yet.