456b885255
Previously, after the run_tempest script, was run, all post-processing was done only if the returncode was 0. If it was non-zero, an error message was given. This was an oversight, as the run_tempest script will return a code of 1 when any test case fails. This leaves the unintended side effect of refstack not parsing the results or producing the output json when Tempest successfully runs, but a test case fails. Checking for the existence of the expected subunit file will at least validate that the Tempest test was at least started successfully, and didn't fail to run due to some misconfiguration in the environment. Change-Id: I541b798db41b713525efeb8dedfb9347be780a3f
refstack-client
refstack-client is a command line utility that allows you to execute Tempest test runs based on configurations you specify. When finished running Tempest it sends the passed test data back to the Refstack API server.
Environment setup
We've created an "easy button" for Ubuntu, Centos, RHEL and openSuSe.
$ ./setup_env
- Options:
-
a. -t option allows to specify tag in Tempest repository which will be installed. By default, Tempest from last tag release will be used.
Usage
Prepare a tempest configuration file that is customized to your cloud environment.
Change the directory to the refstack-client directory (i.e. cd ~/refstack-client).
Source to use the correct Python environment.
source .venv/bin/activate
Test your cloud by typing:
./refstack-client test -c <Path of the tempest configuration file to use>
Note:
- Adding -v option will show the summary output.
- Adding -vv option will show the Tempest test result output.
- Adding -t option will only test a particular test case or a test group. This option can be used for quick verification of the target test cases (i.e. -t "tempest.api.identity.admin.test_roles").
- Adding --url option will upload the test results to the specified Refstack API server instead of the default Refstack API server. server instead of the default Refstack API server.
- Adding --offline option will have your test results not be uploaded.
Upload:
If you previously ran a test with refstack-client using the --offline option, you can upload your results to a Refstack API server by using the following command:
./refstack-client upload <Path of results file>
The results file is a JSON file generated by refstack-client when a test has completed. This is saved in .venv/src/tempest/.testrepository. When you use the 'upload' command, you can also override the Refstack API server uploaded to with the --url option.