166 lines
6.5 KiB
ReStructuredText
166 lines
6.5 KiB
ReStructuredText
========================
|
||
RefStack Client
|
||
========================
|
||
|
||
RefStack-client team and repository tags
|
||
########################################
|
||
|
||
|
||
.. image:: https://governance.openstack.org/badges/refstack-client.svg
|
||
:target: https://governance.openstack.org/reference/tags/index.html
|
||
|
||
|
||
Overview
|
||
########
|
||
|
||
refstack-client is a command line utility that allows you to execute Tempest
|
||
test runs based on configurations you specify. When finished running Tempest
|
||
it can send the passed test data to a RefStack API server.
|
||
|
||
Environment setup
|
||
#################
|
||
|
||
We've created an "easy button" for Ubuntu, Centos, RHEL and openSuSe.
|
||
|
||
1. Make sure you have ``git`` installed
|
||
2. Get the refstack client: ``git clone https://github.com/openstack/refstack-client``
|
||
3. Go into the refstack-client directory: ``cd refstack-client``
|
||
4. Run the "easy button" setup: ``./setup_env``
|
||
|
||
**Options:**
|
||
|
||
a. -c option allows to specify SHA of commit or branch in Tempest repository
|
||
which will be installed.
|
||
|
||
b. -t option allows to specify tag in Tempest repository which will be installed.
|
||
For example: execute ``./setup_env -t tags/3`` to install Tempest tag-3.
|
||
|
||
c. By default, Tempest will be installed from commit
|
||
cc255bbbf431e114a4fc0ef587cd3d72333f750a (October, 2017).
|
||
|
||
Usage
|
||
#####
|
||
|
||
1. Prepare a tempest configuration file that is customized to your cloud
|
||
environment.
|
||
|
||
Note: Use Tempest Pre-Provisioned credentials_ to provide user test accounts. ::
|
||
|
||
.. _credentials: https://docs.openstack.org/tempest/latest/configuration.html#pre-provisioned-credentials
|
||
|
||
2. Go into the refstack-client directory::
|
||
|
||
cd ~/refstack-client
|
||
|
||
3. Source to use the correct Python environment::
|
||
|
||
source .venv/bin/activate
|
||
|
||
4. Validate your setup by running a short test::
|
||
|
||
refstack-client test -c <Path of the tempest configuration file to use> -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
|
||
|
||
5. Run tests.
|
||
|
||
To run the entire API test set::
|
||
|
||
refstack-client test -c <Path of the tempest configuration file to use> -v
|
||
|
||
To run only those tests specified in an OpenStack Powered (TM) Guideline::
|
||
|
||
refstack-client test -c <Path of the tempest configuration file to use> -v -–test-list <Absolute path of test list>
|
||
|
||
For example::
|
||
|
||
refstack-client test -c ~/tempest.conf -v --test-list "https://refstack.openstack.org/api/v1/guidelines/2017.09/tests?target=platform&type=required&alias=true&flag=false"
|
||
|
||
This will run only the test cases required by the 2017.09 guidelines
|
||
that have not been flagged.
|
||
|
||
**Note:**
|
||
|
||
a. Adding the ``-v`` option will show the Tempest test result output.
|
||
b. Adding the ``--upload`` option will have your test results be uploaded to the
|
||
default RefStack API server or the server specified by ``--url``.
|
||
c. Adding the ``--test-list`` option will allow you to specify the file path or URL of
|
||
a test list text file. This test list should contain specific test cases that
|
||
should be tested. Tests lists passed in using this argument will be normalized
|
||
with the current Tempest environment to eliminate any attribute mismatches.
|
||
d. Adding the ``--url`` option will allow you to change where test results should
|
||
be uploaded.
|
||
e. Adding the ``-r`` option with a string will prefix the JSON result file with the
|
||
given string (e.g. ``-r my-test`` will yield a result file like
|
||
'my-test-0.json').
|
||
f. Adding ``--`` enables you to pass arbitrary arguments to tempest run.
|
||
After the first ``--``, all other subsequent arguments will be passed to
|
||
tempest run as is. This is mainly used for quick verification of the
|
||
target test cases. (e.g. ``-- --regex tempest.api.identity.v2.test_token``)
|
||
g. If you have provisioned multiple user/project accounts you can run parallel
|
||
test execution by enabling the ``--parallel`` flag.
|
||
|
||
Use ``refstack-client test --help`` for the full list of arguments.
|
||
|
||
6. Upload your results.
|
||
|
||
If you previously ran a test with refstack-client without the ``--upload``
|
||
option, you can later upload your results to a RefStack API server
|
||
anonymously or with your digital signature. By default, results uploaded
|
||
with signature are private and you can decide to share or delete the results
|
||
later. Uploading results with signature is the recommended option.
|
||
|
||
Following is the command to upload your result anonymously::
|
||
|
||
refstack-client upload <Path of results file>
|
||
|
||
The results file is a JSON file generated by refstack-client when a test has
|
||
completed. This is saved in .tempest/.testrepository. When you use the
|
||
``upload`` command, you can also override the RefStack API server uploaded to
|
||
with the ``--url`` option.
|
||
|
||
Alternatively, you can use the ``upload-subunit`` command to upload results
|
||
using an existing subunit file. This requires that you pass in the Keystone
|
||
endpoint URL for the cloud that was tested to generate the subunit data::
|
||
|
||
refstack-client upload-subunit --keystone-endpoint http://some.url:5000/v3 <Path of subunit file>
|
||
|
||
Result can also be uploaded with signature by adding
|
||
``-i <path-to-private-key>`` option. Intructions for uploading data with
|
||
signature can be found at
|
||
https://github.com/openstack/refstack/blob/master/doc/source/uploading_private_results.rst
|
||
|
||
7. Create a JSON web token to use for authentication to your privately
|
||
uploaded data
|
||
|
||
In order to authenticate to the refstack-server to which you have uploaded
|
||
your data, you will need to generate a JSON webtoken. To generate a valid
|
||
token, use the command::
|
||
|
||
jwt --key="$( cat %path to private key% )" --alg=RS256 user_openid=%openstackid% exp=+100500
|
||
|
||
To test authentication in the API, use the command::
|
||
|
||
curl -k --header "Authorization: Bearer %token%" https://localhost.org/v1/profile
|
||
|
||
8. List uploaded test set.
|
||
|
||
You can list previously uploaded data from a RefStack API server by using
|
||
the following command::
|
||
|
||
refstack-client list --url <URL of the RefStack API server>
|
||
|
||
|
||
Tempest hacking
|
||
###############
|
||
|
||
By default, refstack-client installs Tempest into the ``.tempest`` directory.
|
||
If you're interested in working with Tempest directly for debugging or
|
||
configuration, you can activate a working Tempest environment by
|
||
switching to that directory and using the installed dependencies.
|
||
|
||
1. ``cd .tempest``
|
||
2. ``source ./.venv/bin/activate``
|
||
and run tests manually with ``tempest run``.
|
||
|
||
This will make the entire Tempest environment available for you to run,
|
||
including ``tempest run``.
|