openinfra/refstack-client
Code Issues Proposed changes

Update Readme

Browse Source 
* remove old links
* mention new options of setup_env script
* mention OpenStack Marketing Programs and interop repo
* mention ansible-role-refstack-client

Change-Id: Ia62b400d38eeeebd22e79b33616a04877b9ea33b
This commit is contained in:
Martin Kopec 2021-06-18 10:26:30 +00:00
parent 7fbf6dad17
commit fa73ef2524
1 changed files with 75 additions and 41 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

116
README.rst
View File

@ -2,18 +2,10 @@
RefStack Client
===============
RefStack-client team and repository tags
########################################
.. image:: https://governance.openstack.org/tc/badges/refstack-client.svg
:target: https://governance.openstack.org/tc/reference/tags/index.html
Overview
########
refstack-client is a command line utility that allows you to execute Tempest
``refstack-client`` is a command line utility that allows you to execute Tempest
test runs based on configurations you specify. When finished running Tempest
it can send the passed test data to a RefStack API server.
@ -24,7 +16,7 @@ We've created an "easy button" for Ubuntu, Centos, RHEL and openSUSE.
1. Make sure you have ``git`` installed
2. Get the refstack client: ``git clone https://opendev.org/osf/refstack-client.git``
3. Go into the refstack-client directory: ``cd refstack-client``
3. Go into the ``refstack-client`` directory: ``cd refstack-client``
4. Run the "easy button" setup: ``./setup_env``
**Options:**
@ -34,24 +26,34 @@ We've created an "easy button" for Ubuntu, Centos, RHEL and openSUSE.
b. -t option allows to specify tag in Tempest repository which will be installed.
For example: execute ``./setup_env -t tags/3`` to install Tempest tag-3.
c. By default, Tempest will be installed from commit
By default, Tempest will be installed from commit
e64c78dcf720202a0542bb1e1184f5229a11524f (Oct, 2019).
c. -p option allows to specify python version - python 2.7 (-p 2), 3.6 (-p 3)
or any specific one by -p X.X.X. Default to python 3.6.
d. -q option makes ``refstack-client`` run quitely - if ``.tempest``
directory exists ``refstack-client`` is considered as installed.
e. -s option makes ``refstack-client`` use ``python-tempestconf`` from the
given source (path) - used when running f.e. in Zuul.
Usage
#####
1. Prepare a tempest configuration file that is customized to your cloud
environment. Samples of minimal Tempest configurations are provided in
the ``etc`` directory in ``tempest.conf.sample`` and ``accounts.yaml.sample``.
1. Prepare a tempest configuration file (or let ``refstack-client`` generate it
for you, see step #4) that is customized to your cloud environment.
Samples of minimal Tempest configurations are provided in the ``etc``
directory in ``tempest.conf.sample`` and ``accounts.yaml.sample``.
Note that these samples will likely need changes or additional information
to work with your cloud.
Note: Use Tempest Pre-Provisioned credentials_ to provide user test accounts. ::
**Note**: Use Tempest Pre-Provisioned credentials_ to provide user test
accounts.
.. _credentials: https://docs.openstack.org/tempest/latest/configuration.html#pre-provisioned-credentials
2. Go into the refstack-client directory::
2. Go into the ``refstack-client`` directory::
cd ~/refstack-client
@ -59,7 +61,7 @@ Usage
source .venv/bin/activate
4. Generate tempest.conf using refstack-client::
4. (optional) Generate tempest.conf using ``refstack-client``::
refstack-client config --use-test-accounts <path to account file>
@ -74,7 +76,9 @@ Usage
5. Validate your setup by running a short test::
refstack-client test -c <Path of the tempest configuration file to use> -v -- --regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
refstack-client test \
-c <Path of the tempest configuration file to use> -v -- \
--regex tempest.api.identity.v3.test_tokens.TokensV3Test.test_create_token
6. Run tests.
@ -88,10 +92,20 @@ Usage
For example::
refstack-client test -c ~/tempest.conf -v --test-list "https://refstack.openstack.org/api/v1/guidelines/2018.02/tests?target=platform&type=required&alias=true&flag=false"
refstack-client test \
-c ~/tempest.conf -v \
--test-list "https://refstack.openstack.org/api/v1/guidelines/2020.11/tests?target=platform&type=required&alias=true&flag=false"
This will run only the test cases required by the 2018.02 guidelines
that have not been flagged.
This will run only the test cases required by the 2020.11 guidelines under
Platform OpenStack Marketing Program that have not been flagged. More about
the marketing programs at `Interop and OpenStack Marketing Programs`_.
For example tests under the compute program are available:
https://refstack.openstack.org/api/v1/guidelines/2020.11/tests?target=compute&type=required&alias=true&flag=false
Tests of add-on programs can be found similarly, f.e. tests under dns program:
https://refstack.openstack.org/api/v1/guidelines/dns.2020.11/tests?target=dns&type=required&alias=true&flag=false
or tests under orchestration program:
https://refstack.openstack.org/api/v1/guidelines/orchestration.2020.11/tests?target=orchestration&type=required&alias=true&flag=false
**Note:**
@ -118,7 +132,7 @@ Usage
6. Upload your results.
If you previously ran a test with refstack-client without the ``--upload``
If you previously ran a test with ``refstack-client`` without the ``--upload``
option, you can later upload your results to a RefStack API server
with your digital signature. By default, the results are private and you can
decide to share or delete the results later.
@ -127,7 +141,7 @@ Usage
refstack-client upload <Path of results file> -i <path-to-private-key>
The results file is a JSON file generated by refstack-client when a test has
The results file is a JSON file generated by ``refstack-client`` when a test has
completed. This is saved in .tempest/.stestr. When you use the
``upload`` command, you can also override the RefStack API server uploaded to
with the ``--url`` option.
@ -143,31 +157,23 @@ Usage
Intructions for uploading data with signature can be found at
https://opendev.org/osf/refstack/src/branch/master/doc/source/uploading_private_results.rst
7. Create a JSON web token to use for authentication to your privately
uploaded data
In order to authenticate to the refstack-server to which you have uploaded
your data, you will need to generate a JSON webtoken. To generate a valid
token, use the command::
jwt --key="$( cat %path to private key% )" --alg=RS256 user_openid=%openstackid% exp=+100500
To test authentication in the API, use the command::
curl -k --header "Authorization: Bearer %token%" https://localhost.org/v1/profile
8. List uploaded test set.
7. View uploaded test set.
You can list previously uploaded data from a RefStack API server by using
the following command::
refstack-client list --url <URL of the RefStack API server>
refstack-client list --url <URL of the RefStack API server> -i <path to private key>
Alternatively, if you uploaded the results to the official RefStack_ server
you can view them by using RefStack_ page where all uploaded results
associated with the particular account (the account private key used to
upload the results belongs to) will be shown and may be further managed.
Tempest hacking
###############
By default, refstack-client installs Tempest into the ``.tempest`` directory.
By default, ``refstack-client`` installs Tempest into the ``.tempest`` directory.
If you're interested in working with Tempest directly for debugging or
configuration, you can activate a working Tempest environment by
switching to that directory and using the installed dependencies.
@ -177,4 +183,32 @@ switching to that directory and using the installed dependencies.
and run tests manually with ``tempest run``.
This will make the entire Tempest environment available for you to run,
including ``tempest run``.
including ``tempest run``. More about Tempest can be found at its documentation_.
.. _documentation: https://docs.openstack.org/tempest/latest/
Interop and OpenStack Marketing Programs
########################################
The tests ``refstack-client`` runs are defined within interop_ repository
and divided into several OpenStack Marketing Programs, the list of the programs
can be found at RefStack_ page.
.. _interop: https://opendev.org/osf/interop
.. _RefStack: https://refstack.openstack.org/#/
ansible-role-refstack-client
############################
We have created an ansible role called ansible-role-refstack-client_ in order
to simplify and automate running of ``refstack-client``. The role can be easily
integrated to an automation machinery - f.e. we use the role for running
``refstack-client`` on a devstack_ environment in Zuul where we run tests of
every OpenStack Marketing Program of the current guideline. The latest builds
can be found here__.
.. _ansible-role-refstack-client: https://opendev.org/x/ansible-role-refstack-client
.. _devstack: https://opendev.org/openstack/devstack/
.. __builds: https://zuul.openstack.org/builds?project=x%2Fansible-role-refstack-client