openinfra/interop
Code Issues Proposed changes

Add the procedure.rst for defcore version 2015.07

Browse Source 
In the defcore version 2015.05, if only run tempest, the result is not json
file, so we have no way to use it.

In the defcore version 2015.07, we recommend run testcase of tempest under
refstack-client. When the test is finished, there are two copy files in the
<.tempest/.testrepository/>. The one is xxx.json, that is only pass testcases
in the file. The other file is xxx about detail infos.

In this patch, add a method about how to upload the result(json file) to
refstack-server and check it.

Change-Id: I61ed39730149b6f038d8e0b3793fed6e35219333
This commit is contained in:
wangxf 2015-10-30 14:20:20 +08:00 committed by WangXiaoFei
parent 125f197265
commit e94c3f8d9a
1 changed files with 17 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
2015.07/procedure.rst
View File

@ -10,7 +10,8 @@ Tempest can be run standalone, or under a test runner such as refstack-client
or rally. If only testing against Defcore capabilities, you can use the
--load-list argument and a file containing a list of the Defcore tests. If
run with the refstack-client, test output will be parsed to list only
passing tests in a json formatted file.
passing tests in a json formatted file. We recommend running under
refstack-client.
The test names of the capabilities are derived from a recent release of
Tempest, from the time of capabilitiy approval. Keep in mind that Tempest
@ -39,7 +40,7 @@ or CentOS 7 have been verified) with administrator privileges.
* Download the refstack client:
``git clone https://git.openstack.org/stackforge/refstack-client``
``git clone https://git.openstack.org/openstack/refstack-client``
* In the refstack-client directory, install tempest and required dependencies.
You may specify a specific tag of tempest with the -t option. refstack-client
@ -56,21 +57,27 @@ or CentOS 7 have been verified) with administrator privileges.
configuration guide at
https://git.openstack.org/cgit/openstack/tempest/tree/doc/source/configuration.rst
* Once you have a working config, run tempest. You can run within the refstack
client, or run tempest directly. For direct running, from the refstack-client
directory::
* You can run within the refstack, from the refstack-client directory:
cd .tempest
./run_tempest.sh -C <your_tempest.conf> -- --load-list 2015.07.required.txt
./refstack-client test -c ~/tempest.conf -vv --test-list
http://git.openstack.org/cgit/openstack/defcore/plain/2015.07/2015.07.required.txt
* Review the test results, and when you're satisfied, send them to
interop@openstack.org. The results are stored in a json file in the directory
* Review the test results, and when you're satisfied, upload it to Refstack server
then send them to interop@openstack.org.
./refstack-client upload <Path of results file>
* The results are stored in a json file in the directory. You can also check your
result on the Refstack server https://refstack.openstack.org:
``.tempest/.testrepository``
* Every effort should be made to pass all of the required tests, but you
will want to compare any lists of failed tests to the list of flagged tests.
http://git.openstack.org/cgit/openstack/defcore/tree/2015.07/2015.07.required.txt
http://git.openstack.org/cgit/openstack/defcore/tree/2015.07/2015.07.flagged.txt
The refstack.openstack.org also gives you result where you can easily identify
exactly which tests still need to be passed.
Known Issues and Recommendations
--------------------------------