Recently, we added new job "gate-rally-dsvm-verify" for Rally[1].
This job implied functional testing integration Rally with Tempest[2].
This patch adds tests/ci/rally-verify.sh script, which is used by new job.
rally-verify.sh does:
- tempest installation
- run "rally verify start" twice and print results
- compare results of two verifications
- list verifications
- generate html page based on results
To implement gate-rally-dsvm-verify some changes were requered to
existing Rally code:
- Added ability for rally/ui/utils.py to accept arguments to render html-pages
- Fixed logging debug-messages in tempest verifier
- Fixed check "is debug mode turned on or not"(also, added hacking rule for it)
TODO for future patches:
- add launch of rally task for Tempest
- add launch of random test set
- add check for successful tests
[1] https://review.openstack.org/#/c/137232
[2] https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler
Closes-Bug: #1400465
Closes-Bug: #1400518
Change-Id: I8e1fbab22c2da109bbc442f040fe259e5d22a62a
eef45465ca