![Andrey Kurilin](/assets/img/avatar_default.png)
Recently, we added new job "gate-rally-dsvm-verify" for Rally[1]. This job implied functional testing integration Rally with Tempest[2]. This patch adds tests/ci/rally-verify.sh script, which is used by new job. rally-verify.sh does: - tempest installation - run "rally verify start" twice and print results - compare results of two verifications - list verifications - generate html page based on results To implement gate-rally-dsvm-verify some changes were requered to existing Rally code: - Added ability for rally/ui/utils.py to accept arguments to render html-pages - Fixed logging debug-messages in tempest verifier - Fixed check "is debug mode turned on or not"(also, added hacking rule for it) TODO for future patches: - add launch of rally task for Tempest - add launch of random test set - add check for successful tests [1] https://review.openstack.org/#/c/137232 [2] https://www.mirantis.com/blog/rally-openstack-tempest-testing-made-simpler Closes-Bug: #1400465 Closes-Bug: #1400518 Change-Id: I8e1fbab22c2da109bbc442f040fe259e5d22a62a
Rally Style Commandments
- Step 1: Read the OpenStack Style Commandments http://docs.openstack.org/developer/hacking/
- Step 2: Read on
Rally Specific Commandments
- * [N30x] - Reserved for rules related to
mock
library -
- [N301] - Ensure that
assert_*
methods frommock
library is used correctly - [N302] - Ensure that nonexistent "assert_called" is not used
- [N303] - Ensure that nonexistent "assert_called_once" is not used
- [N301] - Ensure that
- * [N310-N314] - Reserved for rules related to logging
-
- [N310] - Ensure that
rally.log
is used instead ofrally.openstack.common.log
- [N311] - Validate that debug level logs are not translated
- [N312] - Validate correctness of debug on check.
- [N310] - Ensure that
- * [N32x] - Reserved for rules related to assert* methods
-
- [N320] - Ensure that
assertTrue(isinstance(A, B))
is not used - [N321] - Ensure that
assertEqual(type(A), B)
is not used - [N322] - Ensure that
assertEqual(A, None)
andassertEqual(None, A)
are not used - [N323] - Ensure that
assertTrue/assertFalse(A in/not in B)
are not used with collection contents - [N324] - Ensure that
assertEqual(A in/not in B, True/False)
andassertEqual(True/False, A in/not in B)
are not used with collection contents
- [N320] - Ensure that