Boris Pavlovic 254c0d68e8 Input task templates and task cmd cleanup
Implement task templates based on jinja2.

This allow us to pass as a task jinja2 template
and it's argument via arguments --task-args and
--task-args-file that should be dict in JSON or YAML
presentations.

So now command looks like:

rally task start <file> --task-args <template-args-json-or-yaml> \
  --task-args-file <file-with-args-in-json-yaml>

If both --task-args and --task-args-file then file dict is updated
by task args file.

Extend rally CI performance job. Now we can set template args
via file with name: ${TASK}_args.yaml

Bonus:
* Better message on InvalidTask format
* Remove redudant catch of "keyboardinterrupt"
  it should be implement in different way.
* Replace ' -> " in rally.cmd.commands.task
  and tests.unit.cmd.commands.task
* Imporve a bit CLI messages on rally task start
* Remove old plot2html command (it's enough deprecated)
* Improve test coverage of rally/cmd/commands/task
* Fix rally/cmd/commands/validate return 1 if bad format
* Write errors to stderr (in whole cmd/commands/task.py)

Change-Id: I7dadf2986bb10407865bc73bb2fb8c96a5162d9a
2015-01-15 16:29:34 +03:00
..
2014-10-07 13:50:40 +00:00
2014-10-07 13:50:40 +00:00

Rally Gate Jobs

For each patch submitted for review on Gerrit, there is a set of tests called gate jobs to be run against it. These tests check whether the Rally code works correctly after applying the patch and provide additional guarantees that it won't break the software when it gets merged. Rally gate jobs contain tests checking the codestyle (via pep8), unit tests suites, functional tests and a set of Rally benchmark tasks that are executed against a real devstack deployment.

rally-gate.sh

This script runs a set of real Rally benchmark tasks and fetches their results in textual / visualized form (available via a special html page by clicking the corresponding job title in Gerrit). It checks that scenarios don't fail while being executed against a devstack deployment and also tests SLA criteria to ensure that benchmark tasks have completed successfully.

rally-integrated.sh

This script runs a functional tests suite for Rally CLI. The tests call a range of Rally CLI commands and check that their output contains the expected data.