rally-openstack/doc/source/verification/howto/add_new_reporter.rst

HowTo add new reporting mechanism

Reporting mechanism for verifications is pluggable. Custom plugins can be used for custom output formats or for exporting results to external systems.

We hardly recommend to read plugins page to understand how do Rally Plugins work.

Spec

All reporters should inherit rally.verification.reporter.VerificationReporter and implement all abstract methods. Here you can find its interface:

rally.verification.reporter.VerificationReporter

Example of custom JSON Reporter

Basically, you need to implement only two methods "validate" and "generate".

Method "validate" should check that destination of the report is right. Method "generate" should build a report or export results somewhere; actually, it is up to you what it should do but return format is strict, see Spec section for what it can return.

import json

from rally.verification import reporter


@reporter.configure("summary-in-json")
class SummaryInJsonReporter(reporter.VerificationReporter):
    """Store summary of verification(s) in JSON format"""

    # ISO 8601
    TIME_FORMAT = "%Y-%m-%dT%H:%M:%S%z"

    @classmethod
    def validate(cls, output_destination):
        # we do not have any restrictions for destination, so nothing to
        # check
        pass

    def generate(self):
        report = {}

        for v in self.verifications:
            report[v.uuid] = {
                "started_at": v.created_at.strftime(self.TIME_FORMAT),
                "finished_at": v.updated_at.strftime(self.TIME_FORMAT),
                "status": v.status,
                "run_args": v.run_args,
                "tests_count": v.tests_count,
                "tests_duration": v.tests_duration,
                "skipped": v.skipped,
                "success": v.success,
                "expected_failures": v.expected_failures,
                "unexpected_success": v.unexpected_success,
                "failures": v.failures,
                # v.tests includes all information about launched tests,
                # but for simplification of this fake reporters, let's
                # save just names
                "launched_tests": [test["name"]
                                   for test in v.tests.values()]
            }

        raw_report = json.dumps(report, indent=4)

        if self.output_destination:
            # In case of output_destination existence report will be saved
            # to hard drive and there is nothing to print to stdout, so
            # "print" key is not used
            return {"files": {self.output_destination: raw_report},
                    "open": self.output_destination}
        else:
            # it is something that will be print at CLI layer.
            return {"print": raw_report}