History

Kevin Benton cd16f8b85c Use new network for each subnet scenario run

Using _get_or_create_network was resulting in all concurrent
runners creating/updating/deleting subnets on a single Neutron
network. This was problematic for two reasons.

First, a network in Neutron in any normal deployment does not
have that many concurrent subnet create/update/deletes happening
on it at once because networks very rarely have more than 1 or 2
subnets on them.

Second, it made it appear as though Neutron was having
performance problems for handling lots of concurrent subnet
requests because Neutron internally performs overlapping checks
for subnets within a given network.

So we weren't getting performance measurements for normal heavy
usage (lots of subnet activity across lots of different networks)
and we were getting performance measurements for a non-standard
case (significant subnet activity within a single network).

This adjusts the scenario to always create a network for each
test run, which brings it inline with normal heavy use deployment
behavior.

Change-Id: I254dbd166a79ca07e18a61da92bc536b035452e1

2017-01-26 03:25:34 -08:00

[Verify] Add docs with common workflow for verifications

2017-01-25 22:24:42 +04:00

functional

[CLI] Allow changing deployment config on recreate

2017-01-25 21:06:06 +03:00

hacking

[cli]remove deprecated rally show command

2016-12-27 21:39:55 +08:00

unit

Use new network for each subnet scenario run

2017-01-26 03:25:34 -08:00

__init__.py

Move tests to root dirrectory and fix tox

2013-08-28 19:08:07 +04:00

README.rst

Add simple wrapper for pytest

2016-09-06 15:10:57 +03:00

README.rst

Testing

Please, don't hesitate to write tests ;)

Unit tests

Files: /tests/unit/*

The goal of unit tests is to ensure that internal parts of the code work properly. All internal methods should be fully covered by unit tests with a reasonable mocks usage.

About Rally unit tests:

All unit tests are located inside /tests/unit/*
Tests are written on top of: testtools and mock libs
Tox is used to run unit tests

To run unit tests locally:

$ pip install tox
$ tox

To run py27, py34, py35 or pep8 only:

$ tox -e <name>

# NOTE: <name> is one of py27, py34, py35 or pep8

To run py27/py34/py35 against mysql or psql

$ export RALLY_UNITTEST_DB_URL="mysql://user:secret@localhost/rally" $ tox -epy27

To run specific test of py27/py34/py35:

$ tox -e py27 -- tests.unit.test_osclients

To get test coverage:

$ tox -e cover

# NOTE: Results will be in ./cover/index.html

To generate docs:

$ tox -e docs

# NOTE: Documentation will be in doc/source/_build/html/index.html

Functional tests

Files: /tests/functional/*

The goal of functional tests is to check that everything works well together. Fuctional tests use Rally API only and check responses without touching internal parts.

To run functional tests locally:

$ source openrc
$ rally deployment create --fromenv --name testing
$ tox -e cli

# NOTE: openrc file with OpenStack admin credentials

Output of every Rally execution will be collected under some reports root in directory structure like: reports_root/ClassName/MethodName_suffix.extension This functionality implemented in tests.functional.utils.Rally.__call__ method. Use 'gen_report_path' method of 'Rally' class to get automatically generated file path and name if you need. You can use it to publish html reports, generated during tests. Reports root can be passed through environment variable 'REPORTS_ROOT'. Default is 'rally-cli-output-files'.

Rally CI scripts

Files: /tests/ci/*

This directory contains scripts and files related to the Rally CI system.

Rally Style Commandments

File: /tests/hacking/checks.py

This module contains Rally specific hacking rules for checking commandments.