Release notes v0.1.1
Change-Id: Ifaa083a54fb72cb5c195c80da064ec91032292d7
This commit is contained in:
Boris Pavlovic
parent
ecdd65affc
commit
7bf612e81a
510
doc/release_notes/archive/v0.1.0.rst
Normal file
510
doc/release_notes/archive/v0.1.0.rst
Normal file
@ -0,0 +1,510 @@
|
||||
============
|
||||
Rally v0.1.0
|
||||
============
|
||||
|
||||
Information
|
||||
-----------
|
||||
|
||||
+------------------+-----------------------+
|
||||
| Commits | **355** |
|
||||
+------------------+-----------------------+
|
||||
| Bug fixes | **90** |
|
||||
+------------------+-----------------------+
|
||||
| Dev cycle | **132 days** |
|
||||
+------------------+-----------------------+
|
||||
| Release date | **25/September/2015** |
|
||||
+------------------+-----------------------+
|
||||
|
||||
|
||||
Details
|
||||
-------
|
||||
|
||||
This release contains new features, new 42 plugins, 90 bug fixes,
|
||||
various code and API improvements.
|
||||
|
||||
|
||||
New Features & API changes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
* **Improved installation script**
|
||||
|
||||
* Add parameters:
|
||||
|
||||
* ``--develop`` parameter to install rally in editable (develop) mode
|
||||
|
||||
* ``--no-color`` to switch off output colorizing
|
||||
useful for automated output parsing and terminals that don't
|
||||
support colors.
|
||||
|
||||
* Puts rally.conf under virtualenv etc/rally/ so you can have several
|
||||
rally installations in virtualenv
|
||||
|
||||
* Many fixes related to access of different file, like: rally.conf,
|
||||
rally db file in case of sqlite
|
||||
|
||||
* Update pip before Rally installation
|
||||
|
||||
* Fix reinstallation
|
||||
|
||||
|
||||
* **Separated Rally plugins & framework**
|
||||
|
||||
Now plugins are here:
|
||||
https://github.com/openstack/rally/tree/master/rally/plugins
|
||||
|
||||
Plugins are as well separated common/* for common plugins
|
||||
that can be use no matter what is tested and OpenStack related
|
||||
plugins
|
||||
|
||||
|
||||
* **New Rally Task framework**
|
||||
|
||||
* All plugins has the same Plugin base:
|
||||
rally.common.plugin.pluing.Plugin They have the same mechanisms for:
|
||||
discovering, providing information based on docstrings, and in future
|
||||
they will use the same deprecation/rename mechanism.
|
||||
|
||||
* Some of files are moved:
|
||||
|
||||
* rally/benchmark -> rally/task
|
||||
|
||||
*This was done to unify naming of rally task command and
|
||||
actually code that implements it.*
|
||||
|
||||
* rally/benchmark/sla/base.py -> rally/task/sla.py
|
||||
|
||||
* rally/benchmark/context/base.py -> rally/task/context.py
|
||||
|
||||
* rally/benchmark/scenarios/base.py -> rally/task/scenario.py
|
||||
|
||||
* rally/benchmark/runners/base.py -> rally/task/runner.py
|
||||
|
||||
* rally/benchmark/scenarios/utils.py -> rally/task/utils.py
|
||||
|
||||
This was done to:
|
||||
|
||||
* avoid doing rally.benchamrk.scenarios import base as scenario_base
|
||||
|
||||
* remove one level of nesting
|
||||
|
||||
* simplify framework structure
|
||||
|
||||
|
||||
* Some of classes and methods were renamed
|
||||
|
||||
* Plugin configuration:
|
||||
|
||||
* context.context() -> context.configure()
|
||||
|
||||
* scenario.scenario() -> scenario.configure()
|
||||
|
||||
* Introduced runner.configure()
|
||||
|
||||
* Introduced sla.configure()
|
||||
|
||||
This resolves 3 problems:
|
||||
|
||||
* Unifies configuration of different types of plugins
|
||||
|
||||
* Simplifies plugin interface
|
||||
|
||||
* Looks nice with new modules path:
|
||||
>>> from rally.task import scenario
|
||||
>>> @scenario.configure()
|
||||
|
||||
|
||||
* Atomic Actions were changed:
|
||||
|
||||
* New rally.task.atomic module
|
||||
|
||||
This allow us in future to reuse atomic actions in Context plugins
|
||||
|
||||
* Renames:
|
||||
|
||||
rally.benchmark.scenarios.base.AtomicAction
|
||||
-> rally.task.atomic.ActionTimer
|
||||
|
||||
rally.benchmark.scenarios.base.atomic_action()
|
||||
-> rally.task.atomic.action_timer()
|
||||
|
||||
* **Context plugins decide how to map their data for scenario**
|
||||
|
||||
Now Context.map_for_scenario method can be override to decide
|
||||
how to pass context object to each iteration of scenario.
|
||||
|
||||
* Samples of NEW vs OLD context, sla, scenario and runner plugins:
|
||||
|
||||
* Context
|
||||
.. code-block:: python
|
||||
|
||||
# Old
|
||||
from rally.benchmark.context import base
|
||||
|
||||
@base.context(name="users", order=100)
|
||||
class YourContext(base.Context):
|
||||
|
||||
def setup(self):
|
||||
# ...
|
||||
|
||||
def cleanup(self):
|
||||
# ...
|
||||
|
||||
# New
|
||||
from rally.task import context
|
||||
|
||||
@context.configure(name="users", order=100)
|
||||
class YourContext(context.Context):
|
||||
|
||||
def setup(self):
|
||||
# ...
|
||||
|
||||
def cleanup(self):
|
||||
# ...
|
||||
|
||||
def map_for_scenario(self):
|
||||
# Maps context object to the scenario context object
|
||||
# like context["users"] -> context["user"] and so on.
|
||||
..
|
||||
|
||||
|
||||
* Scenario
|
||||
.. code-block:: python
|
||||
|
||||
# Old Scenario
|
||||
|
||||
from rally.benchmark.scenarios import base
|
||||
from rally.benchmark import validation
|
||||
|
||||
class ScenarioPlugin(base.Scenario):
|
||||
|
||||
@base.scenario()
|
||||
def some(self):
|
||||
self._do_some_action()
|
||||
|
||||
|
||||
@base.atomic_action_timer("some_timer")
|
||||
def _do_some_action(self):
|
||||
# ...
|
||||
|
||||
# New Scenario
|
||||
|
||||
from rally.task import atomic
|
||||
from rally.task import scenario
|
||||
from rally.task import validation
|
||||
|
||||
# OpenStack scenario has different base now:
|
||||
# rally.plugins.openstack.scenario.OpenStackScenario
|
||||
class ScenarioPlugin(scenario.Scenario):
|
||||
|
||||
@scenario.configure()
|
||||
def some(self):
|
||||
self._do_some_action()
|
||||
|
||||
@atomic.action_timer("some_action")
|
||||
def _do_some_action(self):
|
||||
# ...
|
||||
..
|
||||
|
||||
* Runner
|
||||
.. code-block:: python
|
||||
|
||||
## Old
|
||||
|
||||
from rally.benchmark.runners import base
|
||||
|
||||
class SomeRunner(base.ScenarioRunner):
|
||||
|
||||
__execution_type__ = "some_runner"
|
||||
|
||||
def _run_scenario(self, cls, method_name, context, args)
|
||||
# Load generation
|
||||
|
||||
def abort(self):
|
||||
# Method that aborts load generation
|
||||
|
||||
## New
|
||||
|
||||
from rally.task import runner
|
||||
|
||||
@runner.configure(name="some_runner")
|
||||
class SomeRunner(runner.ScenarioRunner):
|
||||
|
||||
def _run_scenario(self, cls, method_name, context, args)
|
||||
# Load generation
|
||||
|
||||
def abort(self):
|
||||
# Method that aborts load generation
|
||||
|
||||
..
|
||||
|
||||
* SLA
|
||||
.. code-block:: python
|
||||
|
||||
# Old
|
||||
|
||||
from rally.benchmark import sla
|
||||
|
||||
class FailureRate(sla.SLA):
|
||||
# ...
|
||||
|
||||
# New
|
||||
|
||||
from rally.task import sla
|
||||
|
||||
@sla.configure(name="failure_rate")
|
||||
class FailureRate(sla.SLA):
|
||||
# ...
|
||||
..
|
||||
|
||||
|
||||
* **Rally Task aborted command**
|
||||
|
||||
Finally you can gracefully shutdown running task by calling:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
rally task abort <task_uuid>
|
||||
..
|
||||
|
||||
* **Rally CLI changes**
|
||||
|
||||
* [add] ``rally --plugin-paths`` specify the list of directories with plugins
|
||||
|
||||
* [add] ``rally task report --junit`` - generate a JUnit report
|
||||
This allows users to feed reports to tools such as Jenkins.
|
||||
|
||||
* [add] ``rally task abort`` - aborts running Rally task
|
||||
when run with the ``--soft`` key, the ``rally task abort`` command is
|
||||
waiting until the currently running subtask is finished, otherwise the
|
||||
command interrupts subtask immediately after current scenario iterations
|
||||
are finished.
|
||||
|
||||
* [add] ``rally plugin show`` prints detailed information about plugin
|
||||
|
||||
* [add] ``rally plugin list`` prints table with rally plugin names and titles
|
||||
|
||||
* [add] ``rally verify genconfig`` generates tempest.conf without running it.
|
||||
|
||||
* [add] ``rally verify install`` install tempest for specified deployment
|
||||
|
||||
* [add] ``rally verify reinstall`` removes tempest for specified deployment
|
||||
|
||||
* [add] ``rally verify uninstall`` uninstall tempest of specified deployment
|
||||
|
||||
* [fix] ``rally verify start --no-use`` --no-use was always turned on
|
||||
|
||||
* [remove] ``rally use`` now each command has subcommand ``use``
|
||||
|
||||
* [remove] ``rally info``
|
||||
|
||||
* [remove] ``rally-manage tempest`` now it is covered by ``rally verify``
|
||||
|
||||
|
||||
* **New Rally task reports**
|
||||
|
||||
* New code is based on OOP style which is base step to make plugable Reports
|
||||
|
||||
* Reports are now generated for only one iteration over the resulting data
|
||||
which resolves scalability issues when we are working with large
|
||||
amount of iterations.
|
||||
|
||||
* New Load profiler plot that shows amount of iterations that are working
|
||||
in parallel
|
||||
|
||||
* Failed iterations are shown as a red areas on stacked are graphic.
|
||||
|
||||
Non backward compatible changes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* [remove] ``rally use`` cli command
|
||||
|
||||
* [remove] ``rally info`` cli command
|
||||
|
||||
* [remove] ``--uuid`` parameter from ``rally deployment <any>``
|
||||
|
||||
* [remove ``--deploy-id`` parameter from:
|
||||
``rally task <any>``, ``rally verify <any>``, ``rally show <any>``
|
||||
|
||||
Specs & Feature requests
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
[feature request] Explicitly specify existing users for scenarios
|
||||
|
||||
[feature request] Improve install script and add --unistall and --version
|
||||
|
||||
[feature request] Allows specific repos & packages in install-rally.sh
|
||||
|
||||
[feature request] Add abbility to caputre logs from tested services
|
||||
|
||||
[feature request] Check RPC queue perfdata
|
||||
|
||||
[spec] Refactoring Rally cleanup
|
||||
|
||||
[spec] Consistent resource names
|
||||
|
||||
Plugins
|
||||
~~~~~~~
|
||||
|
||||
* **Scenarios**:
|
||||
|
||||
[new] CinderVolumes.create_volume_backup
|
||||
|
||||
[new] CinderVolumes.create_and_restore_volume_backup
|
||||
|
||||
[new] KeystoneBasic.add_and_remove_user_role
|
||||
|
||||
[new] KeystoneBasic.create_and_delete_role
|
||||
|
||||
[new] KeystoneBasic.create_add_and_list_user_roles
|
||||
|
||||
[new] FuelEnvironments.list_environments
|
||||
|
||||
[new] CinderVolumes.modify_volume_metadata
|
||||
|
||||
[new] NovaServers.boot_and_delete_multiple_servers
|
||||
|
||||
[new] NeutronLoadbalancerV1.create_and_list_pool
|
||||
|
||||
[new] ManilaShares.list_shares
|
||||
|
||||
[new] CeilometerEvents.create_user_and_get_event
|
||||
|
||||
[new] CeilometerEvents.create_user_and_list_event_types
|
||||
|
||||
[new] CeilometerEvents.create_user_and_list_events
|
||||
|
||||
[new] CeilometerTraits.create_user_and_list_trait_descriptions
|
||||
|
||||
[new] CeilometerTraits.create_user_and_list_traits
|
||||
|
||||
[new] NeutronLoadbalancerV1.create_and_delete_pools
|
||||
|
||||
[new] NeutronLoadbalancerV1.create_and_update_pools
|
||||
|
||||
[new] ManilaShares.create_and_delete_share
|
||||
|
||||
[new] ManilaShares.create_share_network_and_delete
|
||||
|
||||
[new] ManilaShares.create_share_network_and_list
|
||||
|
||||
[new] HeatStacks.create_and_delete_stack
|
||||
|
||||
[new] ManilaShares.list_share_servers
|
||||
|
||||
[new] HeatStacks.create_snapshot_restore_delete_stack
|
||||
|
||||
[new] KeystoneBasic.create_and_delete_ec2credential
|
||||
|
||||
[new] KeystoneBasic.create_and_list_ec2credentials
|
||||
|
||||
[new] HeatStacks.create_stack_and_scale
|
||||
|
||||
[new] ManilaShares.create_security_service_and_delete
|
||||
|
||||
[new] KeystoneBasic.create_user_set_enabled_and_delete
|
||||
|
||||
[new] ManilaShares.attach_security_service_to_share_network
|
||||
|
||||
[new] IronicNodes.create_and_delete_node
|
||||
|
||||
[new] IronicNodes.create_and_list_node
|
||||
|
||||
[new] CinderVolumes.create_and_list_volume_backups
|
||||
|
||||
[new] NovaNetworks.create_and_list_networks
|
||||
|
||||
[new] NovaNetworks.create_and_delete_network
|
||||
|
||||
[new] EC2Servers.list_servers
|
||||
|
||||
[new] VMTasks.boot_runcommand_delete_custom_imagea
|
||||
|
||||
[new] CinderVolumes.create_and_update_volume
|
||||
|
||||
|
||||
* **Contexts**:
|
||||
|
||||
[new] ManilaQuotas
|
||||
|
||||
Add context for setting up Manila quotas:
|
||||
shares, gigabytes, snapshots, snapshot_gigabytes, share_networks
|
||||
|
||||
[new] ManilaShareNetworks
|
||||
|
||||
Context for share networks that will be used in case of usage
|
||||
deployment with existing users. Provided share networks via context
|
||||
option "share_networks" will be balanced between all share creations
|
||||
of scenarios.
|
||||
|
||||
[new] Lbaas
|
||||
|
||||
Context to create LBaaS-v1 resources
|
||||
|
||||
[new] ImageCommandCustomizerContext
|
||||
|
||||
Allows image customization using side effects of a command execution.
|
||||
E.g. one can install an application to the image and use these image
|
||||
for 'boot_runcommand_delete' scenario afterwards.
|
||||
|
||||
[new] EC2ServerGenerator
|
||||
|
||||
Context that creates servers using EC2 api
|
||||
|
||||
[new] ExistingNetwork
|
||||
|
||||
This context lets you use existing networks that have already been
|
||||
created instead of creating new networks with Rally. This is useful
|
||||
when, for instance, you are using Neutron with a dumb router that is
|
||||
not capable of creating new networks on the fly.
|
||||
|
||||
|
||||
* **SLA**:
|
||||
|
||||
[remove] max_failure_rate - use failure_rate instead
|
||||
|
||||
|
||||
Bug fixes
|
||||
~~~~~~~~~
|
||||
|
||||
**90 bugs were fixed, the most critical are**:
|
||||
|
||||
* Many fixes related that fixes access of rally.conf and DB files
|
||||
|
||||
* Incorrect apt-get "-yes" parameter in install_rally.sh script
|
||||
|
||||
* Rally bash completion doesn't exist in a virtualenv
|
||||
|
||||
* Rally show networks CLI command worked only with nova networks
|
||||
|
||||
* RPS runner was not properly generating load
|
||||
|
||||
* Check is dhcp_agent_scheduler support or not in network cleanup
|
||||
|
||||
* NetworkContext doesn't work with Nova V2.1
|
||||
|
||||
* Rally task input file was not able to use jinja2 include directive
|
||||
|
||||
* Rally in docker image was not able to
|
||||
|
||||
* Rally docker image didn't contain samples
|
||||
|
||||
* Do not update the average duration when iteration failed
|
||||
|
||||
|
||||
Documentation
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
* **Add plugin reference page**
|
||||
|
||||
:ref:`Rally Plugins Reference page <plugin_reference>` page contains a
|
||||
full list with
|
||||
|
||||
* **Add maintainers section on project info page**
|
||||
|
||||
:ref:`Rally Maintainers section <project_info>` contains information
|
||||
about core contributors of OpenStack Rally their responsibilities and
|
||||
contacts. This will help us to make our community more transparent and open
|
||||
for newbies.
|
||||
|
||||
* **Added who is using section in docs**
|
||||
|
||||
* **Many small fixes**
|
||||
@ -1,510 +1,131 @@
|
||||
============
|
||||
Rally v0.1.0
|
||||
Rally v0.1.1
|
||||
============
|
||||
|
||||
Information
|
||||
-----------
|
||||
|
||||
+------------------+-----------------------+
|
||||
| Commits | **355** |
|
||||
| Commits | **32** |
|
||||
+------------------+-----------------------+
|
||||
| Bug fixes | **90** |
|
||||
| Bug fixes | **9** |
|
||||
+------------------+-----------------------+
|
||||
| Dev cycle | **132 days** |
|
||||
| Dev cycle | **11 days** |
|
||||
+------------------+-----------------------+
|
||||
| Release date | **25/September/2015** |
|
||||
| Release date | **6/October/2015** |
|
||||
+------------------+-----------------------+
|
||||
|
||||
|
||||
Details
|
||||
-------
|
||||
|
||||
This release contains new features, new 42 plugins, 90 bug fixes,
|
||||
This release contains new features, new 6 plugins, 9 bug fixes,
|
||||
various code and API improvements.
|
||||
|
||||
|
||||
New Features & API changes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
* **Improved installation script**
|
||||
New Features
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* Add parameters:
|
||||
* **Rally verify generates proper tempest.conf file now**
|
||||
|
||||
* ``--develop`` parameter to install rally in editable (develop) mode
|
||||
Improved script that generates tempest.conf, now it works out of box for
|
||||
most of the clouds and most of Tempest tests will pass without hacking it.
|
||||
|
||||
* ``--no-color`` to switch off output colorizing
|
||||
useful for automated output parsing and terminals that don't
|
||||
support colors.
|
||||
* **Import Tempest results to Rally DB**
|
||||
|
||||
* Puts rally.conf under virtualenv etc/rally/ so you can have several
|
||||
rally installations in virtualenv
|
||||
``rally verify import`` command allows you to import already existing Tempest
|
||||
results and work with them as regular "rally verify start" results:
|
||||
generate HTML/CSV reports & compare different runs.
|
||||
|
||||
* Many fixes related to access of different file, like: rally.conf,
|
||||
rally db file in case of sqlite
|
||||
|
||||
* Update pip before Rally installation
|
||||
API Changes
|
||||
~~~~~~~~~~~~
|
||||
|
||||
* Fix reinstallation
|
||||
**Rally CLI changes**
|
||||
|
||||
* [add] ``rally verify import`` imports raw Tempest results to Rally
|
||||
|
||||
* **Separated Rally plugins & framework**
|
||||
|
||||
Now plugins are here:
|
||||
https://github.com/openstack/rally/tree/master/rally/plugins
|
||||
|
||||
Plugins are as well separated common/* for common plugins
|
||||
that can be use no matter what is tested and OpenStack related
|
||||
plugins
|
||||
|
||||
|
||||
* **New Rally Task framework**
|
||||
|
||||
* All plugins has the same Plugin base:
|
||||
rally.common.plugin.pluing.Plugin They have the same mechanisms for:
|
||||
discovering, providing information based on docstrings, and in future
|
||||
they will use the same deprecation/rename mechanism.
|
||||
|
||||
* Some of files are moved:
|
||||
|
||||
* rally/benchmark -> rally/task
|
||||
|
||||
*This was done to unify naming of rally task command and
|
||||
actually code that implements it.*
|
||||
|
||||
* rally/benchmark/sla/base.py -> rally/task/sla.py
|
||||
|
||||
* rally/benchmark/context/base.py -> rally/task/context.py
|
||||
|
||||
* rally/benchmark/scenarios/base.py -> rally/task/scenario.py
|
||||
|
||||
* rally/benchmark/runners/base.py -> rally/task/runner.py
|
||||
|
||||
* rally/benchmark/scenarios/utils.py -> rally/task/utils.py
|
||||
|
||||
This was done to:
|
||||
|
||||
* avoid doing rally.benchamrk.scenarios import base as scenario_base
|
||||
|
||||
* remove one level of nesting
|
||||
|
||||
* simplify framework structure
|
||||
|
||||
|
||||
* Some of classes and methods were renamed
|
||||
|
||||
* Plugin configuration:
|
||||
|
||||
* context.context() -> context.configure()
|
||||
|
||||
* scenario.scenario() -> scenario.configure()
|
||||
|
||||
* Introduced runner.configure()
|
||||
|
||||
* Introduced sla.configure()
|
||||
|
||||
This resolves 3 problems:
|
||||
|
||||
* Unifies configuration of different types of plugins
|
||||
|
||||
* Simplifies plugin interface
|
||||
|
||||
* Looks nice with new modules path:
|
||||
>>> from rally.task import scenario
|
||||
>>> @scenario.configure()
|
||||
|
||||
|
||||
* Atomic Actions were changed:
|
||||
|
||||
* New rally.task.atomic module
|
||||
|
||||
This allow us in future to reuse atomic actions in Context plugins
|
||||
|
||||
* Renames:
|
||||
|
||||
rally.benchmark.scenarios.base.AtomicAction
|
||||
-> rally.task.atomic.ActionTimer
|
||||
|
||||
rally.benchmark.scenarios.base.atomic_action()
|
||||
-> rally.task.atomic.action_timer()
|
||||
|
||||
* **Context plugins decide how to map their data for scenario**
|
||||
|
||||
Now Context.map_for_scenario method can be override to decide
|
||||
how to pass context object to each iteration of scenario.
|
||||
|
||||
* Samples of NEW vs OLD context, sla, scenario and runner plugins:
|
||||
|
||||
* Context
|
||||
.. code-block:: python
|
||||
|
||||
# Old
|
||||
from rally.benchmark.context import base
|
||||
|
||||
@base.context(name="users", order=100)
|
||||
class YourContext(base.Context):
|
||||
|
||||
def setup(self):
|
||||
# ...
|
||||
|
||||
def cleanup(self):
|
||||
# ...
|
||||
|
||||
# New
|
||||
from rally.task import context
|
||||
|
||||
@context.configure(name="users", order=100)
|
||||
class YourContext(context.Context):
|
||||
|
||||
def setup(self):
|
||||
# ...
|
||||
|
||||
def cleanup(self):
|
||||
# ...
|
||||
|
||||
def map_for_scenario(self):
|
||||
# Maps context object to the scenario context object
|
||||
# like context["users"] -> context["user"] and so on.
|
||||
..
|
||||
|
||||
|
||||
* Scenario
|
||||
.. code-block:: python
|
||||
|
||||
# Old Scenario
|
||||
|
||||
from rally.benchmark.scenarios import base
|
||||
from rally.benchmark import validation
|
||||
|
||||
class ScenarioPlugin(base.Scenario):
|
||||
|
||||
@base.scenario()
|
||||
def some(self):
|
||||
self._do_some_action()
|
||||
|
||||
|
||||
@base.atomic_action_timer("some_timer")
|
||||
def _do_some_action(self):
|
||||
# ...
|
||||
|
||||
# New Scenario
|
||||
|
||||
from rally.task import atomic
|
||||
from rally.task import scenario
|
||||
from rally.task import validation
|
||||
|
||||
# OpenStack scenario has different base now:
|
||||
# rally.plugins.openstack.scenario.OpenStackScenario
|
||||
class ScenarioPlugin(scenario.Scenario):
|
||||
|
||||
@scenario.configure()
|
||||
def some(self):
|
||||
self._do_some_action()
|
||||
|
||||
@atomic.action_timer("some_action")
|
||||
def _do_some_action(self):
|
||||
# ...
|
||||
..
|
||||
|
||||
* Runner
|
||||
.. code-block:: python
|
||||
|
||||
## Old
|
||||
|
||||
from rally.benchmark.runners import base
|
||||
|
||||
class SomeRunner(base.ScenarioRunner):
|
||||
|
||||
__execution_type__ = "some_runner"
|
||||
|
||||
def _run_scenario(self, cls, method_name, context, args)
|
||||
# Load generation
|
||||
|
||||
def abort(self):
|
||||
# Method that aborts load generation
|
||||
|
||||
## New
|
||||
|
||||
from rally.task import runner
|
||||
|
||||
@runner.configure(name="some_runner")
|
||||
class SomeRunner(runner.ScenarioRunner):
|
||||
|
||||
def _run_scenario(self, cls, method_name, context, args)
|
||||
# Load generation
|
||||
|
||||
def abort(self):
|
||||
# Method that aborts load generation
|
||||
|
||||
..
|
||||
|
||||
* SLA
|
||||
.. code-block:: python
|
||||
|
||||
# Old
|
||||
|
||||
from rally.benchmark import sla
|
||||
|
||||
class FailureRate(sla.SLA):
|
||||
# ...
|
||||
|
||||
# New
|
||||
|
||||
from rally.task import sla
|
||||
|
||||
@sla.configure(name="failure_rate")
|
||||
class FailureRate(sla.SLA):
|
||||
# ...
|
||||
..
|
||||
|
||||
|
||||
* **Rally Task aborted command**
|
||||
|
||||
Finally you can gracefully shutdown running task by calling:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
rally task abort <task_uuid>
|
||||
..
|
||||
|
||||
* **Rally CLI changes**
|
||||
|
||||
* [add] ``rally --plugin-paths`` specify the list of directories with plugins
|
||||
|
||||
* [add] ``rally task report --junit`` - generate a JUnit report
|
||||
This allows users to feed reports to tools such as Jenkins.
|
||||
|
||||
* [add] ``rally task abort`` - aborts running Rally task
|
||||
when run with the ``--soft`` key, the ``rally task abort`` command is
|
||||
waiting until the currently running subtask is finished, otherwise the
|
||||
command interrupts subtask immediately after current scenario iterations
|
||||
are finished.
|
||||
|
||||
* [add] ``rally plugin show`` prints detailed information about plugin
|
||||
|
||||
* [add] ``rally plugin list`` prints table with rally plugin names and titles
|
||||
|
||||
* [add] ``rally verify genconfig`` generates tempest.conf without running it.
|
||||
|
||||
* [add] ``rally verify install`` install tempest for specified deployment
|
||||
|
||||
* [add] ``rally verify reinstall`` removes tempest for specified deployment
|
||||
|
||||
* [add] ``rally verify uninstall`` uninstall tempest of specified deployment
|
||||
|
||||
* [fix] ``rally verify start --no-use`` --no-use was always turned on
|
||||
|
||||
* [remove] ``rally use`` now each command has subcommand ``use``
|
||||
|
||||
* [remove] ``rally info``
|
||||
|
||||
* [remove] ``rally-manage tempest`` now it is covered by ``rally verify``
|
||||
|
||||
|
||||
* **New Rally task reports**
|
||||
|
||||
* New code is based on OOP style which is base step to make plugable Reports
|
||||
|
||||
* Reports are now generated for only one iteration over the resulting data
|
||||
which resolves scalability issues when we are working with large
|
||||
amount of iterations.
|
||||
|
||||
* New Load profiler plot that shows amount of iterations that are working
|
||||
in parallel
|
||||
|
||||
* Failed iterations are shown as a red areas on stacked are graphic.
|
||||
|
||||
Non backward compatible changes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* [remove] ``rally use`` cli command
|
||||
|
||||
* [remove] ``rally info`` cli command
|
||||
|
||||
* [remove] ``--uuid`` parameter from ``rally deployment <any>``
|
||||
|
||||
* [remove ``--deploy-id`` parameter from:
|
||||
``rally task <any>``, ``rally verify <any>``, ``rally show <any>``
|
||||
|
||||
Specs & Feature requests
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
[feature request] Explicitly specify existing users for scenarios
|
||||
|
||||
[feature request] Improve install script and add --unistall and --version
|
||||
|
||||
[feature request] Allows specific repos & packages in install-rally.sh
|
||||
|
||||
[feature request] Add abbility to caputre logs from tested services
|
||||
|
||||
[feature request] Check RPC queue perfdata
|
||||
|
||||
[spec] Refactoring Rally cleanup
|
||||
|
||||
[spec] Consistent resource names
|
||||
There is no new specs and feature requests.
|
||||
|
||||
Plugins
|
||||
~~~~~~~
|
||||
|
||||
* **Scenarios**:
|
||||
|
||||
[new] CinderVolumes.create_volume_backup
|
||||
[new] NeutronNetworks.create_and_list_floating_ips
|
||||
|
||||
[new] CinderVolumes.create_and_restore_volume_backup
|
||||
[new] NeutronNetworks.create_and_delete_floating_ips
|
||||
|
||||
[new] KeystoneBasic.add_and_remove_user_role
|
||||
[new] MuranoPackages.import_and_list_packages
|
||||
|
||||
[new] KeystoneBasic.create_and_delete_role
|
||||
[new] MuranoPackages.import_and_delete_package
|
||||
|
||||
[new] KeystoneBasic.create_add_and_list_user_roles
|
||||
[new] MuranoPackages.import_and_filter_applications
|
||||
|
||||
[new] FuelEnvironments.list_environments
|
||||
[new] MuranoPackages.package_lifecycle
|
||||
|
||||
[new] CinderVolumes.modify_volume_metadata
|
||||
[improved] NovaKeypair.boot_and_delete_server_with_keypair
|
||||
|
||||
[new] NovaServers.boot_and_delete_multiple_servers
|
||||
New argument ``server_kwargs``, these kwargs are used to boot server.
|
||||
|
||||
[new] NeutronLoadbalancerV1.create_and_list_pool
|
||||
[fix] NeutronLoadbalancerV1.create_and_delete_vips
|
||||
|
||||
[new] ManilaShares.list_shares
|
||||
|
||||
[new] CeilometerEvents.create_user_and_get_event
|
||||
|
||||
[new] CeilometerEvents.create_user_and_list_event_types
|
||||
|
||||
[new] CeilometerEvents.create_user_and_list_events
|
||||
|
||||
[new] CeilometerTraits.create_user_and_list_trait_descriptions
|
||||
|
||||
[new] CeilometerTraits.create_user_and_list_traits
|
||||
|
||||
[new] NeutronLoadbalancerV1.create_and_delete_pools
|
||||
|
||||
[new] NeutronLoadbalancerV1.create_and_update_pools
|
||||
|
||||
[new] ManilaShares.create_and_delete_share
|
||||
|
||||
[new] ManilaShares.create_share_network_and_delete
|
||||
|
||||
[new] ManilaShares.create_share_network_and_list
|
||||
|
||||
[new] HeatStacks.create_and_delete_stack
|
||||
|
||||
[new] ManilaShares.list_share_servers
|
||||
|
||||
[new] HeatStacks.create_snapshot_restore_delete_stack
|
||||
|
||||
[new] KeystoneBasic.create_and_delete_ec2credential
|
||||
|
||||
[new] KeystoneBasic.create_and_list_ec2credentials
|
||||
|
||||
[new] HeatStacks.create_stack_and_scale
|
||||
|
||||
[new] ManilaShares.create_security_service_and_delete
|
||||
|
||||
[new] KeystoneBasic.create_user_set_enabled_and_delete
|
||||
|
||||
[new] ManilaShares.attach_security_service_to_share_network
|
||||
|
||||
[new] IronicNodes.create_and_delete_node
|
||||
|
||||
[new] IronicNodes.create_and_list_node
|
||||
|
||||
[new] CinderVolumes.create_and_list_volume_backups
|
||||
|
||||
[new] NovaNetworks.create_and_list_networks
|
||||
|
||||
[new] NovaNetworks.create_and_delete_network
|
||||
|
||||
[new] EC2Servers.list_servers
|
||||
|
||||
[new] VMTasks.boot_runcommand_delete_custom_imagea
|
||||
|
||||
[new] CinderVolumes.create_and_update_volume
|
||||
Now it works in case of concurrency > 1
|
||||
|
||||
|
||||
* **Contexts**:
|
||||
|
||||
[new] ManilaQuotas
|
||||
[improved] network
|
||||
|
||||
Add context for setting up Manila quotas:
|
||||
shares, gigabytes, snapshots, snapshot_gigabytes, share_networks
|
||||
Network context accepts two new arguments:
|
||||
``subnets_per_network`` and ``network_create_args``.
|
||||
|
||||
[new] ManilaShareNetworks
|
||||
[fix] network
|
||||
|
||||
Context for share networks that will be used in case of usage
|
||||
deployment with existing users. Provided share networks via context
|
||||
option "share_networks" will be balanced between all share creations
|
||||
of scenarios.
|
||||
Fix cleanup if nova-network is used. Networks should be dissociate from
|
||||
project before deletion
|
||||
|
||||
[new] Lbaas
|
||||
[fix] custom_image
|
||||
|
||||
Context to create LBaaS-v1 resources
|
||||
|
||||
[new] ImageCommandCustomizerContext
|
||||
|
||||
Allows image customization using side effects of a command execution.
|
||||
E.g. one can install an application to the image and use these image
|
||||
for 'boot_runcommand_delete' scenario afterwards.
|
||||
|
||||
[new] EC2ServerGenerator
|
||||
|
||||
Context that creates servers using EC2 api
|
||||
|
||||
[new] ExistingNetwork
|
||||
|
||||
This context lets you use existing networks that have already been
|
||||
created instead of creating new networks with Rally. This is useful
|
||||
when, for instance, you are using Neutron with a dumb router that is
|
||||
not capable of creating new networks on the fly.
|
||||
|
||||
|
||||
* **SLA**:
|
||||
|
||||
[remove] max_failure_rate - use failure_rate instead
|
||||
Nova server that is used to create custom image was not deleted if
|
||||
script that prepares server failed.
|
||||
|
||||
|
||||
Bug fixes
|
||||
~~~~~~~~~
|
||||
|
||||
**90 bugs were fixed, the most critical are**:
|
||||
**9 bugs were fixed, the most critical are**:
|
||||
|
||||
* Many fixes related that fixes access of rally.conf and DB files
|
||||
* Fix install_rally.sh script
|
||||
|
||||
* Incorrect apt-get "-yes" parameter in install_rally.sh script
|
||||
Set 777 access to /var/lib/rally/database file if system-wide method of
|
||||
installation is used.
|
||||
|
||||
* Rally bash completion doesn't exist in a virtualenv
|
||||
* Rally HTML reports Overview table had few mistakes
|
||||
|
||||
* Rally show networks CLI command worked only with nova networks
|
||||
* Success rate was always 100%
|
||||
|
||||
* RPS runner was not properly generating load
|
||||
* Percentiles were wrongly calculated
|
||||
|
||||
* Check is dhcp_agent_scheduler support or not in network cleanup
|
||||
* Missing Ironic, Murano and Workload(vm) options in default config file
|
||||
|
||||
* NetworkContext doesn't work with Nova V2.1
|
||||
* ``rally verify start`` failed while getting network_id
|
||||
|
||||
* Rally task input file was not able to use jinja2 include directive
|
||||
|
||||
* Rally in docker image was not able to
|
||||
|
||||
* Rally docker image didn't contain samples
|
||||
|
||||
* Do not update the average duration when iteration failed
|
||||
* ``rally verify genconfig`` hangs forever if Horizon is not available
|
||||
|
||||
|
||||
Documentation
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
* **Add plugin reference page**
|
||||
* **Fix project maintainers page**
|
||||
|
||||
:ref:`Rally Plugins Reference page <plugin_reference>` page contains a
|
||||
full list with
|
||||
Update the information about Rally maintainers
|
||||
|
||||
* **Add maintainers section on project info page**
|
||||
* **Document rally --plugin-paths CLI argument**
|
||||
|
||||
:ref:`Rally Maintainers section <project_info>` contains information
|
||||
about core contributors of OpenStack Rally their responsibilities and
|
||||
contacts. This will help us to make our community more transparent and open
|
||||
for newbies.
|
||||
* **Code blocks in documentation looks prettier now**
|
||||
|
||||
* **Added who is using section in docs**
|
||||
|
||||
* **Many small fixes**
|
||||
|
||||
Loading…
Reference in New Issue
Block a user