Release notes v0.1.1
Change-Id: Ifaa083a54fb72cb5c195c80da064ec91032292d7
This commit is contained in:
Boris Pavlovic
parent
ecdd65affc
commit
7bf612e81a
510
doc/release_notes/archive/v0.1.0.rst
Normal file
510
doc/release_notes/archive/v0.1.0.rst
Normal file
@ -0,0 +1,510 @@
|
|||||||
|
============
|
||||||
|
Rally v0.1.0
|
||||||
|
============
|
||||||
|
|
||||||
|
Information
|
||||||
|
-----------
|
||||||
|
|
||||||
|
+------------------+-----------------------+
|
||||||
|
| Commits | **355** |
|
||||||
|
+------------------+-----------------------+
|
||||||
|
| Bug fixes | **90** |
|
||||||
|
+------------------+-----------------------+
|
||||||
|
| Dev cycle | **132 days** |
|
||||||
|
+------------------+-----------------------+
|
||||||
|
| Release date | **25/September/2015** |
|
||||||
|
+------------------+-----------------------+
|
||||||
|
|
||||||
|
|
||||||
|
Details
|
||||||
|
-------
|
||||||
|
|
||||||
|
This release contains new features, new 42 plugins, 90 bug fixes,
|
||||||
|
various code and API improvements.
|
||||||
|
|
||||||
|
|
||||||
|
New Features & API changes
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
* **Improved installation script**
|
||||||
|
|
||||||
|
* Add parameters:
|
||||||
|
|
||||||
|
* ``--develop`` parameter to install rally in editable (develop) mode
|
||||||
|
|
||||||
|
* ``--no-color`` to switch off output colorizing
|
||||||
|
useful for automated output parsing and terminals that don't
|
||||||
|
support colors.
|
||||||
|
|
||||||
|
* Puts rally.conf under virtualenv etc/rally/ so you can have several
|
||||||
|
rally installations in virtualenv
|
||||||
|
|
||||||
|
* Many fixes related to access of different file, like: rally.conf,
|
||||||
|
rally db file in case of sqlite
|
||||||
|
|
||||||
|
* Update pip before Rally installation
|
||||||
|
|
||||||
|
* Fix reinstallation
|
||||||
|
|
||||||
|
|
||||||
|
* **Separated Rally plugins & framework**
|
||||||
|
|
||||||
|
Now plugins are here:
|
||||||
|
https://github.com/openstack/rally/tree/master/rally/plugins
|
||||||
|
|
||||||
|
Plugins are as well separated common/* for common plugins
|
||||||
|
that can be use no matter what is tested and OpenStack related
|
||||||
|
plugins
|
||||||
|
|
||||||
|
|
||||||
|
* **New Rally Task framework**
|
||||||
|
|
||||||
|
* All plugins has the same Plugin base:
|
||||||
|
rally.common.plugin.pluing.Plugin They have the same mechanisms for:
|
||||||
|
discovering, providing information based on docstrings, and in future
|
||||||
|
they will use the same deprecation/rename mechanism.
|
||||||
|
|
||||||
|
* Some of files are moved:
|
||||||
|
|
||||||
|
* rally/benchmark -> rally/task
|
||||||
|
|
||||||
|
*This was done to unify naming of rally task command and
|
||||||
|
actually code that implements it.*
|
||||||
|
|
||||||
|
* rally/benchmark/sla/base.py -> rally/task/sla.py
|
||||||
|
|
||||||
|
* rally/benchmark/context/base.py -> rally/task/context.py
|
||||||
|
|
||||||
|
* rally/benchmark/scenarios/base.py -> rally/task/scenario.py
|
||||||
|
|
||||||
|
* rally/benchmark/runners/base.py -> rally/task/runner.py
|
||||||
|
|
||||||
|
* rally/benchmark/scenarios/utils.py -> rally/task/utils.py
|
||||||
|
|
||||||
|
This was done to:
|
||||||
|
|
||||||
|
* avoid doing rally.benchamrk.scenarios import base as scenario_base
|
||||||
|
|
||||||
|
* remove one level of nesting
|
||||||
|
|
||||||
|
* simplify framework structure
|
||||||
|
|
||||||
|
|
||||||
|
* Some of classes and methods were renamed
|
||||||
|
|
||||||
|
* Plugin configuration:
|
||||||
|
|
||||||
|
* context.context() -> context.configure()
|
||||||
|
|
||||||
|
* scenario.scenario() -> scenario.configure()
|
||||||
|
|
||||||
|
* Introduced runner.configure()
|
||||||
|
|
||||||
|
* Introduced sla.configure()
|
||||||
|
|
||||||
|
This resolves 3 problems:
|
||||||
|
|
||||||
|
* Unifies configuration of different types of plugins
|
||||||
|
|
||||||
|
* Simplifies plugin interface
|
||||||
|
|
||||||
|
* Looks nice with new modules path:
|
||||||
|
>>> from rally.task import scenario
|
||||||
|
>>> @scenario.configure()
|
||||||
|
|
||||||
|
|
||||||
|
* Atomic Actions were changed:
|
||||||
|
|
||||||
|
* New rally.task.atomic module
|
||||||
|
|
||||||
|
This allow us in future to reuse atomic actions in Context plugins
|
||||||
|
|
||||||
|
* Renames:
|
||||||
|
|
||||||
|
rally.benchmark.scenarios.base.AtomicAction
|
||||||
|
-> rally.task.atomic.ActionTimer
|
||||||
|
|
||||||
|
rally.benchmark.scenarios.base.atomic_action()
|
||||||
|
-> rally.task.atomic.action_timer()
|
||||||
|
|
||||||
|
* **Context plugins decide how to map their data for scenario**
|
||||||
|
|
||||||
|
Now Context.map_for_scenario method can be override to decide
|
||||||
|
how to pass context object to each iteration of scenario.
|
||||||
|
|
||||||
|
* Samples of NEW vs OLD context, sla, scenario and runner plugins:
|
||||||
|
|
||||||
|
* Context
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# Old
|
||||||
|
from rally.benchmark.context import base
|
||||||
|
|
||||||
|
@base.context(name="users", order=100)
|
||||||
|
class YourContext(base.Context):
|
||||||
|
|
||||||
|
def setup(self):
|
||||||
|
# ...
|
||||||
|
|
||||||
|
def cleanup(self):
|
||||||
|
# ...
|
||||||
|
|
||||||
|
# New
|
||||||
|
from rally.task import context
|
||||||
|
|
||||||
|
@context.configure(name="users", order=100)
|
||||||
|
class YourContext(context.Context):
|
||||||
|
|
||||||
|
def setup(self):
|
||||||
|
# ...
|
||||||
|
|
||||||
|
def cleanup(self):
|
||||||
|
# ...
|
||||||
|
|
||||||
|
def map_for_scenario(self):
|
||||||
|
# Maps context object to the scenario context object
|
||||||
|
# like context["users"] -> context["user"] and so on.
|
||||||
|
..
|
||||||
|
|
||||||
|
|
||||||
|
* Scenario
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# Old Scenario
|
||||||
|
|
||||||
|
from rally.benchmark.scenarios import base
|
||||||
|
from rally.benchmark import validation
|
||||||
|
|
||||||
|
class ScenarioPlugin(base.Scenario):
|
||||||
|
|
||||||
|
@base.scenario()
|
||||||
|
def some(self):
|
||||||
|
self._do_some_action()
|
||||||
|
|
||||||
|
|
||||||
|
@base.atomic_action_timer("some_timer")
|
||||||
|
def _do_some_action(self):
|
||||||
|
# ...
|
||||||
|
|
||||||
|
# New Scenario
|
||||||
|
|
||||||
|
from rally.task import atomic
|
||||||
|
from rally.task import scenario
|
||||||
|
from rally.task import validation
|
||||||
|
|
||||||
|
# OpenStack scenario has different base now:
|
||||||
|
# rally.plugins.openstack.scenario.OpenStackScenario
|
||||||
|
class ScenarioPlugin(scenario.Scenario):
|
||||||
|
|
||||||
|
@scenario.configure()
|
||||||
|
def some(self):
|
||||||
|
self._do_some_action()
|
||||||
|
|
||||||
|
@atomic.action_timer("some_action")
|
||||||
|
def _do_some_action(self):
|
||||||
|
# ...
|
||||||
|
..
|
||||||
|
|
||||||
|
* Runner
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
## Old
|
||||||
|
|
||||||
|
from rally.benchmark.runners import base
|
||||||
|
|
||||||
|
class SomeRunner(base.ScenarioRunner):
|
||||||
|
|
||||||
|
__execution_type__ = "some_runner"
|
||||||
|
|
||||||
|
def _run_scenario(self, cls, method_name, context, args)
|
||||||
|
# Load generation
|
||||||
|
|
||||||
|
def abort(self):
|
||||||
|
# Method that aborts load generation
|
||||||
|
|
||||||
|
## New
|
||||||
|
|
||||||
|
from rally.task import runner
|
||||||
|
|
||||||
|
@runner.configure(name="some_runner")
|
||||||
|
class SomeRunner(runner.ScenarioRunner):
|
||||||
|
|
||||||
|
def _run_scenario(self, cls, method_name, context, args)
|
||||||
|
# Load generation
|
||||||
|
|
||||||
|
def abort(self):
|
||||||
|
# Method that aborts load generation
|
||||||
|
|
||||||
|
..
|
||||||
|
|
||||||
|
* SLA
|
||||||
|
.. code-block:: python
|
||||||
|
|
||||||
|
# Old
|
||||||
|
|
||||||
|
from rally.benchmark import sla
|
||||||
|
|
||||||
|
class FailureRate(sla.SLA):
|
||||||
|
# ...
|
||||||
|
|
||||||
|
# New
|
||||||
|
|
||||||
|
from rally.task import sla
|
||||||
|
|
||||||
|
@sla.configure(name="failure_rate")
|
||||||
|
class FailureRate(sla.SLA):
|
||||||
|
# ...
|
||||||
|
..
|
||||||
|
|
||||||
|
|
||||||
|
* **Rally Task aborted command**
|
||||||
|
|
||||||
|
Finally you can gracefully shutdown running task by calling:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
rally task abort <task_uuid>
|
||||||
|
..
|
||||||
|
|
||||||
|
* **Rally CLI changes**
|
||||||
|
|
||||||
|
* [add] ``rally --plugin-paths`` specify the list of directories with plugins
|
||||||
|
|
||||||
|
* [add] ``rally task report --junit`` - generate a JUnit report
|
||||||
|
This allows users to feed reports to tools such as Jenkins.
|
||||||
|
|
||||||
|
* [add] ``rally task abort`` - aborts running Rally task
|
||||||
|
when run with the ``--soft`` key, the ``rally task abort`` command is
|
||||||
|
waiting until the currently running subtask is finished, otherwise the
|
||||||
|
command interrupts subtask immediately after current scenario iterations
|
||||||
|
are finished.
|
||||||
|
|
||||||
|
* [add] ``rally plugin show`` prints detailed information about plugin
|
||||||
|
|
||||||
|
* [add] ``rally plugin list`` prints table with rally plugin names and titles
|
||||||
|
|
||||||
|
* [add] ``rally verify genconfig`` generates tempest.conf without running it.
|
||||||
|
|
||||||
|
* [add] ``rally verify install`` install tempest for specified deployment
|
||||||
|
|
||||||
|
* [add] ``rally verify reinstall`` removes tempest for specified deployment
|
||||||
|
|
||||||
|
* [add] ``rally verify uninstall`` uninstall tempest of specified deployment
|
||||||
|
|
||||||
|
* [fix] ``rally verify start --no-use`` --no-use was always turned on
|
||||||
|
|
||||||
|
* [remove] ``rally use`` now each command has subcommand ``use``
|
||||||
|
|
||||||
|
* [remove] ``rally info``
|
||||||
|
|
||||||
|
* [remove] ``rally-manage tempest`` now it is covered by ``rally verify``
|
||||||
|
|
||||||
|
|
||||||
|
* **New Rally task reports**
|
||||||
|
|
||||||
|
* New code is based on OOP style which is base step to make plugable Reports
|
||||||
|
|
||||||
|
* Reports are now generated for only one iteration over the resulting data
|
||||||
|
which resolves scalability issues when we are working with large
|
||||||
|
amount of iterations.
|
||||||
|
|
||||||
|
* New Load profiler plot that shows amount of iterations that are working
|
||||||
|
in parallel
|
||||||
|
|
||||||
|
* Failed iterations are shown as a red areas on stacked are graphic.
|
||||||
|
|
||||||
|
Non backward compatible changes
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
* [remove] ``rally use`` cli command
|
||||||
|
|
||||||
|
* [remove] ``rally info`` cli command
|
||||||
|
|
||||||
|
* [remove] ``--uuid`` parameter from ``rally deployment <any>``
|
||||||
|
|
||||||
|
* [remove ``--deploy-id`` parameter from:
|
||||||
|
``rally task <any>``, ``rally verify <any>``, ``rally show <any>``
|
||||||
|
|
||||||
|
Specs & Feature requests
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
[feature request] Explicitly specify existing users for scenarios
|
||||||
|
|
||||||
|
[feature request] Improve install script and add --unistall and --version
|
||||||
|
|
||||||
|
[feature request] Allows specific repos & packages in install-rally.sh
|
||||||
|
|
||||||
|
[feature request] Add abbility to caputre logs from tested services
|
||||||
|
|
||||||
|
[feature request] Check RPC queue perfdata
|
||||||
|
|
||||||
|
[spec] Refactoring Rally cleanup
|
||||||
|
|
||||||
|
[spec] Consistent resource names
|
||||||
|
|
||||||
|
Plugins
|
||||||
|
~~~~~~~
|
||||||
|
|
||||||
|
* **Scenarios**:
|
||||||
|
|
||||||
|
[new] CinderVolumes.create_volume_backup
|
||||||
|
|
||||||
|
[new] CinderVolumes.create_and_restore_volume_backup
|
||||||
|
|
||||||
|
[new] KeystoneBasic.add_and_remove_user_role
|
||||||
|
|
||||||
|
[new] KeystoneBasic.create_and_delete_role
|
||||||
|
|
||||||
|
[new] KeystoneBasic.create_add_and_list_user_roles
|
||||||
|
|
||||||
|
[new] FuelEnvironments.list_environments
|
||||||
|
|
||||||
|
[new] CinderVolumes.modify_volume_metadata
|
||||||
|
|
||||||
|
[new] NovaServers.boot_and_delete_multiple_servers
|
||||||
|
|
||||||
|
[new] NeutronLoadbalancerV1.create_and_list_pool
|
||||||
|
|
||||||
|
[new] ManilaShares.list_shares
|
||||||
|
|
||||||
|
[new] CeilometerEvents.create_user_and_get_event
|
||||||
|
|
||||||
|
[new] CeilometerEvents.create_user_and_list_event_types
|
||||||
|
|
||||||
|
[new] CeilometerEvents.create_user_and_list_events
|
||||||
|
|
||||||
|
[new] CeilometerTraits.create_user_and_list_trait_descriptions
|
||||||
|
|
||||||
|
[new] CeilometerTraits.create_user_and_list_traits
|
||||||
|
|
||||||
|
[new] NeutronLoadbalancerV1.create_and_delete_pools
|
||||||
|
|
||||||
|
[new] NeutronLoadbalancerV1.create_and_update_pools
|
||||||
|
|
||||||
|
[new] ManilaShares.create_and_delete_share
|
||||||
|
|
||||||
|
[new] ManilaShares.create_share_network_and_delete
|
||||||
|
|
||||||
|
[new] ManilaShares.create_share_network_and_list
|
||||||
|
|
||||||
|
[new] HeatStacks.create_and_delete_stack
|
||||||
|
|
||||||
|
[new] ManilaShares.list_share_servers
|
||||||
|
|
||||||
|
[new] HeatStacks.create_snapshot_restore_delete_stack
|
||||||
|
|
||||||
|
[new] KeystoneBasic.create_and_delete_ec2credential
|
||||||
|
|
||||||
|
[new] KeystoneBasic.create_and_list_ec2credentials
|
||||||
|
|
||||||
|
[new] HeatStacks.create_stack_and_scale
|
||||||
|
|
||||||
|
[new] ManilaShares.create_security_service_and_delete
|
||||||
|
|
||||||
|
[new] KeystoneBasic.create_user_set_enabled_and_delete
|
||||||
|
|
||||||
|
[new] ManilaShares.attach_security_service_to_share_network
|
||||||
|
|
||||||
|
[new] IronicNodes.create_and_delete_node
|
||||||
|
|
||||||
|
[new] IronicNodes.create_and_list_node
|
||||||
|
|
||||||
|
[new] CinderVolumes.create_and_list_volume_backups
|
||||||
|
|
||||||
|
[new] NovaNetworks.create_and_list_networks
|
||||||
|
|
||||||
|
[new] NovaNetworks.create_and_delete_network
|
||||||
|
|
||||||
|
[new] EC2Servers.list_servers
|
||||||
|
|
||||||
|
[new] VMTasks.boot_runcommand_delete_custom_imagea
|
||||||
|
|
||||||
|
[new] CinderVolumes.create_and_update_volume
|
||||||
|
|
||||||
|
|
||||||
|
* **Contexts**:
|
||||||
|
|
||||||
|
[new] ManilaQuotas
|
||||||
|
|
||||||
|
Add context for setting up Manila quotas:
|
||||||
|
shares, gigabytes, snapshots, snapshot_gigabytes, share_networks
|
||||||
|
|
||||||
|
[new] ManilaShareNetworks
|
||||||
|
|
||||||
|
Context for share networks that will be used in case of usage
|
||||||
|
deployment with existing users. Provided share networks via context
|
||||||
|
option "share_networks" will be balanced between all share creations
|
||||||
|
of scenarios.
|
||||||
|
|
||||||
|
[new] Lbaas
|
||||||
|
|
||||||
|
Context to create LBaaS-v1 resources
|
||||||
|
|
||||||
|
[new] ImageCommandCustomizerContext
|
||||||
|
|
||||||
|
Allows image customization using side effects of a command execution.
|
||||||
|
E.g. one can install an application to the image and use these image
|
||||||
|
for 'boot_runcommand_delete' scenario afterwards.
|
||||||
|
|
||||||
|
[new] EC2ServerGenerator
|
||||||
|
|
||||||
|
Context that creates servers using EC2 api
|
||||||
|
|
||||||
|
[new] ExistingNetwork
|
||||||
|
|
||||||
|
This context lets you use existing networks that have already been
|
||||||
|
created instead of creating new networks with Rally. This is useful
|
||||||
|
when, for instance, you are using Neutron with a dumb router that is
|
||||||
|
not capable of creating new networks on the fly.
|
||||||
|
|
||||||
|
|
||||||
|
* **SLA**:
|
||||||
|
|
||||||
|
[remove] max_failure_rate - use failure_rate instead
|
||||||
|
|
||||||
|
|
||||||
|
Bug fixes
|
||||||
|
~~~~~~~~~
|
||||||
|
|
||||||
|
**90 bugs were fixed, the most critical are**:
|
||||||
|
|
||||||
|
* Many fixes related that fixes access of rally.conf and DB files
|
||||||
|
|
||||||
|
* Incorrect apt-get "-yes" parameter in install_rally.sh script
|
||||||
|
|
||||||
|
* Rally bash completion doesn't exist in a virtualenv
|
||||||
|
|
||||||
|
* Rally show networks CLI command worked only with nova networks
|
||||||
|
|
||||||
|
* RPS runner was not properly generating load
|
||||||
|
|
||||||
|
* Check is dhcp_agent_scheduler support or not in network cleanup
|
||||||
|
|
||||||
|
* NetworkContext doesn't work with Nova V2.1
|
||||||
|
|
||||||
|
* Rally task input file was not able to use jinja2 include directive
|
||||||
|
|
||||||
|
* Rally in docker image was not able to
|
||||||
|
|
||||||
|
* Rally docker image didn't contain samples
|
||||||
|
|
||||||
|
* Do not update the average duration when iteration failed
|
||||||
|
|
||||||
|
|
||||||
|
Documentation
|
||||||
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
* **Add plugin reference page**
|
||||||
|
|
||||||
|
:ref:`Rally Plugins Reference page <plugin_reference>` page contains a
|
||||||
|
full list with
|
||||||
|
|
||||||
|
* **Add maintainers section on project info page**
|
||||||
|
|
||||||
|
:ref:`Rally Maintainers section <project_info>` contains information
|
||||||
|
about core contributors of OpenStack Rally their responsibilities and
|
||||||
|
contacts. This will help us to make our community more transparent and open
|
||||||
|
for newbies.
|
||||||
|
|
||||||
|
* **Added who is using section in docs**
|
||||||
|
|
||||||
|
* **Many small fixes**
|
||||||
@ -1,510 +1,131 @@
|
|||||||
============
|
============
|
||||||
Rally v0.1.0
|
Rally v0.1.1
|
||||||
============
|
============
|
||||||
|
|
||||||
Information
|
Information
|
||||||
-----------
|
-----------
|
||||||
|
|
||||||
+------------------+-----------------------+
|
+------------------+-----------------------+
|
||||||
| Commits | **355** |
|
| Commits | **32** |
|
||||||
+------------------+-----------------------+
|
+------------------+-----------------------+
|
||||||
| Bug fixes | **90** |
|
| Bug fixes | **9** |
|
||||||
+------------------+-----------------------+
|
+------------------+-----------------------+
|
||||||
| Dev cycle | **132 days** |
|
| Dev cycle | **11 days** |
|
||||||
+------------------+-----------------------+
|
+------------------+-----------------------+
|
||||||
| Release date | **25/September/2015** |
|
| Release date | **6/October/2015** |
|
||||||
+------------------+-----------------------+
|
+------------------+-----------------------+
|
||||||
|
|
||||||
|
|
||||||
Details
|
Details
|
||||||
-------
|
-------
|
||||||
|
|
||||||
This release contains new features, new 42 plugins, 90 bug fixes,
|
This release contains new features, new 6 plugins, 9 bug fixes,
|
||||||
various code and API improvements.
|
various code and API improvements.
|
||||||
|
|
||||||
|
|
||||||
New Features & API changes
|
New Features
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~
|
||||||
* **Improved installation script**
|
|
||||||
|
|
||||||
* Add parameters:
|
* **Rally verify generates proper tempest.conf file now**
|
||||||
|
|
||||||
* ``--develop`` parameter to install rally in editable (develop) mode
|
Improved script that generates tempest.conf, now it works out of box for
|
||||||
|
most of the clouds and most of Tempest tests will pass without hacking it.
|
||||||
|
|
||||||
* ``--no-color`` to switch off output colorizing
|
* **Import Tempest results to Rally DB**
|
||||||
useful for automated output parsing and terminals that don't
|
|
||||||
support colors.
|
|
||||||
|
|
||||||
* Puts rally.conf under virtualenv etc/rally/ so you can have several
|
``rally verify import`` command allows you to import already existing Tempest
|
||||||
rally installations in virtualenv
|
results and work with them as regular "rally verify start" results:
|
||||||
|
generate HTML/CSV reports & compare different runs.
|
||||||
|
|
||||||
* Many fixes related to access of different file, like: rally.conf,
|
|
||||||
rally db file in case of sqlite
|
|
||||||
|
|
||||||
* Update pip before Rally installation
|
API Changes
|
||||||
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
* Fix reinstallation
|
**Rally CLI changes**
|
||||||
|
|
||||||
|
* [add] ``rally verify import`` imports raw Tempest results to Rally
|
||||||
|
|
||||||
* **Separated Rally plugins & framework**
|
|
||||||
|
|
||||||
Now plugins are here:
|
|
||||||
https://github.com/openstack/rally/tree/master/rally/plugins
|
|
||||||
|
|
||||||
Plugins are as well separated common/* for common plugins
|
|
||||||
that can be use no matter what is tested and OpenStack related
|
|
||||||
plugins
|
|
||||||
|
|
||||||
|
|
||||||
* **New Rally Task framework**
|
|
||||||
|
|
||||||
* All plugins has the same Plugin base:
|
|
||||||
rally.common.plugin.pluing.Plugin They have the same mechanisms for:
|
|
||||||
discovering, providing information based on docstrings, and in future
|
|
||||||
they will use the same deprecation/rename mechanism.
|
|
||||||
|
|
||||||
* Some of files are moved:
|
|
||||||
|
|
||||||
* rally/benchmark -> rally/task
|
|
||||||
|
|
||||||
*This was done to unify naming of rally task command and
|
|
||||||
actually code that implements it.*
|
|
||||||
|
|
||||||
* rally/benchmark/sla/base.py -> rally/task/sla.py
|
|
||||||
|
|
||||||
* rally/benchmark/context/base.py -> rally/task/context.py
|
|
||||||
|
|
||||||
* rally/benchmark/scenarios/base.py -> rally/task/scenario.py
|
|
||||||
|
|
||||||
* rally/benchmark/runners/base.py -> rally/task/runner.py
|
|
||||||
|
|
||||||
* rally/benchmark/scenarios/utils.py -> rally/task/utils.py
|
|
||||||
|
|
||||||
This was done to:
|
|
||||||
|
|
||||||
* avoid doing rally.benchamrk.scenarios import base as scenario_base
|
|
||||||
|
|
||||||
* remove one level of nesting
|
|
||||||
|
|
||||||
* simplify framework structure
|
|
||||||
|
|
||||||
|
|
||||||
* Some of classes and methods were renamed
|
|
||||||
|
|
||||||
* Plugin configuration:
|
|
||||||
|
|
||||||
* context.context() -> context.configure()
|
|
||||||
|
|
||||||
* scenario.scenario() -> scenario.configure()
|
|
||||||
|
|
||||||
* Introduced runner.configure()
|
|
||||||
|
|
||||||
* Introduced sla.configure()
|
|
||||||
|
|
||||||
This resolves 3 problems:
|
|
||||||
|
|
||||||
* Unifies configuration of different types of plugins
|
|
||||||
|
|
||||||
* Simplifies plugin interface
|
|
||||||
|
|
||||||
* Looks nice with new modules path:
|
|
||||||
>>> from rally.task import scenario
|
|
||||||
>>> @scenario.configure()
|
|
||||||
|
|
||||||
|
|
||||||
* Atomic Actions were changed:
|
|
||||||
|
|
||||||
* New rally.task.atomic module
|
|
||||||
|
|
||||||
This allow us in future to reuse atomic actions in Context plugins
|
|
||||||
|
|
||||||
* Renames:
|
|
||||||
|
|
||||||
rally.benchmark.scenarios.base.AtomicAction
|
|
||||||
-> rally.task.atomic.ActionTimer
|
|
||||||
|
|
||||||
rally.benchmark.scenarios.base.atomic_action()
|
|
||||||
-> rally.task.atomic.action_timer()
|
|
||||||
|
|
||||||
* **Context plugins decide how to map their data for scenario**
|
|
||||||
|
|
||||||
Now Context.map_for_scenario method can be override to decide
|
|
||||||
how to pass context object to each iteration of scenario.
|
|
||||||
|
|
||||||
* Samples of NEW vs OLD context, sla, scenario and runner plugins:
|
|
||||||
|
|
||||||
* Context
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
# Old
|
|
||||||
from rally.benchmark.context import base
|
|
||||||
|
|
||||||
@base.context(name="users", order=100)
|
|
||||||
class YourContext(base.Context):
|
|
||||||
|
|
||||||
def setup(self):
|
|
||||||
# ...
|
|
||||||
|
|
||||||
def cleanup(self):
|
|
||||||
# ...
|
|
||||||
|
|
||||||
# New
|
|
||||||
from rally.task import context
|
|
||||||
|
|
||||||
@context.configure(name="users", order=100)
|
|
||||||
class YourContext(context.Context):
|
|
||||||
|
|
||||||
def setup(self):
|
|
||||||
# ...
|
|
||||||
|
|
||||||
def cleanup(self):
|
|
||||||
# ...
|
|
||||||
|
|
||||||
def map_for_scenario(self):
|
|
||||||
# Maps context object to the scenario context object
|
|
||||||
# like context["users"] -> context["user"] and so on.
|
|
||||||
..
|
|
||||||
|
|
||||||
|
|
||||||
* Scenario
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
# Old Scenario
|
|
||||||
|
|
||||||
from rally.benchmark.scenarios import base
|
|
||||||
from rally.benchmark import validation
|
|
||||||
|
|
||||||
class ScenarioPlugin(base.Scenario):
|
|
||||||
|
|
||||||
@base.scenario()
|
|
||||||
def some(self):
|
|
||||||
self._do_some_action()
|
|
||||||
|
|
||||||
|
|
||||||
@base.atomic_action_timer("some_timer")
|
|
||||||
def _do_some_action(self):
|
|
||||||
# ...
|
|
||||||
|
|
||||||
# New Scenario
|
|
||||||
|
|
||||||
from rally.task import atomic
|
|
||||||
from rally.task import scenario
|
|
||||||
from rally.task import validation
|
|
||||||
|
|
||||||
# OpenStack scenario has different base now:
|
|
||||||
# rally.plugins.openstack.scenario.OpenStackScenario
|
|
||||||
class ScenarioPlugin(scenario.Scenario):
|
|
||||||
|
|
||||||
@scenario.configure()
|
|
||||||
def some(self):
|
|
||||||
self._do_some_action()
|
|
||||||
|
|
||||||
@atomic.action_timer("some_action")
|
|
||||||
def _do_some_action(self):
|
|
||||||
# ...
|
|
||||||
..
|
|
||||||
|
|
||||||
* Runner
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
## Old
|
|
||||||
|
|
||||||
from rally.benchmark.runners import base
|
|
||||||
|
|
||||||
class SomeRunner(base.ScenarioRunner):
|
|
||||||
|
|
||||||
__execution_type__ = "some_runner"
|
|
||||||
|
|
||||||
def _run_scenario(self, cls, method_name, context, args)
|
|
||||||
# Load generation
|
|
||||||
|
|
||||||
def abort(self):
|
|
||||||
# Method that aborts load generation
|
|
||||||
|
|
||||||
## New
|
|
||||||
|
|
||||||
from rally.task import runner
|
|
||||||
|
|
||||||
@runner.configure(name="some_runner")
|
|
||||||
class SomeRunner(runner.ScenarioRunner):
|
|
||||||
|
|
||||||
def _run_scenario(self, cls, method_name, context, args)
|
|
||||||
# Load generation
|
|
||||||
|
|
||||||
def abort(self):
|
|
||||||
# Method that aborts load generation
|
|
||||||
|
|
||||||
..
|
|
||||||
|
|
||||||
* SLA
|
|
||||||
.. code-block:: python
|
|
||||||
|
|
||||||
# Old
|
|
||||||
|
|
||||||
from rally.benchmark import sla
|
|
||||||
|
|
||||||
class FailureRate(sla.SLA):
|
|
||||||
# ...
|
|
||||||
|
|
||||||
# New
|
|
||||||
|
|
||||||
from rally.task import sla
|
|
||||||
|
|
||||||
@sla.configure(name="failure_rate")
|
|
||||||
class FailureRate(sla.SLA):
|
|
||||||
# ...
|
|
||||||
..
|
|
||||||
|
|
||||||
|
|
||||||
* **Rally Task aborted command**
|
|
||||||
|
|
||||||
Finally you can gracefully shutdown running task by calling:
|
|
||||||
|
|
||||||
.. code:: bash
|
|
||||||
|
|
||||||
rally task abort <task_uuid>
|
|
||||||
..
|
|
||||||
|
|
||||||
* **Rally CLI changes**
|
|
||||||
|
|
||||||
* [add] ``rally --plugin-paths`` specify the list of directories with plugins
|
|
||||||
|
|
||||||
* [add] ``rally task report --junit`` - generate a JUnit report
|
|
||||||
This allows users to feed reports to tools such as Jenkins.
|
|
||||||
|
|
||||||
* [add] ``rally task abort`` - aborts running Rally task
|
|
||||||
when run with the ``--soft`` key, the ``rally task abort`` command is
|
|
||||||
waiting until the currently running subtask is finished, otherwise the
|
|
||||||
command interrupts subtask immediately after current scenario iterations
|
|
||||||
are finished.
|
|
||||||
|
|
||||||
* [add] ``rally plugin show`` prints detailed information about plugin
|
|
||||||
|
|
||||||
* [add] ``rally plugin list`` prints table with rally plugin names and titles
|
|
||||||
|
|
||||||
* [add] ``rally verify genconfig`` generates tempest.conf without running it.
|
|
||||||
|
|
||||||
* [add] ``rally verify install`` install tempest for specified deployment
|
|
||||||
|
|
||||||
* [add] ``rally verify reinstall`` removes tempest for specified deployment
|
|
||||||
|
|
||||||
* [add] ``rally verify uninstall`` uninstall tempest of specified deployment
|
|
||||||
|
|
||||||
* [fix] ``rally verify start --no-use`` --no-use was always turned on
|
|
||||||
|
|
||||||
* [remove] ``rally use`` now each command has subcommand ``use``
|
|
||||||
|
|
||||||
* [remove] ``rally info``
|
|
||||||
|
|
||||||
* [remove] ``rally-manage tempest`` now it is covered by ``rally verify``
|
|
||||||
|
|
||||||
|
|
||||||
* **New Rally task reports**
|
|
||||||
|
|
||||||
* New code is based on OOP style which is base step to make plugable Reports
|
|
||||||
|
|
||||||
* Reports are now generated for only one iteration over the resulting data
|
|
||||||
which resolves scalability issues when we are working with large
|
|
||||||
amount of iterations.
|
|
||||||
|
|
||||||
* New Load profiler plot that shows amount of iterations that are working
|
|
||||||
in parallel
|
|
||||||
|
|
||||||
* Failed iterations are shown as a red areas on stacked are graphic.
|
|
||||||
|
|
||||||
Non backward compatible changes
|
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
||||||
|
|
||||||
* [remove] ``rally use`` cli command
|
|
||||||
|
|
||||||
* [remove] ``rally info`` cli command
|
|
||||||
|
|
||||||
* [remove] ``--uuid`` parameter from ``rally deployment <any>``
|
|
||||||
|
|
||||||
* [remove ``--deploy-id`` parameter from:
|
|
||||||
``rally task <any>``, ``rally verify <any>``, ``rally show <any>``
|
|
||||||
|
|
||||||
Specs & Feature requests
|
Specs & Feature requests
|
||||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
[feature request] Explicitly specify existing users for scenarios
|
There is no new specs and feature requests.
|
||||||
|
|
||||||
[feature request] Improve install script and add --unistall and --version
|
|
||||||
|
|
||||||
[feature request] Allows specific repos & packages in install-rally.sh
|
|
||||||
|
|
||||||
[feature request] Add abbility to caputre logs from tested services
|
|
||||||
|
|
||||||
[feature request] Check RPC queue perfdata
|
|
||||||
|
|
||||||
[spec] Refactoring Rally cleanup
|
|
||||||
|
|
||||||
[spec] Consistent resource names
|
|
||||||
|
|
||||||
Plugins
|
Plugins
|
||||||
~~~~~~~
|
~~~~~~~
|
||||||
|
|
||||||
* **Scenarios**:
|
* **Scenarios**:
|
||||||
|
|
||||||
[new] CinderVolumes.create_volume_backup
|
[new] NeutronNetworks.create_and_list_floating_ips
|
||||||
|
|
||||||
[new] CinderVolumes.create_and_restore_volume_backup
|
[new] NeutronNetworks.create_and_delete_floating_ips
|
||||||
|
|
||||||
[new] KeystoneBasic.add_and_remove_user_role
|
[new] MuranoPackages.import_and_list_packages
|
||||||
|
|
||||||
[new] KeystoneBasic.create_and_delete_role
|
[new] MuranoPackages.import_and_delete_package
|
||||||
|
|
||||||
[new] KeystoneBasic.create_add_and_list_user_roles
|
[new] MuranoPackages.import_and_filter_applications
|
||||||
|
|
||||||
[new] FuelEnvironments.list_environments
|
[new] MuranoPackages.package_lifecycle
|
||||||
|
|
||||||
[new] CinderVolumes.modify_volume_metadata
|
[improved] NovaKeypair.boot_and_delete_server_with_keypair
|
||||||
|
|
||||||
[new] NovaServers.boot_and_delete_multiple_servers
|
New argument ``server_kwargs``, these kwargs are used to boot server.
|
||||||
|
|
||||||
[new] NeutronLoadbalancerV1.create_and_list_pool
|
[fix] NeutronLoadbalancerV1.create_and_delete_vips
|
||||||
|
|
||||||
[new] ManilaShares.list_shares
|
Now it works in case of concurrency > 1
|
||||||
|
|
||||||
[new] CeilometerEvents.create_user_and_get_event
|
|
||||||
|
|
||||||
[new] CeilometerEvents.create_user_and_list_event_types
|
|
||||||
|
|
||||||
[new] CeilometerEvents.create_user_and_list_events
|
|
||||||
|
|
||||||
[new] CeilometerTraits.create_user_and_list_trait_descriptions
|
|
||||||
|
|
||||||
[new] CeilometerTraits.create_user_and_list_traits
|
|
||||||
|
|
||||||
[new] NeutronLoadbalancerV1.create_and_delete_pools
|
|
||||||
|
|
||||||
[new] NeutronLoadbalancerV1.create_and_update_pools
|
|
||||||
|
|
||||||
[new] ManilaShares.create_and_delete_share
|
|
||||||
|
|
||||||
[new] ManilaShares.create_share_network_and_delete
|
|
||||||
|
|
||||||
[new] ManilaShares.create_share_network_and_list
|
|
||||||
|
|
||||||
[new] HeatStacks.create_and_delete_stack
|
|
||||||
|
|
||||||
[new] ManilaShares.list_share_servers
|
|
||||||
|
|
||||||
[new] HeatStacks.create_snapshot_restore_delete_stack
|
|
||||||
|
|
||||||
[new] KeystoneBasic.create_and_delete_ec2credential
|
|
||||||
|
|
||||||
[new] KeystoneBasic.create_and_list_ec2credentials
|
|
||||||
|
|
||||||
[new] HeatStacks.create_stack_and_scale
|
|
||||||
|
|
||||||
[new] ManilaShares.create_security_service_and_delete
|
|
||||||
|
|
||||||
[new] KeystoneBasic.create_user_set_enabled_and_delete
|
|
||||||
|
|
||||||
[new] ManilaShares.attach_security_service_to_share_network
|
|
||||||
|
|
||||||
[new] IronicNodes.create_and_delete_node
|
|
||||||
|
|
||||||
[new] IronicNodes.create_and_list_node
|
|
||||||
|
|
||||||
[new] CinderVolumes.create_and_list_volume_backups
|
|
||||||
|
|
||||||
[new] NovaNetworks.create_and_list_networks
|
|
||||||
|
|
||||||
[new] NovaNetworks.create_and_delete_network
|
|
||||||
|
|
||||||
[new] EC2Servers.list_servers
|
|
||||||
|
|
||||||
[new] VMTasks.boot_runcommand_delete_custom_imagea
|
|
||||||
|
|
||||||
[new] CinderVolumes.create_and_update_volume
|
|
||||||
|
|
||||||
|
|
||||||
* **Contexts**:
|
* **Contexts**:
|
||||||
|
|
||||||
[new] ManilaQuotas
|
[improved] network
|
||||||
|
|
||||||
Add context for setting up Manila quotas:
|
Network context accepts two new arguments:
|
||||||
shares, gigabytes, snapshots, snapshot_gigabytes, share_networks
|
``subnets_per_network`` and ``network_create_args``.
|
||||||
|
|
||||||
[new] ManilaShareNetworks
|
[fix] network
|
||||||
|
|
||||||
Context for share networks that will be used in case of usage
|
Fix cleanup if nova-network is used. Networks should be dissociate from
|
||||||
deployment with existing users. Provided share networks via context
|
project before deletion
|
||||||
option "share_networks" will be balanced between all share creations
|
|
||||||
of scenarios.
|
|
||||||
|
|
||||||
[new] Lbaas
|
[fix] custom_image
|
||||||
|
|
||||||
Context to create LBaaS-v1 resources
|
Nova server that is used to create custom image was not deleted if
|
||||||
|
script that prepares server failed.
|
||||||
[new] ImageCommandCustomizerContext
|
|
||||||
|
|
||||||
Allows image customization using side effects of a command execution.
|
|
||||||
E.g. one can install an application to the image and use these image
|
|
||||||
for 'boot_runcommand_delete' scenario afterwards.
|
|
||||||
|
|
||||||
[new] EC2ServerGenerator
|
|
||||||
|
|
||||||
Context that creates servers using EC2 api
|
|
||||||
|
|
||||||
[new] ExistingNetwork
|
|
||||||
|
|
||||||
This context lets you use existing networks that have already been
|
|
||||||
created instead of creating new networks with Rally. This is useful
|
|
||||||
when, for instance, you are using Neutron with a dumb router that is
|
|
||||||
not capable of creating new networks on the fly.
|
|
||||||
|
|
||||||
|
|
||||||
* **SLA**:
|
|
||||||
|
|
||||||
[remove] max_failure_rate - use failure_rate instead
|
|
||||||
|
|
||||||
|
|
||||||
Bug fixes
|
Bug fixes
|
||||||
~~~~~~~~~
|
~~~~~~~~~
|
||||||
|
|
||||||
**90 bugs were fixed, the most critical are**:
|
**9 bugs were fixed, the most critical are**:
|
||||||
|
|
||||||
* Many fixes related that fixes access of rally.conf and DB files
|
* Fix install_rally.sh script
|
||||||
|
|
||||||
* Incorrect apt-get "-yes" parameter in install_rally.sh script
|
Set 777 access to /var/lib/rally/database file if system-wide method of
|
||||||
|
installation is used.
|
||||||
|
|
||||||
* Rally bash completion doesn't exist in a virtualenv
|
* Rally HTML reports Overview table had few mistakes
|
||||||
|
|
||||||
* Rally show networks CLI command worked only with nova networks
|
* Success rate was always 100%
|
||||||
|
|
||||||
* RPS runner was not properly generating load
|
* Percentiles were wrongly calculated
|
||||||
|
|
||||||
* Check is dhcp_agent_scheduler support or not in network cleanup
|
* Missing Ironic, Murano and Workload(vm) options in default config file
|
||||||
|
|
||||||
* NetworkContext doesn't work with Nova V2.1
|
* ``rally verify start`` failed while getting network_id
|
||||||
|
|
||||||
* Rally task input file was not able to use jinja2 include directive
|
* ``rally verify genconfig`` hangs forever if Horizon is not available
|
||||||
|
|
||||||
* Rally in docker image was not able to
|
|
||||||
|
|
||||||
* Rally docker image didn't contain samples
|
|
||||||
|
|
||||||
* Do not update the average duration when iteration failed
|
|
||||||
|
|
||||||
|
|
||||||
Documentation
|
Documentation
|
||||||
~~~~~~~~~~~~~
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
* **Add plugin reference page**
|
* **Fix project maintainers page**
|
||||||
|
|
||||||
:ref:`Rally Plugins Reference page <plugin_reference>` page contains a
|
Update the information about Rally maintainers
|
||||||
full list with
|
|
||||||
|
|
||||||
* **Add maintainers section on project info page**
|
* **Document rally --plugin-paths CLI argument**
|
||||||
|
|
||||||
:ref:`Rally Maintainers section <project_info>` contains information
|
* **Code blocks in documentation looks prettier now**
|
||||||
about core contributors of OpenStack Rally their responsibilities and
|
|
||||||
contacts. This will help us to make our community more transparent and open
|
|
||||||
for newbies.
|
|
||||||
|
|
||||||
* **Added who is using section in docs**
|
|
||||||
|
|
||||||
* **Many small fixes**
|
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user