Release 0.0.4
* Add release notes for 0.0.4 * Move 0.0.3 release notes to archive Change-Id: I23e2ed90b806967321b2ade7fa52a268278d1f84
This commit is contained in:
Mikhail Dubov
parent
a9f2cbefc5
commit
a3f88d4672
153
doc/release_notes/archive/v0.0.3.rst
Normal file
153
doc/release_notes/archive/v0.0.3.rst
Normal file
@ -0,0 +1,153 @@
|
||||
============
|
||||
Rally v0.0.3
|
||||
============
|
||||
|
||||
Information
|
||||
-----------
|
||||
|
||||
+------------------+-----------------+
|
||||
| Commits | **53** |
|
||||
+------------------+-----------------+
|
||||
| Bug fixes | **14** |
|
||||
+------------------+-----------------+
|
||||
| Dev cycle | **33 days** |
|
||||
+------------------+-----------------+
|
||||
| Release date | **14/Apr/2015** |
|
||||
+------------------+-----------------+
|
||||
|
||||
|
||||
Details
|
||||
-------
|
||||
|
||||
This release contains new features, new benchmark plugins, bug fixes,
|
||||
various code and API improvements.
|
||||
|
||||
|
||||
New Features & API changes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
|
||||
* Add the ability to specify versions for clients in benchmark scenarios
|
||||
|
||||
You can call self.clients("glance", "2") and get any client for
|
||||
specific version.
|
||||
|
||||
* Add API for tempest uninstall
|
||||
|
||||
$ rally-manage tempest uninstall
|
||||
# removes fully tempest for active deployment
|
||||
|
||||
|
||||
* Add a --uuids-only option to rally task list
|
||||
|
||||
$ rally task list --uuids-only # returns list with only task uuids
|
||||
|
||||
* Adds endpoint to --fromenv deployment creation
|
||||
|
||||
$ rally deployment create --fromenv
|
||||
# recognizes standard OS_ENDPOINT environment variable
|
||||
|
||||
* Configure SSL per deployment
|
||||
|
||||
Now SSL information is deployment specific not Rally specific and
|
||||
rally.conf option is deprecated
|
||||
|
||||
Like in this sample
|
||||
https://github.com/openstack/rally/blob/14d0b5ba0c75ececfdb6a6c121d9cf2810571f77/samples/deployments/existing.json#L11-L12
|
||||
|
||||
|
||||
Specs
|
||||
~~~~~
|
||||
|
||||
* [spec] Proposal for new task input file format
|
||||
|
||||
This spec describes new task input format that will allow us to generate
|
||||
multi scenario load which is crucial for HA and more real life testing:
|
||||
|
||||
https://github.com/openstack/rally/blob/master/doc/specs/in-progress/new_rally_input_task_format.rst
|
||||
|
||||
|
||||
Plugins
|
||||
~~~~~~~
|
||||
|
||||
* **Benchmark Scenario Runners**:
|
||||
|
||||
* Add a maximum concurrency option to rps runner
|
||||
|
||||
To avoid running to heavy load you can set 'concurrency' to configuration
|
||||
and in case if cloud is not able to process all requests it won't start
|
||||
more parallel requests then 'concurrency' value.
|
||||
|
||||
|
||||
* **Benchmark Scenarios**:
|
||||
|
||||
[new] CeilometerAlarms.create_alarm_and_get_history
|
||||
|
||||
[new] KeystoneBasic.get_entities
|
||||
|
||||
[new] EC2Servers.boot_server
|
||||
|
||||
[new] KeystoneBasic.create_and_delete_service
|
||||
|
||||
[new] MuranoEnvironments.list_environments
|
||||
|
||||
[new] MuranoEnvironments.create_and_delete_environment
|
||||
|
||||
[new] NovaServers.suspend_and_resume_server
|
||||
|
||||
[new] NovaServers.pause_and_unpause_server
|
||||
|
||||
[new] NovaServers.boot_and_rebuild_server
|
||||
|
||||
[new] KeystoneBasic.create_and_list_services
|
||||
|
||||
[new] HeatStacks.list_stacks_and_events
|
||||
|
||||
[improved] VMTask.boot_runcommand_delete
|
||||
|
||||
restore ability to use fixed IP and floating IP to connect to VM
|
||||
via ssh
|
||||
|
||||
[fix] NovaServers.boot_server_attach_created_volume_and_live_migrate
|
||||
|
||||
Kwargs in nova scenario were wrongly passed
|
||||
|
||||
|
||||
|
||||
* **Benchmark SLA**:
|
||||
|
||||
* [new] aborted_on_sla
|
||||
|
||||
This is internal SLA criteria, that is added if task was aborted
|
||||
|
||||
|
||||
* [new] something_went_wrong
|
||||
|
||||
This is internal SLA criteria, that is added if something went wrong,
|
||||
context failed to create or runner raised some exceptions
|
||||
|
||||
|
||||
Bug fixes
|
||||
~~~~~~~~~
|
||||
|
||||
**14 bugs were fixed, the most critical are**:
|
||||
|
||||
* Set default task uuid to running task. Before it was set only after
|
||||
task was fully finished.
|
||||
|
||||
* The "rally task results" command showed a disorienting "task not found"
|
||||
message for a task that is currently running.
|
||||
|
||||
* Rally didn't know how to reconnect to OpenStack in case if token
|
||||
expired.
|
||||
|
||||
|
||||
Documentation
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
* New tutorial **task templates**
|
||||
|
||||
https://rally.readthedocs.org/en/latest/tutorial/step_4_task_templates.html
|
||||
|
||||
* Various fixes
|
||||
|
||||
180
doc/release_notes/latest.rst
Normal file
180
doc/release_notes/latest.rst
Normal file
@ -0,0 +1,180 @@
|
||||
============
|
||||
Rally v0.0.4
|
||||
============
|
||||
|
||||
Information
|
||||
-----------
|
||||
|
||||
+------------------+-----------------+
|
||||
| Commits | **87** |
|
||||
+------------------+-----------------+
|
||||
| Bug fixes | **21** |
|
||||
+------------------+-----------------+
|
||||
| Dev cycle | **30 days** |
|
||||
+------------------+-----------------+
|
||||
| Release date | **14/May/2015** |
|
||||
+------------------+-----------------+
|
||||
|
||||
|
||||
Details
|
||||
-------
|
||||
|
||||
This release contains new features, new benchmark plugins, bug fixes, various code and API improvements.
|
||||
|
||||
|
||||
New Features & API changes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* Rally now can generate load with users that already exist
|
||||
|
||||
Now one can use Rally for benchmarking OpenStack clouds that are using LDAP, AD or any other read-only keystone backend where it is not possible to create any users. To do this, one should set up the "users" section of the deployment configuration of the ExistingCloud type. This feature also makes it safer to run Rally against production clouds: when run from an isolated group of users, Rally won’t affect rest of the cloud users if something goes wrong.
|
||||
|
||||
* New decorator *@osclients.Clients.register* can add new OpenStack clients at runtime
|
||||
|
||||
It is now possible to add a new OpenStack client dynamically at runtime. The added client will be available from osclients.Clients at the module level and cached. Example:
|
||||
|
||||
.. code-block:: none
|
||||
|
||||
>>> from rally import osclients
|
||||
>>> @osclients.Clients.register("supernova")
|
||||
... def another_nova_client(self):
|
||||
... from novaclient import client as nova
|
||||
... return nova.Client("2", auth_token=self.keystone().auth_token,
|
||||
... **self._get_auth_info(password_key="key"))
|
||||
...
|
||||
>>> clients = osclients.Clients.create_from_env()
|
||||
>>> clients.supernova().services.list()[:2]
|
||||
[<Service: nova-conductor>, <Service: nova-cert>]
|
||||
|
||||
* Assert methods now available for scenarios and contexts
|
||||
|
||||
There is now a new *FunctionalMixin* class that implements basic unittest assert methods. The *base.Context* and *base.Scenario* classes inherit from this mixin, so now it is possible to use *base.assertX()* methods in scenarios and contexts.
|
||||
|
||||
* Improved installation script
|
||||
|
||||
The installation script has been almost completely rewritten. After this change, it can be run from an unprivileged user, supports different database types, allows to specify a custom python binary, always asks confirmation before doing potentially dangerous actions, automatically install needed software if run as root, and also automatically cleans up the virtualenv and/or the downloaded repository if interrupted.
|
||||
|
||||
|
||||
Specs & Feature requests
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* [Spec] Reorder plugins
|
||||
|
||||
The spec describes how to split Rally framework and plugins codebase to make it simpler for newbies to understand how Rally code is organized and how it works.
|
||||
|
||||
* [Feature request] Specify what benchmarks to execute in task
|
||||
|
||||
This feature request proposes to add the ability to specify benchmark(s) to be executed when the user runs the *rally task start* command. A possible solution would be to add a special flag to the *rally task start* command.
|
||||
|
||||
|
||||
Plugins
|
||||
~~~~~~~
|
||||
|
||||
* **Benchmark Scenario Runners**:
|
||||
|
||||
* Add limits for maximum Core usage to constant and rps runners
|
||||
|
||||
The new 'max_cpu_usage' parameter can be used to avoid possible 100% usage of all available CPU cores by reducing the number of CPU cores available for processes started by the corresponding runner.
|
||||
|
||||
|
||||
* **Benchmark Scenarios**:
|
||||
|
||||
* [new] KeystoneBasic.create_update_and_delete_tenant
|
||||
|
||||
* [new] KeystoneBasic.create_user_update_password
|
||||
|
||||
* [new] NovaServers.shelve_and_unshelve_server
|
||||
|
||||
* [new] NovaServers.boot_and_associate_floating_ip
|
||||
|
||||
* [new] NovaServers.boot_lock_unlock_and_delete
|
||||
|
||||
* [new] NovaHypervisors.list_hypervisors
|
||||
|
||||
* [new] CeilometerSamples.list_samples
|
||||
|
||||
* [new] CeilometerResource.get_resources_on_tenant
|
||||
|
||||
* [new] SwiftObjects.create_container_and_object_then_delete_all
|
||||
|
||||
* [new] SwiftObjects.create_container_and_object_then_download_object
|
||||
|
||||
* [new] SwiftObjects.create_container_and_object_then_list_objects
|
||||
|
||||
* [new] MuranoEnvironments.create_and_deploy_environment
|
||||
|
||||
* [new] HttpRequests.check_random_request
|
||||
|
||||
* [new] HttpRequests.check_request
|
||||
|
||||
* [improved] NovaServers live migrate benchmarks
|
||||
|
||||
add 'min_sleep' and 'max_sleep' parameters to simulate a pause between VM booting and running live migration
|
||||
|
||||
* [improved] NovaServers.boot_and_live_migrate_server
|
||||
|
||||
add a usage sample to samples/tasks
|
||||
|
||||
* [improved] CinderVolumes benchmarks
|
||||
|
||||
support size range to be passed to the 'size' argument as a dictionary
|
||||
*{"min": <minimum_size>, "max": <maximum_size>}*
|
||||
|
||||
|
||||
* **Benchmark Contexts**:
|
||||
|
||||
* [new] MuranoPackage
|
||||
|
||||
This new context can upload a package to Murano from some specified path.
|
||||
|
||||
* [new] CeilometerSampleGenerator
|
||||
|
||||
Context that can be used for creating samples and collecting resources for benchmarks in a list.
|
||||
|
||||
|
||||
* **Benchmark SLA**:
|
||||
|
||||
* [new] outliers
|
||||
|
||||
This new SLA checks that the number of outliers (calculated from the mean and standard deviation of the iteration durations) does not exceed some maximum value. The SLA is highly configurable: the parameters used for outliers threshold calculation can be set by the user.
|
||||
|
||||
|
||||
Bug fixes
|
||||
~~~~~~~~~
|
||||
|
||||
**21 bugs were fixed, the most critical are**:
|
||||
|
||||
* Make it possible to use relative imports for plugins that are outside of rally package.
|
||||
|
||||
* Fix heat stacks cleanup by deleting them only 1 time per tenant (get rid of "stack not found" errors in logs).
|
||||
|
||||
* Fix the wrong behaviour of 'rally task detailed --iterations-data' (it lacked the iteration info before).
|
||||
|
||||
* Fix security groups cleanup: a security group called "default", created automatically by Neutron, did not get deleted for each tenant.
|
||||
|
||||
|
||||
Other changes
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
* Streaming algorithms that scale
|
||||
|
||||
This release introduces the common/streaming_algorithms.py module. This module is going to contain implementations of benchmark data processing algorithms that scale: these algorithms do not store exhaustive information about every single benchmark iteration duration processed. For now, the module contains implementations of algorithms for computation of mean & standard deviation.
|
||||
|
||||
* Coverage job to check that new patches come with unit tests
|
||||
|
||||
Rally now has a coverage job that checks that every patch submitted for review does not decrease the number of lines covered by unit tests (at least too much). This job allows to mark most patches with no unit tests with '-1'.
|
||||
|
||||
* Splitting the plugins code (Runners & SLA) into common/openstack plugins
|
||||
|
||||
According to the spec "Reorder plugins" (see above), the plugins code for runners and SLA has been moved to the *plugins/common/* directory. Only base classes now remain in the *benchmark/* directory.
|
||||
|
||||
|
||||
Documentation
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
* Various fixes
|
||||
|
||||
* Remove obsolete *.rst* files (*deploy_engines.rst* / *server_providers.rst* / ...)
|
||||
* Restructure the docs files to make them easier to natigate through
|
||||
* Move the chapter on task templates to the 4th step in the tutorial
|
||||
* Update the information about meetings (new release meeting & time changes)
|
||||
Loading…
Reference in New Issue
Block a user