openstack/performance-docs
Code Issues Proposed changes

Add plan<->result references everywhere

Browse Source 
Change-Id: I68acbcd3f656faa57540d2d7d4b8e1f2ffb8cfae
Closes-Bug: #1609927
This commit is contained in:
Dina Belova 2016-08-11 10:09:36 -07:00
parent d391be06bc
commit 20b0943204
34 changed files with 162 additions and 21 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
doc/source/test_plans/1000_nodes/plan.rst
View File

@ -202,3 +202,8 @@ Priority Value Measurement Units Description
spawn
======== =============== ================= =================================
Reports
=======
Test plan execution reports:
* :ref:`1000_nodes_report`

21
doc/source/test_plans/container_cluster_systems/plan.rst
View File

@ -20,14 +20,12 @@ Measuring performance of container cluster systems
Kubernetes, and Docker Swarm,
which provide various ways to manage groups of containers.
:Conventions:
Test Plan
=========
Test Environment
----------------
Preparation
^^^^^^^^^^^
Typical container frameworks consist of several components:
@ -46,6 +44,7 @@ framework.
Test Case #1: A new application creation time
---------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the create
@ -138,6 +137,7 @@ Measuring values
Test Case #2: A application CPU update time
-------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update CPU
@ -234,6 +234,7 @@ Measuring values
Test Case #3: A application memory update time
----------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update memory
@ -330,6 +331,7 @@ Measuring values
Test Case #4: A application disk update time
--------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update disk
@ -426,6 +428,7 @@ Measuring values
Test Case #5: A application instances update time
-------------------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the update number of
@ -521,6 +524,7 @@ Measuring values
Test Case #6: A application restart time
----------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the restart
@ -617,6 +621,7 @@ Measuring values
Test Case #7: A application delete time
---------------------------------------
Description
^^^^^^^^^^^
This test is aimed at measuring the total elapsed time of the delete
@ -713,6 +718,7 @@ Measuring values
Test Case #8: Health check Performance
--------------------------------------
Description
^^^^^^^^^^^
This test is aimed at validating the response time of a health check when many
@ -783,6 +789,7 @@ Measuring REAL_INTERVAL(TASK_COUNT) values
Applications
============
list of container platforms
---------------------------
@ -795,3 +802,9 @@ list of container platforms
+-----------------------------+---------+
| Docker | |
+-----------------------------+---------+
Reports
=======
Test plan execution reports:
* :ref:`Results_of_measuring_performance_of_Mesos_Marathon`

8
doc/source/test_plans/container_repositories/plan.rst
View File

@ -330,3 +330,11 @@ Proposed docker file
.. _Docker Registry2: https://docs.docker.com/registry
.. _JFrog Artifactory: https://www.jfrog.com/artifactory
.. _Docker: https://www.docker.com
Reports
=======
Test plan execution reports:
* :ref:`Measuring_performance_of_Sonatype_Nexus`
* :ref:`Measuring_performance_of_JFrog_Artifactory_Pro`
* :ref:`Measuring_performance_of_docker_registry`

6
doc/source/test_plans/control_plane/plan.rst
View File

@ -364,3 +364,9 @@ Example of Rally scenario configuration
.. literalinclude:: test_plans/keystone_authenticate.json
:language: bash
Reports
=======
Test plan execution reports:
* :ref:`openstack_control_plane_performance_report`

6
doc/source/test_plans/controlplane_density/plan.rst
View File

@ -185,3 +185,9 @@ may look like:
+===============+================+====================+===================+=============+=============+
| create_volume | <mean_numbers> | <90%ile_numbers> | <50%ile_numbers> | <max_value> | <min_value> |
+---------------+----------------+--------------------+-------------------+-------------+-------------+
Reports
=======
Test plan execution reports:
* :ref:`Results_of_OpenStack_services_density_testing`

7
doc/source/test_plans/db/plan.rst
View File

@ -323,3 +323,10 @@ Elapsed time or latency for high-level OpenStack operations like virtual
machine creation or network configuration may involve many database queries.
The maximum query throughput the database layer can process with
reasonable latency determines the maximum cluster size that can be supported.
Reports
=======
Test plan execution reports:
* :ref:`db_performance_mysql`
* :ref:`db_performance_mariadb`

6
doc/source/test_plans/hardware_features/hardware_offloads/plan.rst
View File

@ -517,3 +517,9 @@ Priority Value Measurement Units Description
======== =============== ================= =====================================================
.. _Intel XL710 controller: http://www.intel.com/content/www/us/en/embedded/products/networking/xl710-10-40-gbe-controller-brief.html
Reports
=======
Test plan execution reports:
* :ref:`hw_features_offloads`

6
doc/source/test_plans/keystone/plan.rst
View File

@ -418,3 +418,9 @@ Priority Value Measurement Units Description
======== ============== ================= =========================================
1 Operation time milliseconds Time spent on every HTTP/RPC/DB operation
======== ============== ================= =========================================
Reports
=======
Test plan execution reports:
* :ref:`keystone_performance_all_in_one`

7
doc/source/test_plans/mq/plan.rst
View File

@ -434,3 +434,10 @@ This section contains tools that can be used to perform the test plan.
.. _RPC call: http://docs.openstack.org/developer/oslo.messaging/rpcclient.html#oslo_messaging.RPCClient.call
.. _RPC cast: http://docs.openstack.org/developer/oslo.messaging/rpcclient.html#oslo_messaging.RPCClient.cast
.. _Notification: http://docs.openstack.org/developer/oslo.messaging/notifier.html#notifier
Reports
=======
Test plan execution reports:
* :ref:`mq_rabbit_report`
* :ref:`zeromq_report`

3
doc/source/test_plans/mq_ha/plan.rst
View File

@ -242,7 +242,8 @@ Reports
=======
Test plan execution reports:
* :ref:`mq_ha_rabbit_report`
* :ref:`mq_ha_rabbit_report_disabled`
* :ref:`mq_ha_rabbit_report_enabled`
.. references:

7
doc/source/test_plans/neutron_features/l3_ha/plan.rst
View File

@ -319,3 +319,10 @@ Priority Value Measurement Units Description
======== =============== ================= ====================================
1 UDP bandwidth % Loss of UDP packets of 64 bytes size
======== =============== ================= ====================================
Reports
=======
Test plan execution reports:
* :ref:`neutron_l3_ha_test_report_liberty`
* :ref:`neutron_l3_ha_test_report_mitaka`

6
doc/source/test_plans/openstack_load/plan.rst
View File

@ -186,3 +186,9 @@ In this scenario Rally boots Nova servers at constant rate:
#. boot server
.. literalinclude:: rally_scenarios/nova_boot_servers.json
Reports
=======
Test plan execution reports:
* :ref:`openstack_load_report`

6
doc/source/test_plans/provisioning/plan.rst
View File

@ -354,3 +354,9 @@ Full script for collecting performance metrics
.. _dstat: http://dag.wiee.rs/home-made/dstat/
.. _Cobbler: http://cobbler.github.io/
.. _Razor: https://github.com/puppetlabs/razor-server
Reports
=======
Test plan execution reports:
* :ref:`Measuring_performance_of_Cobbler`

6
doc/source/test_plans/reliability/plan.rst
View File

@ -375,3 +375,9 @@ Detailed results description
In this section, specify detailed description of test results,
including factor impact.
Reports
=======
Test plan execution reports:
* :ref:`reliability_testing_results`

6
doc/source/test_plans/tenant_networking/plan.rst
View File

@ -423,3 +423,9 @@ Tools
This section contains tools that can be used to perform the test plan.
.. include:: shaker.rst
Reports
=======
Test plan execution reports:
* :ref:`tenant_networking_report_vxlan_dvr_200_nodes`

8
doc/source/test_results/1000_nodes/index.rst
View File

@ -1,6 +1,14 @@
.. raw:: pdf
PageBreak oneColumn
.. _1000_nodes_report:
1000 Compute nodes resource scalability testing
===============================================
This report is generated for :ref:`1000_nodes` test plan.
Environment setup
-----------------

9
doc/source/test_results/container_cluster_systems/mesos-marathon/mesos-marathon.rst
View File

@ -7,13 +7,12 @@ Results of measuring performance of Mesos Marathon
:Abstract:
This document includes performance test results for `Mesos`_ as a container
platform for `Docker`_ with `Marathon`_. All test have been performed
regarding
:ref:`Measuring_performance_of_mesos_marathon`
platform for `Docker`_ with `Marathon`_. All tests have been performed
regarding :ref:`Measuring_performance_of_container_cluster_systems`
Environment description
=======================
Cluster description
-------------------
* 3 master nodes with Mesos, Marathon and Zookeeper
@ -30,8 +29,6 @@ Running options
---------------
* Marathon extra options are "-Xmx2048m --max_tasks_per_offer 20"
Hardware configuration of each server
-------------------------------------

6
doc/source/test_results/container_repositories/artifactory/index.rst
View File

@ -14,6 +14,7 @@ Results of measuring performance of JFrog Artifactory Pro
Environment description
=======================
Hardware configuration of each server
-------------------------------------
@ -79,6 +80,7 @@ p1p1 interface of a server:
Software configuration of the Artifactory service
-------------------------------------------------
Installation of Artifactory:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
@ -125,6 +127,7 @@ You can find outputs of some commands and /etc folder in the following archive:
Software configuration of the node with test tool
-------------------------------------------------
Test tool:
^^^^^^^^^^
Firstly we need to install docker-engine:
@ -212,8 +215,10 @@ As a result of this part we got the following CSV files:
Results
=======
Push action results
-------------------
PUSH_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for push action in depend on
@ -277,6 +282,7 @@ parameter.
Pull action results
-------------------
PULL_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for pull action in depend on

8
doc/source/test_results/container_repositories/nexus/index.rst
View File

@ -14,6 +14,7 @@ Results of measuring performance of Sonatype Nexus
Environment description
=======================
Hardware configuration of each server
-------------------------------------
@ -78,7 +79,8 @@ p1p1 interface of a server:
no snmp trap link-status
Software configuration of the Nexus service
----------------------------------------------------
-------------------------------------------
Installation of Nexus:
^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
@ -136,6 +138,7 @@ You can find outputs of some commands and /etc folder in the following archive:
Software configuration of the node with test tool
-------------------------------------------------
Test tool:
^^^^^^^^^^
Firstly we need to install docker-engine:
@ -222,8 +225,10 @@ As a result of this part we got the following CSV files:
Results
=======
Push action results
-------------------
PUSH_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for push action in depend on
@ -286,6 +291,7 @@ parameter.
Pull action results
-------------------
PULL_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for pull action in depend on

6
doc/source/test_results/container_repositories/registry2/index.rst
View File

@ -14,6 +14,7 @@ Results of measuring performance of Docker Registry
Environment description
=======================
Hardware configuration of each server
-------------------------------------
@ -79,6 +80,7 @@ p1p1 interface of a server:
Software configuration of the DockerRegistry service
----------------------------------------------------
Installation of Registry2:
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code:: bash
@ -106,6 +108,7 @@ You can find outputs of some commands and /etc folder in the following archive:
Software configuration of the node with test tool
-------------------------------------------------
Test tool:
^^^^^^^^^^
Firstly we need to install docker-engine:
@ -191,8 +194,10 @@ As a result of this part we got the following CSV files:
Results
=======
Push action results
-------------------
PUSH_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for push action in depend on
@ -255,6 +260,7 @@ parameter.
Pull action results
-------------------
PULL_TIME(ITERATION)
^^^^^^^^^^^^^^^^^^^^
After simple processing results the following plots for pull action in depend on

2
doc/source/test_results/controlplane_density/index.rst
View File

@ -9,7 +9,7 @@ Results of OpenStack Services density testing
This document includes density test results of OpenStack
services. All tests have been performed
regarding :ref:`controlplane_density_`
regarding :ref:`controlplane_density`
Environment description
=======================

2
doc/source/test_results/db/mariadb/index.rst
View File

@ -2,6 +2,8 @@
PageBreak oneColumn
.. _db_performance_mariadb:
===================================
MariaDB Database performance report
===================================

2
doc/source/test_results/db/mysql/index.rst
View File

@ -2,6 +2,8 @@
PageBreak oneColumn
.. _db_performance_mysql:
=================================
MySQL Database performance report
=================================

5
doc/source/test_results/hardware_features/hardware_offloads/test_results.rst
View File

@ -1,7 +1,12 @@
.. _hw_features_offloads:
================================
Hardware Offloads - Test results
================================
This report is generated for :ref:`hardware_offloads_performance_analysis` test
plan.
Baseline physical scenario
--------------------------

2
doc/source/test_results/keystone/all-in-one/index.rst
View File

@ -1,3 +1,5 @@
.. _keystone_performance_all_in_one:
Keystone DB / cache operations analysis
---------------------------------------

2
doc/source/test_results/mq/rabbitmq/index.rst
View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _mq_rabbit_report:
===========================
RabbitMQ Performance Report

2
doc/source/test_results/mq/zeromq/index.rst
View File

@ -1,3 +1,5 @@
.. _zeromq_report:
ZeroMQ performance
------------------

2
doc/source/test_results/mq_ha/rabbitmq-ha-queues/index.rst
View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _mq_ha_rabbit_report_enabled:
===========================================
RabbitMQ HA Test Reports: HA queues enabled

2
doc/source/test_results/mq_ha/rabbitmq/index.rst
View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _mq_ha_rabbit_report_disabled:
============================================
RabbitMQ HA Test Reports: HA queues disabled

4
doc/source/test_results/neutron_features/l3_ha/test_results_liberty.rst
View File

@ -1,6 +1,10 @@
.. _neutron_l3_ha_test_report_liberty:
Neutron L3 HA test results Liberty
----------------------------------
This report is generated for :ref:`neutron_l3_ha_test_plan` test plan.
Environment description
=======================

4
doc/source/test_results/neutron_features/l3_ha/test_results_mitaka.rst
View File

@ -1,3 +1,5 @@
.. _neutron_l3_ha_test_report_mitaka:
Neutron L3 HA test results Mitaka
---------------------------------
@ -5,6 +7,8 @@ As we have already done initial testing of L3 HA functionality for Liberty,
in this testing load parameters are increased (for example, concurrency for Rally
tests, number of routers for manual tests).
This report is generated for :ref:`neutron_l3_ha_test_plan` test plan.
Environment description
=======================

2
doc/source/test_results/provisioning/cobbler/index.rst
View File

@ -8,7 +8,7 @@ Results of measuring performance of Cobbler
:Abstract:
This document includes performance test results of `Cobbler`_
service as a provisioning system. All test have been performed
service as a provisioning system. All tests have been performed
regarding :ref:`Measuring_performance_of_provisioning_systems`

3
doc/source/test_results/reliability/index.rst
View File

@ -10,7 +10,8 @@ OpenStack reliability testing
:Abstract:
This document describes an abstract methodology for OpenStack cluster
high-availability testing and analysis. OpenStack data plane testing
at this moment is out of scope but will be described in future.
at this moment is out of scope but will be described in future. All tests
have been performed regarding :ref:`reliability_testing`
:Conventions:

2
doc/source/test_results/tenant_networking/neutron_vxlan_dvr_200/index.rst
View File

@ -2,7 +2,7 @@
PageBreak oneColumn
.. _mq_ha_rabbit_report:
.. _tenant_networking_report_vxlan_dvr_200_nodes:
===================================================
Tenant Networking: Neutron DVR + VXLAN at 200 nodes