Merge "Add test plan for testing of API of k8s."
This commit is contained in:
Jenkins
Gerrit Code Review
commit
bfd23fbace
232
doc/source/test_plans/container_cluster_systems/API_latency.rst
Normal file
232
doc/source/test_plans/container_cluster_systems/API_latency.rst
Normal file
@ -0,0 +1,232 @@
|
||||
|
||||
.. _Measuring_of_API_performance_of_container_cluster_system:
|
||||
|
||||
*********************************************************
|
||||
Measuring of API performance of container cluster systems
|
||||
*********************************************************
|
||||
|
||||
:status: **ready**
|
||||
:version: 1.0
|
||||
|
||||
:Abstract:
|
||||
|
||||
This document describes a test plan for quantifying the API performance of
|
||||
container cluster systems.
|
||||
|
||||
Test Plan
|
||||
=========
|
||||
Test Environment
|
||||
----------------
|
||||
Preparation
|
||||
^^^^^^^^^^^
|
||||
To test container cluster some tool dedicated to measure CRUD operations
|
||||
latency is needed. For Kubernetes container cluster system we can propose
|
||||
"Load test" of `e2e-tests`_ tool.
|
||||
|
||||
Environment description
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Test results MUST include a description of the environment used. The following
|
||||
items should be described:
|
||||
|
||||
- **Hardware configuration of each server.** If virtual machines are used then
|
||||
both physical and virtual hardware should be fully documented.
|
||||
An example format is given below:
|
||||
|
||||
.. table:: Description of servers hardware
|
||||
|
||||
+-------+----------------+-------+-------+
|
||||
|server |name | | |
|
||||
| +----------------+-------+-------+
|
||||
| |role | | |
|
||||
| +----------------+-------+-------+
|
||||
| |vendor,model | | |
|
||||
| +----------------+-------+-------+
|
||||
| |operating_system| | |
|
||||
+-------+----------------+-------+-------+
|
||||
|CPU |vendor,model | | |
|
||||
| +----------------+-------+-------+
|
||||
| |processor_count | | |
|
||||
| +----------------+-------+-------+
|
||||
| |core_count | | |
|
||||
| +----------------+-------+-------+
|
||||
| |frequency_MHz | | |
|
||||
+-------+----------------+-------+-------+
|
||||
|RAM |vendor,model | | |
|
||||
| +----------------+-------+-------+
|
||||
| |amount_MB | | |
|
||||
+-------+----------------+-------+-------+
|
||||
|NETWORK|interface_name | | |
|
||||
| +----------------+-------+-------+
|
||||
| |vendor,model | | |
|
||||
| +----------------+-------+-------+
|
||||
| |bandwidth | | |
|
||||
+-------+----------------+-------+-------+
|
||||
|STORAGE|dev_name | | |
|
||||
| +----------------+-------+-------+
|
||||
| |vendor,model | | |
|
||||
| +----------------+-------+-------+
|
||||
| |SSD/HDD | | |
|
||||
| +----------------+-------+-------+
|
||||
| |size | | |
|
||||
+-------+----------------+-------+-------+
|
||||
|
||||
- **Configuration of hardware network switches** The configuration file from
|
||||
the switch can be downloaded and attached.
|
||||
|
||||
- **Configuration of virtual machines and virtual networks (if they are used)**
|
||||
The configuration files can be attached, along with the mapping of virtual
|
||||
machines to host machines.
|
||||
|
||||
- **Network scheme.** The plan should show how all hardware is connected and
|
||||
how the components communicate. All ethernet/fibrechannel and VLAN channels
|
||||
should be included. Each interface of every hardware component should be
|
||||
matched with the corresponding L2 channel and IP address.
|
||||
|
||||
- **Software configuration of the container cluster system** `sysctl.conf` and
|
||||
any other kernel file that is changed from the default should be attached.
|
||||
List of installed packages should be attached. Specifications of the
|
||||
operating system, network interfaces configuration, and disk partitioning
|
||||
configuration should be included. If distributed provisioning systems are
|
||||
to be tested then the parts that are distributed need to be described.
|
||||
|
||||
- **Software configuration of the node with test tool** The operating system,
|
||||
disk partitioning scheme, network interface configuration, installed packages
|
||||
and other components of client nodes define limits which a client can
|
||||
experience during sending requests and getting responses to/from docker
|
||||
repository.
|
||||
|
||||
Test Case #1: API latencies of requests which make actions on containers
|
||||
------------------------------------------------------------------------
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
During this test basic actions with containers like start, delete, update
|
||||
should be performed and API latencies should be measured.
|
||||
|
||||
List of performance metrics
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. table:: list of test metrics to be collected during this test
|
||||
|
||||
+-----------------------------+---------------------------------------------+
|
||||
| Parameter |Description |
|
||||
+=============================+=============================================+
|
||||
|| CONTAINER_API_LATENCIES | | The time which a client spends to get a |
|
||||
|| (GET,PUT,POST,DELETE, LIST | | response from container system API to make|
|
||||
|| types of requests) | | an action on a docker container |
|
||||
+-----------------------------+---------------------------------------------+
|
||||
|
||||
Measuring values of API latencies of requests which make actions on containers
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
1.
|
||||
Deploy container cluster system from scratch. We should be sure that there is
|
||||
no data in the container cluster system.
|
||||
|
||||
2.
|
||||
Make create, delete, update and any available requests to perform an action
|
||||
on container to API of container cluster system and measure latencies of
|
||||
the responses. The API client which you use should't crate a big load on the
|
||||
cluster system. The load should be spreading over time.
|
||||
|
||||
3.
|
||||
As a result of the previous step you should be able to provide the table with
|
||||
99 percentile of API latencies in depend on type of requests. You need to
|
||||
fill the table similar the table bellow with calculated values:
|
||||
|
||||
.. table:: API latencies of requests which make actions on containers
|
||||
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| Method | Perc99 |
|
||||
+=================================+=========================================+
|
||||
| PUT | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| GET | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| LIST | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| DELETE | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| POST | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
|
||||
Test Case #2: API latencies of any other supported requests
|
||||
-----------------------------------------------------------
|
||||
Description
|
||||
^^^^^^^^^^^
|
||||
In opposite of
|
||||
`Test Case #1: API latencies of requests which make actions on containers`_ in
|
||||
in this test case requests different from actions with containers should be
|
||||
performed and API latencies should be measured. The supported by API of
|
||||
container cluster system requests depend on the container cluster system. For
|
||||
example for Kubernetes it can be actions with replicationcontrollers or
|
||||
namespaces.
|
||||
|
||||
List of performance metrics
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. table:: list of test metrics to be collected during this test
|
||||
|
||||
+-----------------------------+---------------------------------------------+
|
||||
| Parameter |Description |
|
||||
+=============================+=============================================+
|
||||
|| API_LATENCIES | | The time which a client spends to get a |
|
||||
|| (GET,PUT,POST,DELETE, LIST | | response from container system API |
|
||||
|| types of requests) | | |
|
||||
+-----------------------------+---------------------------------------------+
|
||||
|
||||
Measuring values of API latencies of any other supported requests
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
1.
|
||||
Deploy container cluster system from scratch. We should be sure that there is
|
||||
no data in the container cluster system.
|
||||
|
||||
2.
|
||||
Make available requests to API of container cluster system and measure
|
||||
latencies of the responses. The API client which you use should't crate a big
|
||||
load on the cluster system. The load should be spreading over time.
|
||||
|
||||
3.
|
||||
As a result of the previous step you should be able to provide the table with
|
||||
99 percentile of API latencies in depend on type of requests. You need to
|
||||
fill the table similar the table bellow with calculated values:
|
||||
|
||||
.. table:: Table #1 API latencies of requests which make actions on item #1
|
||||
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| Method | Perc99 |
|
||||
+=================================+=========================================+
|
||||
| PUT | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| GET | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| LIST | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| DELETE | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| POST | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
|
||||
.. table:: Table #2 API latencies of requests which make actions on item #2
|
||||
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| Method | Perc99 |
|
||||
+=================================+=========================================+
|
||||
| PUT | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| GET | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| LIST | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| DELETE | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
| POST | |
|
||||
+---------------------------------+-----------------------------------------+
|
||||
|
||||
.. references:
|
||||
|
||||
.. _e2e-tests: https://github.com/kubernetes/kubernetes/blob/release-1.4/docs/devel/e2e-tests.md
|
||||
|
||||
Reports
|
||||
=======
|
||||
|
||||
Test plan execution reports:
|
||||
* :ref:`Results_of_Measuring_of_API_performance_of_Kubernetes`
|
||||
22
doc/source/test_plans/container_cluster_systems/index.rst
Normal file
22
doc/source/test_plans/container_cluster_systems/index.rst
Normal file
@ -0,0 +1,22 @@
|
||||
Container cluster systems
|
||||
*************************
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
This part of documentation describes different aspects of testing of container
|
||||
cluster systems
|
||||
|
||||
Contents
|
||||
========
|
||||
|
||||
.. toctree::
|
||||
:numbered:
|
||||
:maxdepth: 2
|
||||
|
||||
performance_and_scaling
|
||||
API_latency
|
||||
|
||||
.. raw:: pdf
|
||||
|
||||
PageBreak oneColumn
|
||||
Loading…
Reference in New Issue
Block a user