From a0ad63b4c7e85b94442bdcc1839edb9c5eca024d Mon Sep 17 00:00:00 2001 From: Leontii Istomin Date: Mon, 26 Sep 2016 11:43:37 +0300 Subject: [PATCH] Add test plan for testing of API of k8s. Add test plan for testing of API of container cluster systems. Change-Id: I98c0e1b3289d78736e84cc9a6294b3395a0d7cd9 --- .../container_cluster_systems/API_latency.rst | 232 ++++++++++++++++++ .../container_cluster_systems/index.rst | 22 ++ .../{plan.rst => performance_and_scaling.rst} | 0 3 files changed, 254 insertions(+) create mode 100644 doc/source/test_plans/container_cluster_systems/API_latency.rst create mode 100644 doc/source/test_plans/container_cluster_systems/index.rst rename doc/source/test_plans/container_cluster_systems/{plan.rst => performance_and_scaling.rst} (100%) diff --git a/doc/source/test_plans/container_cluster_systems/API_latency.rst b/doc/source/test_plans/container_cluster_systems/API_latency.rst new file mode 100644 index 0000000..d13d683 --- /dev/null +++ b/doc/source/test_plans/container_cluster_systems/API_latency.rst @@ -0,0 +1,232 @@ + +.. _Measuring_of_API_performance_of_container_cluster_system: + +********************************************************* +Measuring of API performance of container cluster systems +********************************************************* + +:status: **ready** +:version: 1.0 + +:Abstract: + + This document describes a test plan for quantifying the API performance of + container cluster systems. + +Test Plan +========= +Test Environment +---------------- +Preparation +^^^^^^^^^^^ +To test container cluster some tool dedicated to measure CRUD operations +latency is needed. For Kubernetes container cluster system we can propose +"Load test" of `e2e-tests`_ tool. + +Environment description +^^^^^^^^^^^^^^^^^^^^^^^ +Test results MUST include a description of the environment used. The following +items should be described: + +- **Hardware configuration of each server.** If virtual machines are used then + both physical and virtual hardware should be fully documented. + An example format is given below: + +.. table:: Description of servers hardware + + +-------+----------------+-------+-------+ + |server |name | | | + | +----------------+-------+-------+ + | |role | | | + | +----------------+-------+-------+ + | |vendor,model | | | + | +----------------+-------+-------+ + | |operating_system| | | + +-------+----------------+-------+-------+ + |CPU |vendor,model | | | + | +----------------+-------+-------+ + | |processor_count | | | + | +----------------+-------+-------+ + | |core_count | | | + | +----------------+-------+-------+ + | |frequency_MHz | | | + +-------+----------------+-------+-------+ + |RAM |vendor,model | | | + | +----------------+-------+-------+ + | |amount_MB | | | + +-------+----------------+-------+-------+ + |NETWORK|interface_name | | | + | +----------------+-------+-------+ + | |vendor,model | | | + | +----------------+-------+-------+ + | |bandwidth | | | + +-------+----------------+-------+-------+ + |STORAGE|dev_name | | | + | +----------------+-------+-------+ + | |vendor,model | | | + | +----------------+-------+-------+ + | |SSD/HDD | | | + | +----------------+-------+-------+ + | |size | | | + +-------+----------------+-------+-------+ + +- **Configuration of hardware network switches** The configuration file from + the switch can be downloaded and attached. + +- **Configuration of virtual machines and virtual networks (if they are used)** + The configuration files can be attached, along with the mapping of virtual + machines to host machines. + +- **Network scheme.** The plan should show how all hardware is connected and + how the components communicate. All ethernet/fibrechannel and VLAN channels + should be included. Each interface of every hardware component should be + matched with the corresponding L2 channel and IP address. + +- **Software configuration of the container cluster system** `sysctl.conf` and + any other kernel file that is changed from the default should be attached. + List of installed packages should be attached. Specifications of the + operating system, network interfaces configuration, and disk partitioning + configuration should be included. If distributed provisioning systems are + to be tested then the parts that are distributed need to be described. + +- **Software configuration of the node with test tool** The operating system, + disk partitioning scheme, network interface configuration, installed packages + and other components of client nodes define limits which a client can + experience during sending requests and getting responses to/from docker + repository. + +Test Case #1: API latencies of requests which make actions on containers +------------------------------------------------------------------------ +Description +^^^^^^^^^^^ +During this test basic actions with containers like start, delete, update +should be performed and API latencies should be measured. + +List of performance metrics +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. table:: list of test metrics to be collected during this test + + +-----------------------------+---------------------------------------------+ + | Parameter |Description | + +=============================+=============================================+ + || CONTAINER_API_LATENCIES | | The time which a client spends to get a | + || (GET,PUT,POST,DELETE, LIST | | response from container system API to make| + || types of requests) | | an action on a docker container | + +-----------------------------+---------------------------------------------+ + +Measuring values of API latencies of requests which make actions on containers +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +1. + Deploy container cluster system from scratch. We should be sure that there is + no data in the container cluster system. + +2. + Make create, delete, update and any available requests to perform an action + on container to API of container cluster system and measure latencies of + the responses. The API client which you use should't crate a big load on the + cluster system. The load should be spreading over time. + +3. + As a result of the previous step you should be able to provide the table with + 99 percentile of API latencies in depend on type of requests. You need to + fill the table similar the table bellow with calculated values: + +.. table:: API latencies of requests which make actions on containers + + +---------------------------------+-----------------------------------------+ + | Method | Perc99 | + +=================================+=========================================+ + | PUT | | + +---------------------------------+-----------------------------------------+ + | GET | | + +---------------------------------+-----------------------------------------+ + | LIST | | + +---------------------------------+-----------------------------------------+ + | DELETE | | + +---------------------------------+-----------------------------------------+ + | POST | | + +---------------------------------+-----------------------------------------+ + +Test Case #2: API latencies of any other supported requests +----------------------------------------------------------- +Description +^^^^^^^^^^^ +In opposite of +`Test Case #1: API latencies of requests which make actions on containers`_ in +in this test case requests different from actions with containers should be +performed and API latencies should be measured. The supported by API of +container cluster system requests depend on the container cluster system. For +example for Kubernetes it can be actions with replicationcontrollers or +namespaces. + +List of performance metrics +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. table:: list of test metrics to be collected during this test + + +-----------------------------+---------------------------------------------+ + | Parameter |Description | + +=============================+=============================================+ + || API_LATENCIES | | The time which a client spends to get a | + || (GET,PUT,POST,DELETE, LIST | | response from container system API | + || types of requests) | | | + +-----------------------------+---------------------------------------------+ + +Measuring values of API latencies of any other supported requests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +1. + Deploy container cluster system from scratch. We should be sure that there is + no data in the container cluster system. + +2. + Make available requests to API of container cluster system and measure + latencies of the responses. The API client which you use should't crate a big + load on the cluster system. The load should be spreading over time. + +3. + As a result of the previous step you should be able to provide the table with + 99 percentile of API latencies in depend on type of requests. You need to + fill the table similar the table bellow with calculated values: + +.. table:: Table #1 API latencies of requests which make actions on item #1 + + +---------------------------------+-----------------------------------------+ + | Method | Perc99 | + +=================================+=========================================+ + | PUT | | + +---------------------------------+-----------------------------------------+ + | GET | | + +---------------------------------+-----------------------------------------+ + | LIST | | + +---------------------------------+-----------------------------------------+ + | DELETE | | + +---------------------------------+-----------------------------------------+ + | POST | | + +---------------------------------+-----------------------------------------+ + +.. table:: Table #2 API latencies of requests which make actions on item #2 + + +---------------------------------+-----------------------------------------+ + | Method | Perc99 | + +=================================+=========================================+ + | PUT | | + +---------------------------------+-----------------------------------------+ + | GET | | + +---------------------------------+-----------------------------------------+ + | LIST | | + +---------------------------------+-----------------------------------------+ + | DELETE | | + +---------------------------------+-----------------------------------------+ + | POST | | + +---------------------------------+-----------------------------------------+ + +.. references: + +.. _e2e-tests: https://github.com/kubernetes/kubernetes/blob/release-1.4/docs/devel/e2e-tests.md + +Reports +======= + +Test plan execution reports: + * :ref:`Results_of_Measuring_of_API_performance_of_Kubernetes` diff --git a/doc/source/test_plans/container_cluster_systems/index.rst b/doc/source/test_plans/container_cluster_systems/index.rst new file mode 100644 index 0000000..947eaed --- /dev/null +++ b/doc/source/test_plans/container_cluster_systems/index.rst @@ -0,0 +1,22 @@ +Container cluster systems +************************* + +Abstract +======== + +This part of documentation describes different aspects of testing of container +cluster systems + +Contents +======== + +.. toctree:: + :numbered: + :maxdepth: 2 + + performance_and_scaling + API_latency + +.. raw:: pdf + + PageBreak oneColumn \ No newline at end of file diff --git a/doc/source/test_plans/container_cluster_systems/plan.rst b/doc/source/test_plans/container_cluster_systems/performance_and_scaling.rst similarity index 100% rename from doc/source/test_plans/container_cluster_systems/plan.rst rename to doc/source/test_plans/container_cluster_systems/performance_and_scaling.rst