performance-docs/doc/source/test_plans/fuel_ccp_upgrade/plan.rst

Fuel Containerized Control Plane upgradability performance testing

status

ready

version

1.0

Abstract

This test plan aims to provide set of tests to identify OpenStack performance against given containerized OpenStack cloud (installed on the top of pre-deployed Kubernetes cluster) using simple minimalistic set of Rally tests during upgrade from Mitaka to Newton.

Test Plan

This test plan covers basic network performance with long-running test suites to verify cloud network stability and performance during update Open Stack from Mitaka to Newton.

Test Environment

Preparation

This test plan is performed against existing OpenStack cloud installed on top of pre-deployed Kubernetes cluster with fuel-ccp tool with pre-installed Rally framework.

Environment description

The environment description includes hardware specification of servers, network parameters, operation system and OpenStack deployment characteristics.

Hardware

This section contains list of all types of hardware nodes (table below is an example).

Parameter	Value	Comments
model		e.g. Supermicro X9SRD-F
CPU		e.g. 6 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
role		e.g. compute or network

Network

This section contains list of interfaces and network parameters. For complicated cases this section may include topology diagram and switch parameters (table below is an example).

Parameter	Value	Comments
network role		e.g. provider or public
card model		e.g. Intel
driver		e.g. ixgbe
speed		e.g. 10G or 1G
MTU		e.g. 9000
offloading modes		e.g. default

Software

This section describes installed software (table below is an example).

Parameter	Value	Comments
OS		e.g. Ubuntu 14.04.3
OpenStack		e.g. Mitaka
Hypervisor		e.g. KVM
Neutron plugin		e.g. ML2 + OVS
L2 segmentation		e.g. VLAN / VxLAN / GRE
virtual routers		e.g. HA / DVR

Test tool

Rally is a benchmarking tool that was designed specifically for OpenStack API testing. To make this possible, Rally automates and unifies multi-node OpenStack deployment, cloud verification, benchmarking & profiling. This is a simple way to check cloud workability and performance of control plane operations running on it.

Test Case 1: Boot and delete server during Open Stack update

Description

The most user-facing control plane operation is new virtual machine creation. This scenario covers the most basic OpenStack server creation to present the baseline numbers for Nova (OpenStack Compute) control plane.

Parameters

Name	Description
IMAGE	Image from which boot server
FLAVOR	Flavor type from which boot server
ASSIGN_NIC	Bool, whether or not to auto assign NICs in Rally scenario
CONCURRENCY	Amount of parallel executors
ITERATIONS	Total amount of iterations processed by all executors

List of performance metrics

Priority	Value	Measurement Units	Description
1		sec	Time of atomic operations

Measuring performance values

1. Create server with FLAVOR flavor from IMAGE image through Nova API
2. Delete server through Nova API.

These 2 steps executed successively in CONCURRENCY parallel executors. ASSIGN_NIC parameter reflects Rally scenario configuration whether to assign NIC to the booted server in automatic fashion.

One cycle of these 2 steps is called an iteration. ITERATIONS is a total amount of iterations which was processed by executors.

During this testing

At the end of this test case you should calculate average, 90% percentile, 50% percentile, minimum and maximum for each step. You need to fill the following tables with calculated values:

Nova

Operation	Mean (sec)	90%ile (sec)	50%ile (sec)	Max (sec)	Min (sec)
create_server
delete_server

Example of Rally scenario configuration

test_plans/021-nova-boot-and-delete-baseline.yaml