diff --git a/README.rst b/README.rst index fe24de9..f267ad6 100644 --- a/README.rst +++ b/README.rst @@ -1,126 +1,23 @@ -======== -Overview -======== +===================== +KloudBuster version 6 +===================== -How good is your OpenStack data plane or storage plane under real heavy load? +How good is your OpenStack *data plane* or *storage service* under real heavy load? -KloudBuster is a tool that can load the data or storage plane of any OpenStack -cloud at massive scale and can measure how well the cloud behaves under load. +KloudBuster is a tool that can load the data plane or storage infrastructure of any OpenStack +cloud at massive scale and measure how well the cloud behaves under load +where it matters: from the VMs standpoint, where cloud applications run. -Anybody with basic knowledge of OpenStack, data plane and storage -performance concepts can use the tool and get scale numbers for any OpenStack -cloud straight off the box with pre-defined default workloads. +Accessible to anybody with basic knowledge of OpenStack, installs in minutes and runs +off the box with sensible default workloads in a fully automated way. +CLI/REST or Web User Interface.. you pick what works best for you. -No need for complex installation as the KloudBuster installation takes care of -all the dependencies. +Read the full documentation with feature list, snapshots and diagrams, +scale test design, how-to and installation instructions: -Features --------- +`KloudBuster Documentation `_ -* Neutron configuration agnostic (any encapsulation, any overlay, any plugin) - -* OpenStack Storage backend agnostic - -* User can specify any number of tenants, routers, networks, VM instances (only - limited by cloud capacity) and KloudBuster will stage all these resources in a - way that makes sense for operational usage - -* Real VM-level performance and scale numbers (not bare metal) - -* Punishing scale (thousands of VMs and enough load to fill even the fastest NIC - cards or load any storage cluster with ease - if your cloud can even support - that much) - -* Data plane with HTTP traffic load: - - * Can load the data plane with one OpenStack cloud (single-cloud operations - for L3 East-West scale) or 2 OpenStack clouds (dual-cloud operations with - one cloud hosting the HTTP servers and the other loading HTTP traffic for - L3 North-South scale testing) - - * Real HTTP servers (Nginx) running in real Linux images (Ubuntu 14.04) - - * Can specify any number of HTTP servers per tenant (as many as your cloud - can handle) - - * High performance and highly scalable HTTP traffic generators to simulate - huge number of HTTP users and TCP connections (hundreds of thousands to - millions of concurrent and active connections) - - * Overall throughput aggegation and loss-less latency aggregation for every - single HTTP request (typically millions per run) using the open source - HdrHistogram library - - * Traffic shaping to specify on which links traffic should flow - - * Can support periodic reporting and aggregation of results - -* Storage load: - - * VM-level Cinder volume (block storage) or Ephemeral disk file I/O performance measurement - using FIO running inside VMs (not bare metal) - - * Supports random and sequential file access mode - - * Supports read, write and read/write mix - - * IOPs, bandwitdh and loss-less latency aggregation - - * User configurable storage workload profiles - -* Supports automated scale progressions (VM count series in any multiple - increment) to reduce dramatically scale testing time - -* Highly efficient and scalable metric aggregation - -* Automatic cleanup upon termination (can be disabled) - -* Manual cleanup script - -* KloudBuster Server to drive scale test: - - * from any browser (KloudBuster Web UI) - - * from any external programs (KloudBuster REST API) - -* Aggregated results provide an easy to understand way to assess the scale of - the cloud under test - -* KloudBuster VM image pre-built and available from the OpenStack Community App - Catalog (https://apps.openstack.org/) - - -Limitations and Non-Goals -------------------------- - -* Requires Neutron networking (does not support Nova networking) - -* Only supports HTTP and storage traffic in this version - -Unlike some other scaling test frameworks, KloudBuster does *not* attempt to: - -* provide a scale test framework that works across different cloud technologies - (OpenStack + AWS + Google Cloud + ...) - we are only focusing on OpenStack - -* provide a scale test framework that is flexible and programmable to do everything - - we just focus on opinionated and well targeted performance and scale areas - with sensible use cases and available in a fully integrated and easy to consume - packaged format - -* replace bare metal and domain specific native performance and scale frameworks - (line level traffic generators, ceph specific performance and scale tools...) - - -Contributions and Feedbacks ---------------------------- - -If you are interested in OpenStack Performance and Scale, contributions and -feedbacks are welcome! - -If you have any feedbacks or would like to make small or large contributions, -simply send an email to openstack-dev@lists.openstack.org with a '[kloudbuster]' -tag in the subject. Licensing @@ -150,13 +47,3 @@ the FIO binary copy was not modified and can be found directly at ``_ or can be obtained by email request to the maintainer of KloudBuster. - -Links ------ - -* Documentation: ``_ -* `KloudBuster REST API documentation Preview `_ -* Source: ``_ -* Supports/Bugs: ``_ -* Mailing List: kloudbuster-core@lists.launchpad.net - diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst index b8d80a7..3f4afd7 100644 --- a/doc/source/architecture.rst +++ b/doc/source/architecture.rst @@ -1,4 +1,5 @@ -======================== +.. _arch: + KloudBuster Architecture ======================== @@ -20,7 +21,7 @@ that run inside the same cloud. Such traffic can involve: (that is the packets do not have to go through a router) - often called L2 East-West -- Packet routing if the 2 end points belong to differet Neutron networks +- Packet routing if the 2 end points belong to different Neutron networks (packets have to go through router) - often called L3 East-West The KloudBuster data plane scale test exercises L3 East-West traffic by running @@ -32,7 +33,7 @@ corresponding server router as illustrated in the following diagram: .. image:: images/kb-http-east-west.png The KloudBuster App typically runs outside the cloud under test on any server -that has a python interpreter (Macbook, Linux workstation...) with the +that has a python interpreter (MacBook, Linux workstation...) with the requirement to have access to the OpenStack API of the cloud under test. The KloudBuster app basically reads the requested scale config (which contains @@ -60,7 +61,7 @@ Rack to Rack Data Plane Scale ----------------------------- By default KloudBuster will rely on the Nova scheduler to place the various -cient and server VMs. As a result these VMs will be load balanced across all +client and server VMs. As a result these VMs will be load balanced across all servers and causing the data path of the HTTP traffic to be quite random. This can be good to measure the scale on a random traffic pattern but sometimes it is more interesting to shape the HTTP traffic can be shaped to follow certain @@ -83,12 +84,12 @@ North South Data Plane Scale Test --------------------------------- The North South traffic refers to traffic flowing between external sources and -VMs runnin in the cloud. Such traffic follows a very different path than +VMs running in the cloud. Such traffic follows a very different path than East-West traffic as it is generally always routed and requires the used of IP address translation (SNAT and DNAT). One exception to this is the use of a provider network which may avoid routing and NAT completely. -KloudBuster provides a option to test the North-South data plabe traffic by +KloudBuster provides a option to test the North-South data plane traffic by separating the client VMs and server VMs into 2 different OpenStack clouds. .. image:: images/kb-http-north-south.png @@ -101,12 +102,20 @@ Storage Scale Test ++++++++++++++++++ The storage scale test is a relatively simpler version of the data plane scale -test as it only involves 1 tenant 1 network and 1 router. Each test VM runs an -instance of the FIO test client (`FIO `_ is a -popular open source storage test client). +test as it only involves 1 tenant, 1 network and 1 router. Each test VM runs one +instance of the FIO test client (`FIO `_ which is a +widely adopted open source storage test client). .. image:: images/kb-storage.png +VM staging, storage plumbing (using Cinder or Nova for ephemeral disks) is done +by the KloudBuster app using OpenStack APIs. +Because the Cinder API abstracts the storage back-end, it is agnostic of +the technology used (Ceph, EMC...). +After the test, all resources (volumes, VMs, network, router) are cleaned up in +the proper order. + + Progression Runs ++++++++++++++++ @@ -118,7 +127,7 @@ Progression runs are a very convenient feature as it allows to produce result for series in a much shorter time by reusing the same set of staged VMs and iterating the scale test to produce measurements at different scale level. -For example, to get storage performace measurement for 100 to 1000 VMs in +For example, to get storage performance measurement for 100 to 1000 VMs in increments of 100, would require staging and unstaging 100+200+300+....+1,000 = 5,500 VM Instances without progression runs while it would only require staging 1,000 instances with VM reuse. diff --git a/doc/source/cleanup.rst b/doc/source/cleanup.rst index 0ea163f..80493c9 100644 --- a/doc/source/cleanup.rst +++ b/doc/source/cleanup.rst @@ -1,4 +1,5 @@ -=========================== +.. _cleanup: + OpenStack Resources Cleanup =========================== @@ -85,8 +86,8 @@ Example of cleanup log file:: If such file is provided to the cleanup script using the *--file* option, only the resources described in the file will be deleted. -Discoverty with Resource name filter (--filter ) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Discovery with Resource name filter (--filter ) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If no cleanup log file is provided, resources are discovered from OpenStack and selected using a regular expression on the resource name (--filter ). You diff --git a/doc/source/gallery.rst b/doc/source/gallery.rst new file mode 100644 index 0000000..f83eb52 --- /dev/null +++ b/doc/source/gallery.rst @@ -0,0 +1,71 @@ + +.. _gallery: + + +Gallery +======= + +This page has links to examples of scale test reports in HTML format generated by KloudBuster. +These reports were generated within minutes after starting the scale test from a +bare bone OpenStack cloud (not running anything). +Click on the thumbnail images to view the result HTML file in your browser (you will need +access to the Internet to view these files as they reference multiple Java script libraries in CDN). + +Sample HTTP Scale Report +------------------------ + +The following report shows an HTTP scale run with results for 1 to 20 HTTP servers +(running in as many server VMs) in increment of 5, where each HTTP server is receiving +HTTP requests from 1 HTTP traffic generator that runs in a separate VM and emulates 1,000 +users sending 1 request per second each (for a total of 1000 requests per second per HTTP server). +The topology used for the test is 1 tenant, 1 router, 4 networks and 5 HTTP servers per network. +Each iteration is programmed to run for 30 seconds. +This scale settings can be viewed in the Configuration tab. + +The table shows the results for each iteration step, with the requested and measured RPS +(HTTP requests per second) and the corresponding aggregated download throughput (the sum of all +downloads for all clients). + +The latency distribution is shows in the chart, where each line corresponds to one load level (or +iteration in the progression). Lines can be individually shown/hidden by clicking on the +corresponding legend item. + +For example, the largest scale involves 20,000 simultaneous users sending an aggregate of 18,621 +HTTP requests per second and the latency chart tells us that 99.9% of these 18,621 requests are +replied within 34ms, which is actually excellent. + +Note that this test is configured to reuse HTTP connections meaning that we do not have the +overhead of creating a new TCP connection for every HTTP request. This also means that this cloud +had 20,000 TCP active connections at all times during the scale test. + + + +.. image:: images/kb-http-thumbnail.png + :target: https://htmlpreview.github.io/?https://github.com/openstack/kloudbuster/blob/master/doc/source/gallery/http.html + + +Sample Storage Scale Report +--------------------------- + +This is a report for a storage scale test using the default workload suite with a progression run +from 1 VM to 20 VMs in increment of 5, and 30 second run per iteration. +This results in 6 tabs of results (1 per workload). The mixed read/write tabs further split in +2 sub tabs (1 for read and 1 for write results). + +The random read tab shows that each VM could achieve its requested 100 IOPs across the progression. +The lines represent the latency value at given percentile and can be individually shown/hidden by clicking +the corresponding legend item. +As an example, 20 VMs represents a combined 2,000 IOPs measured for a total of 60,000 random read operations. +The latency line tells us that 99.9% of these 60,000 read operations are completed within 1.576 msec. + + + +.. image:: images/kb-storage-thumbnail.png + :target: https://htmlpreview.github.io/?https://github.com/openstack/kloudbuster/blob/master/doc/source/gallery/storage.html + +The sequential write results are more challenging as they show that the VMs cannot achieve their requested +write bandwidth (60MB/s) and can only get 49MB/s each when there are 20 of such VMs. +The latency lines also reflect that stress by peaking at 500 ms for 99.99% of all write operations (although +latency is not nearly as critical for sequential access than for random access). + + diff --git a/doc/source/gallery/http.html b/doc/source/gallery/http.html new file mode 100644 index 0000000..240cacb --- /dev/null +++ b/doc/source/gallery/http.html @@ -0,0 +1,230 @@ + + + + + + + + + + + + + + + + + + + + KloudBuster Report + + + + + + + + + + + + + + + + +
+

KloudBuster HTTP Test Report

+
Latency(ms)
+
+ +
+ + + + + + + + + + + +
+ + {{row.connection}}{{row.server_vms}}{{row.requests}}{{row.sock_err}}{{row.rps}}{{row.rate_limit}}{{row.throughput}} Gbps
+
+ +
+

KloudBuster HTTP Test Configuration

+
+
+ +
+
+
+ + + + + + +
+
+
{{from_outside["time"]}} - KloudBuster {{from_outside["version"]}}
+
+ + + + diff --git a/doc/source/gallery/storage.html b/doc/source/gallery/storage.html new file mode 100644 index 0000000..5d393a7 --- /dev/null +++ b/doc/source/gallery/storage.html @@ -0,0 +1,450 @@ + + + + + + + + + + + + + + + + + + + + KloudBuster Report + + + + + + + + + + + + + + + + +
+

KloudBuster Storage Test Report

+ + + + +

{{current_mode_description}}

+
{{modes[current_mode_name]["title"]}} {{current_title}} ({{storage_target[from_outside.storage_target]}})
+ +
+
{{current_mode["y_axis"]}}Latency(ms)
+ +
+ + + + + + + + + + + + +
{{row.total_client_vms}}{{row.block_size}}b{{row.iodepth}} + {{row.rate_iops}} + + {{row.read_iops}} + + {{row.write_iops}} + + {{row.rate}} KB/s + + {{row.read_bw}} KB/s + + {{row.write_bw}} KB/s +
+
+ +
+

KloudBuster HTTP Test Configuration

+
+
+ +
+
+
+ + +
+
+
{{from_outside["time"]}} - KloudBuster {{from_outside["version"]}}
+
+ + + + diff --git a/doc/source/images/kb-http-thumbnail.png b/doc/source/images/kb-http-thumbnail.png new file mode 100644 index 0000000..73eecef Binary files /dev/null and b/doc/source/images/kb-http-thumbnail.png differ diff --git a/doc/source/images/kb-storage-thumbnail.png b/doc/source/images/kb-storage-thumbnail.png new file mode 100644 index 0000000..8a7ae1e Binary files /dev/null and b/doc/source/images/kb-storage-thumbnail.png differ diff --git a/doc/source/index.rst b/doc/source/index.rst index f1d2321..3228bc5 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -13,6 +13,7 @@ Contents: readme architecture + gallery installation usage cleanup diff --git a/doc/source/installation.rst b/doc/source/installation.rst index 394ea22..39690f7 100644 --- a/doc/source/installation.rst +++ b/doc/source/installation.rst @@ -3,31 +3,41 @@ Installation ============ KloudBuster is already pre-installed as a web service in the KloudBuster VM image -available from the `OpenStack Community App Catalog `_ +available from the `OpenStack Community App Catalog `_. +So if you just need to use the KloudBuster Web user interface you can instantiate +that VM image and point your browser to its public address as described in :ref:`run_server`. There are two alternative ways to install and run KloudBuster tool. -Users of KloudBuster who prefers to use the CLI should use regular PyPI based installation. -Developers of KloudBuster should use the GitHub/OpenStack Repository based installation. + +Users of KloudBuster who prefers to use the CLI or who prefer to run KloudBuster +locally on their workstation or laptop should use the PyPI based installation +(pip install). + +Developers of KloudBuster should use the GitHub/OpenStack Repository based installation +(git clone). Web Service and PyPI based installation will satisfy most use cases -and are the 2 recommended ways for running KloudBuster under production environments, or through an -automated or scheduled job. -The git repository based installation is targeted at developers of KloudBuster. +and are the 2 recommended ways for running KloudBuster under production environments, +or through an automated or scheduled job. .. note:: Installation from PyPI will only have the latest stable version. PyPI based Installation ----------------------- -This is the recommended way to install KloudBuster for non-development use if CLI is required. -KloudBuster is available in the Python Package Index (PyPI): -`KloudBuster PyPI `_ +This is the recommended way to install KloudBuster for non-development use if CLI is preferred +or if you prefer to run KloudBuster locally. -Step 1 -^^^^^^ +KloudBuster is available in the Python Package Index (PyPI) +`KloudBuster PyPI `_ +and can be installed on any system that has python 2.7. + +Step 1: Install pip and the python virtualenv (if not installed already) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You will need to have python 2.7, pip, and some dependencies installed -before installing KloudBuster, run the command based on your distro. +before installing KloudBuster depending on the operating system at the installation site. +These pre-requisites can be skipped if the corresponding dependencies are already installed. Ubuntu/Debian based: @@ -50,8 +60,8 @@ MacOSX: $ sudo easy_install pip $ sudo pip install virtualenv -Step 2 -^^^^^^ +Step 2: Install KloudBuster in a virtual environment +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Create a virtual environment for Python, and install KloudBuster: @@ -92,8 +102,8 @@ It is recommended to run KloudBuster inside a virtual environment. However, it can be skipped if installed in a dedicated VM. -Super quick installation on Ubuntu/Debian -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Quick installation on Ubuntu/Debian +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: bash @@ -105,8 +115,8 @@ Super quick installation on Ubuntu/Debian $ cd kloudbuster $ pip install -r requirements-dev.txt -Super quick installation on RHEL/Fedora/CentOS -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Quick installation on RHEL/Fedora/CentOS +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: bash @@ -118,11 +128,10 @@ Super quick installation on RHEL/Fedora/CentOS $ cd kloudbuster $ pip install -r requirements-dev.txt -Super quick installation on MacOSX -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Quick installation on MacOSX +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -KloudBuster can run natively on MacOSX. These instructions have been verified -to work on MacOSX 10.10 (Yosemite). +KloudBuster can run natively on MacOSX. First, download XCode from App Store, then execute below commands: @@ -157,9 +166,10 @@ Upload KloudBuster Image .. note:: - If your OpenStack Glance is able to access the Internet, and you don't - need to access the KloudBuster Web UI from the pre-built image, you can skip - this section and you are done with the installation. + If your OpenStack Glance is able to access the Internet and you only use + the CLI to launch KloudBuster, you can skip this section (KloudBuster CLI + will request Glance to download the image from the OpenStack App Catalog when + it is not present in Glance). In the cloud under test, KloudBuster needs one "universal" test VM image (referred to as "KloudBuster image") that contains the necessary test software. @@ -169,10 +179,7 @@ application using the appropriate role (HTTP server, HTTP traffic generator, etc.). Pre-built images are available for download from the -`OpenStack App Catalog `_ (preferred method). For -whatever reason the pre-built version doesn't work for you, the image can be -re-built from MacOSX using Vagrant or from any Linux server. See -:ref:`here ` for more details. +`OpenStack App Catalog `_ (preferred method). .. note:: @@ -186,7 +193,8 @@ Manual upload of the KloudBuster VM image In order to upload the KloudBuster Image to the cloud under test, the image must be downloaded from the OpenStack App Catalog either directly from the OpenStack App Catalog (if you have direct access to the Internet) -or through an intermediate location such as a jump host (a jump host has access +or through an intermediate location such as a laptop that has Internet access +or a jump host (a jump host has access to both Internet and the cloud under test and can be used to download the image from the App Catalog and upload to Glance using either a Glance CLI command or via Horizon @@ -199,7 +207,7 @@ with the "kloudbuster_v" prefix and download the latest version from the list. The name of the image in Glance must match exactly the image name in the App Catalog (without the .qcow2 extension), for example to upload the image from -a local copy of that image: +a local copy of that image using the Glance CLI: .. code-block:: bash diff --git a/doc/source/readme.rst b/doc/source/readme.rst index a6210d3..4bb30e5 100644 --- a/doc/source/readme.rst +++ b/doc/source/readme.rst @@ -1 +1,164 @@ -.. include:: ../../README.rst +===================== +KloudBuster version 6 +===================== + +How good is your OpenStack *data plane* or *storage service* under real heavy load? + +KloudBuster is a tool that can load the data plane or storage infrastructure of any OpenStack +cloud at massive scale and measure how well the cloud behaves under load +where it matters: from the VMs standpoint, where cloud applications run. + +Accessible to anybody with basic knowledge of OpenStack, installs in minutes and runs +off the box with sensible default workloads in a fully automated way. +CLI/REST or Web User Interface.. you pick what works best for you. + + +Feature List +------------ + +* Neutron configuration agnostic (any encapsulation, any overlay, any plugin) + +* OpenStack Storage backend agnostic + +* Real VM-level performance and scale numbers (not bare metal) + +* Punishing scale (thousands of VMs and enough load to fill even the fastest NIC + cards or load any storage cluster with ease) + +* Data plane with HTTP traffic load: + + * Can load the data plane with one OpenStack cloud (single-cloud operations + for L3 East-West scale) or 2 OpenStack clouds (dual-cloud operations with + one cloud hosting the HTTP servers and the other loading HTTP traffic for + L3 North-South scale testing) + + * Real HTTP servers (Nginx) running in real Linux images (Ubuntu 14.04) + + * Can specify any number of tenants, routers and networks + + * Can specify any number of HTTP servers per tenant (as many as your cloud + can handle) + + * High performance and highly scalable HTTP traffic generators to simulate + huge number of HTTP users and TCP connections (hundreds of thousands to + millions of concurrent and active connections) + + * Overall throughput aggegation and loss-less millisecond-precision latency + aggregation for every single HTTP request (typically millions per run) + + * Traffic shaping to specify on which links traffic should flow + + * Can support periodic reporting and aggregation of results + +* Storage load: + + * VM-level Cinder volume (block storage) or Ephemeral disk file I/O performance measurement + using FIO running inside VMs (not bare metal) + + * Supports random and sequential file access mode + + * Supports any mix of read/write + + * IOPs, bandwitdh and loss-less millisecond-precision latency aggregation for every + IO operation (typically millions per run) + + * User configurable workload sequence + +* Supports automated scale progressions (VM count series in any multiple + increment) to reduce dramatically scale testing time + +* Highly efficient and scalable metric aggregation + +* Automatic cleanup upon termination + +* Regular expression based cleanup script (`cleanup`_) + +* KloudBuster server mode to drive scale test: + + * from any browser (KloudBuster Web UI) + + * or from any external programs (KloudBuster REST API) + +* Aggregated results provide an easy to understand way to assess the scale of + the cloud under test + +* KloudBuster VM image pre-built and available from the OpenStack Community App + Catalog (https://apps.openstack.org/) + +*Diagrams* describing how the scale test resources are staged and how the traffic flows are available +in :ref:`arch`. + +Scale results are available in json form or in html form with javascript graphical charts generated straight off the tool. + +*Examples of results* are available in :ref:`gallery`. + + +Limitations and Non-Goals +------------------------- + +* Requires Neutron networking (does not support Nova networking) + +* Only supports HTTP and storage traffic in this version + +Unlike some other scaling test frameworks, KloudBuster does *not* attempt to: + +* provide a scale test framework that works across different cloud technologies + (OpenStack + AWS + Google Cloud + ...) - we are only focusing on OpenStack + +* provide a scale test framework that is flexible and programmable to do everything - + we just focus on opinionated and well targeted performance and scale areas + with sensible use cases and available in a fully integrated and easy to consume + packaged format + +* replace bare metal and domain specific native performance and scale frameworks + (line level traffic generators, ceph specific performance and scale tools...) + + +Contributions and Feedbacks +--------------------------- + +If you are interested in OpenStack Performance and Scale, contributions and +feedbacks are welcome! + +If you have any feedbacks or would like to make small or large contributions, +simply send an email to openstack-dev@lists.openstack.org with a '[kloudbuster]' +tag in the subject. + + +Licensing +--------- + +KloudBuster is licensed under the Apache License, Version 2.0 (the "License"). +You may not use this tool except in compliance with the License. You may obtain +a copy of the License at ``_ + +Unless required by applicable law or agreed to in writing, software distributed +under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR +CONDITIONS OF ANY KIND, either express or implied. See the License for the +specific language governing permissions and limitations under the License. + +KloudBuster VM images contain multiple open source license components: + +* nginx: BSD License (http://nginx.org/LICENSE) +* wrk2: Apache License 2.0 + (https://raw.githubusercontent.com/giltene/wrk2/master/LICENSE) +* Redis: BSD License (http://redis.io/topics/license) +* FIO: GPL v2 (https://raw.githubusercontent.com/axboe/fio/master/MORAL-LICENSE) + +Although the VM image includes a binary copy of the FIO code, it does not +include the source code used to build it. In accordance to the GPL V2 license +related to the inclusion of binary copies of FIO, the source code used to build +the FIO binary copy was not modified and can be found directly at +``_ or can be obtained by email request to the +maintainer of KloudBuster. + + +Links +----- + +* Complete documentation: ``_ +* `KloudBuster REST API documentation Preview `_ +* Source: ``_ +* Supports/Bugs: ``_ +* Mailing List: kloudbuster-core@lists.launchpad.net + diff --git a/doc/source/usage.rst b/doc/source/usage.rst index 7c62b1b..2885e60 100644 --- a/doc/source/usage.rst +++ b/doc/source/usage.rst @@ -16,14 +16,15 @@ run KloudBuster: * Admin access to the cloud under test (non-admin might work with some tweaks and limitations) - * 3 available floating IPs if running HTTP data plane testing, 2 available - floating IPs if running Storage testing + * 3 available floating IPs if running the HTTP data plane scale test, + * 2 available floating IPs if running the Storage scale test Regardless of the way you launch KloudBuster, you will need the access info and the credentials to the cloud under test. This information can be downloaded -from a Horizon dashboard (Project|Access&Security|Api Access|Download OpenStack -RC File). Save it to your local filesystem for future use. +from the Horizon dashboard (Project|Access&Security|Api Access|Download OpenStack +RC File). Save it to your local file system for future use. +.. _run_server: Running KloudBuster as a Web/REST Server ---------------------------------------- @@ -37,19 +38,19 @@ and is ready to service HTTP and REST requests once up and running. To get the KloudBuster Web server running in any OpenStack cloud: 1. Follow the steps :ref:`here ` to upload the KloudBuster - image to the openstack cloud that will host your KloudBuster web server + image to the OpenStack cloud that will host your KloudBuster web server .. note:: This could be the same as the cloud under test or a different cloud. -2. If necessary, and as for any VM-based web server application bringup, create +2. If necessary, and as for any VM-based web server application bring up, create and configure the Neutron router and network where the KloudBuster web server VM instance will be attached 3. Create or reuse a security group which allows ingress TCP traffic on port 8080 -4. Launch an instance using the KloudBuster image，with the proper security +4. Launch an instance using the KloudBuster image with the proper security group, and connect to the appropriate network. Leave the Key Pair as blank, as we don't need the SSH access to this VM @@ -82,13 +83,13 @@ is up running:: Starting the KloudBuster Server from a git clone ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -If you use git clone, you can bring up the KloudBuster Web/REST server fron the +If you use git clone, you can bring up the KloudBuster Web/REST server from the CLI. KloudBuster uses the `Pecan `_ web server to host both the KloudBuster REST server and the KloudBuster front-end website (which listens to port 8080 by default). From the root of the KloudBuster repository, go to the kb_server directory. If -this is the first time to start the server, run below command once to setup the +this is the first time you start the server, run the below command once to setup the environment:: $ python setup.py develop @@ -111,8 +112,8 @@ Using any browser, point to the provided URL at port 8080. You will get a Login page where you will need to enter: * The type of scale test (HTTP data plane or storage) - * The location of openrc file of the cloud under test - * The credentials for the cloud under test + * The location of the openrc file of the cloud under test + * The password for the cloud under test Interacting with the KloudBuster Server REST Interface @@ -133,8 +134,9 @@ If you do not really need a Web UI or REST interface, you can simply run KloudBuster scale test straight from CLI. KloudBuster is ready to run with the default configuration, which can be -displayed from the command line using *--show-config* option. By default, -KloudBuster will run on a single cloud and run the default HTTP data plane scale +displayed from the command line using *--show-config* option. + +By default, KloudBuster will run on a single cloud and run the default HTTP data plane scale test: * Create 2 tenants, 2 users, and 2 routers; @@ -143,19 +145,41 @@ test: * Create 1 VM running the Redis server (for orchestration) * Create 1 VM running the HTTP traffic generator (default to 1000 connections, 1000 requests per second, and 30 seconds duration - * Measure/aggegate throughput and latency + * Measure/aggregate throughput and latency * Bring down and cleanup Run KloudBuster with the following options:: kloudbuster --tested-rc --tested-passwd -.. note:: +If you installed KloudBuster using git clone you need to explicitly call the python interpreter +since the "kloudbuster" wrapper is only installed with pip install):: - Simply adding *--storage* to the above command will run KloudBuster with - storage testing. + python kloudbuster.py --tested-rc --tested-passwd -The run should take couple of minutes (depending on how fast the cloud can + +If you want to run the default storage scale test, simply adding *--storage*:: + + kloudbuster --storage --tested-rc --tested-passwd + +The default storage scale test will use the following settings: + * Create 1 tenant + * Create 1 router + * Create 1 private network + * Create 1 VM and attach a 10 GB Cinder volume to it + * Perform the default storage workload sequence + * random access 4KB block size, IO depth 4, 100 IOPs for 30 seconds each + * 100% read + * 100% write + * 70% read, 30% write + * sequential access 64KB block size, IO depth 64, 60 MB/sec for 30 seconds + * 100% read + * 100% write + * 70% read, 30% write + * Measure/aggregate IOPs, throughput and latency + * Bring down and cleanup + +The run should take a few minutes (depending on how fast the cloud can create the resources) and you should see the actions taken by KloudBuster displayed on the console. Once this minimal scale test passes, you can tackle more elaborate scale testing by increasing the scale numbers or providing @@ -169,12 +193,16 @@ KloudBuster Configuration To create a custom scale test configuration, make a copy of the default configuration and modify that file to satisfy our own needs. A copy of the default configuration can be obtained by redirecting the output of -*--show-config* to a new file. Once done, provide that custom configuration +*--show-config* to a new file. For example: + + kloudbuster --show-config >scale1.cfg + +Once done, provide that custom configuration file to the KloudBuster command line using the *--config * option. .. note:: - Note that the default configuration is always loaded by KloudBuster and + The default configuration is always loaded by KloudBuster and any default option can be overridden by providing a custom configuration file that only contains modified options. So you can delete all the lines in the configuration file that you do not intend to change @@ -185,7 +213,7 @@ General Options Each item in cfg.scale.yaml is well documented and self-explained. Below is just a quick-start on some important config items that need to be paid more -attention. +attention to. * **vm_creation_concurrency** @@ -258,7 +286,7 @@ In the case of Storage testing: capacity of storage is reached, the overall performance will start to degrade. - e.g. In the randread and randwrite mode, for example the IOPS is limited to + e.g. In the random read and random write mode, for example the IOPS is limited to 100 IOPS/VM. In the iteration of 10 VMs, the requested IOPS for the system is 100 * 10 = 1000. However, the measured IOPS is degraded to only 800 IOPS. So the degraded percentile is calculated as 800/1000=20% for this set of @@ -395,14 +423,14 @@ command line. Displaying the Results ^^^^^^^^^^^^^^^^^^^^^^ -Results can be saved in a file in json format or in html format. The json format +Results can be saved in a file in json format or in HTML format. The json format is more appropriate for usage by any post-processing tool or script while the -html file is more adapted for human usage. +HTML file is more adapted for human usage. The KloudBuster Web UI will display the results using charts and tables when the test is finished running. The KloudBuster CLI provides an option to generate -the html file from the results (*--html* option). It can also generate the html -file from the json results (*--charts-from-json* option). +the HTML file from the results (*--html* option). It can also generate the HTML +file from the JSON results (*--charts-from-json* option). Examples of running KloudBuster diff --git a/kloudbuster/cfg.scale.yaml b/kloudbuster/cfg.scale.yaml index 643e445..43ceba3 100644 --- a/kloudbuster/cfg.scale.yaml +++ b/kloudbuster/cfg.scale.yaml @@ -43,9 +43,9 @@ vm_creation_concurrency: 5 # example to debug) public_key_file: -# ========================== -# SERVER SIDE CONFIG OPTIONS -# ========================== +# ================================================== +# SERVER SIDE CONFIG OPTIONS (HTTP SERVER SIDE ONLY) +# ================================================== server: # Flavor to use for the test images flavor: @@ -93,9 +93,9 @@ server: html_size: 32768 -# ========================== -# CLIENT SIDE CONFIG OPTIONS -# ========================== +# ==================================================== +# CLIENT SIDE CONFIG OPTIONS (HTTP CLIENT AND STORAGE) +# ==================================================== client: # Flavor to use for the test images flavor: @@ -136,6 +136,8 @@ client: # {'vm_start': 0, 'vm_multiple': 1}: 1, 2, 3, 4, 5... # {'vm_start': 0, 'vm_multiple': 5}: 5, 10, 15, 20... vm_multiple: 2 + + # HTTP ONLY # The stop condition for HTTP benchmarking, it is used for KloudBuster to # determine when to stop the progression, and do the cleanup if needed. # It defines as: @@ -154,6 +156,8 @@ client: # [50, 75, 90, 99, 99.9, 99.99, 99.999] # (3) Sets percentile to 0 to disable HTTP request timeout checks; http_stop_limit: [50, 0] + + # STORAGE ONLY # The stop condition for storage benchmarking, it is used for KloudBuster # to determine when to stop the progression, and do the cleanup if needed. # In the mode of random read and random write, this value indicates the