Cleanup logic enhancements, document enhancements
1. Enhance the cleanup logic; 2. Add the missing keypair into the resource logger; 3. Document enhancements on installation and contribution; 4. Fix the bug when parsing the image version string in DIB; 5. Fix the typo in Rest API specification; Change-Id: Ibf4a1c7828ee4522938b4b55f9a8af5e6c6a133b
This commit is contained in:
Yichen Wang
parent
525010a2d5
commit
d7d9575dda
@ -1,23 +0,0 @@
|
||||
|
||||
|
||||
Feedbacks and contributions to KloudBuster are welcome.
|
||||
|
||||
KloudBuster follows the same workflow as any other OpenStack project.
|
||||
|
||||
If you would like to contribute to the development of OpenStack, you must
|
||||
follow the steps in this page:
|
||||
|
||||
`<http://docs.openstack.org/infra/manual/developers.html>`_
|
||||
|
||||
If you already have a good understanding of how the system works and your
|
||||
OpenStack accounts are set up, you can skip to the development workflow
|
||||
section of this documentation to learn how changes to OpenStack should be
|
||||
submitted for review via the Gerrit tool:
|
||||
|
||||
`<http://docs.openstack.org/infra/manual/developers.html#development-workflow>`_
|
||||
|
||||
Pull requests submitted through GitHub will be ignored.
|
||||
|
||||
Feedbacks and Bugs should be filed on Launchpad, not GitHub:
|
||||
|
||||
`<https://bugs.launchpad.net/kloudbuster>`_
|
||||
23
README.rst
23
README.rst
@ -1,6 +1,6 @@
|
||||
===============================
|
||||
KloudBuster
|
||||
===============================
|
||||
========
|
||||
Overview
|
||||
========
|
||||
|
||||
How good is your OpenStack data plane under real heavy load?
|
||||
|
||||
@ -50,15 +50,9 @@ If you have any feedbacks or would like to make small or large contributions, si
|
||||
openstack-dev@lists.openstack.org with a '[kloudbuster]' tag in the subject.
|
||||
|
||||
|
||||
Links
|
||||
-----
|
||||
|
||||
* Source: `<http://git.openstack.org/cgit/openstack/kloudbuster>`_
|
||||
* Bugs: `<http://bugs.launchpad.net/kloudbuster>`_
|
||||
* Documentation: `<http://kloudbuster.readthedocs.org>`_
|
||||
|
||||
Licensing
|
||||
---------
|
||||
|
||||
KloudBuster is licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at `<http://www.apache.org/licenses/LICENSE-2.0>`_
|
||||
@ -71,3 +65,12 @@ limitations under the License.
|
||||
|
||||
KloudBuster VM images contain multi-open source license components
|
||||
|
||||
|
||||
Links
|
||||
-----
|
||||
|
||||
* Documentation: `<http://kloudbuster.readthedocs.org>`_
|
||||
* Source: `<http://git.openstack.org/cgit/openstack/kloudbuster>`_
|
||||
* Supports/Bugs: `<http://launchpad.net/kloudbuster>`_
|
||||
* Mailing List: kloudbuster-core@lists.launchpad.net
|
||||
|
||||
|
||||
@ -1,4 +1,93 @@
|
||||
============
|
||||
Contributing
|
||||
============
|
||||
.. include:: ../../CONTRIBUTING.rst
|
||||
|
||||
Contribute to KloudBuster
|
||||
-------------------------
|
||||
|
||||
Below are a simplified version of the workflow to work on KloudBuster. For complete instructions, you have to follow the Developer's Guide in OpenStack official documents. Refer to :ref:`below section <developer_guide_of_openstack>` for links.
|
||||
|
||||
|
||||
Start working
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
Before starting, a GitHub/OpenStack respository based installation must be done. Refer :ref:`here <git_installation>` for detailed documentation.
|
||||
|
||||
1. From the root of your workspace, check out a new branch to work on::
|
||||
|
||||
$ git checkout -b <TOPIC-BRANCH>
|
||||
|
||||
2. Happy working on your code for features or bugfixes;
|
||||
|
||||
|
||||
Before Commit
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
There are some criteria that are enforced to commit to KloudBuster. Below commands will perform the check and make sure your code complys with it.
|
||||
|
||||
3. PEP 8::
|
||||
|
||||
$ tox -epep8
|
||||
|
||||
.. note:: The first run usually takes longer, as tox will create a new virtual environment and download all dependencies. Once that is the done, further run will be very fast.
|
||||
|
||||
4. Run the test suite::
|
||||
|
||||
$ tox -epython27
|
||||
|
||||
5. If you made a documentation change (i.e. changes to .rst files), make sure the documentation is built as you expected::
|
||||
|
||||
$ cd <kloudbuster-ws-root>/doc
|
||||
$ make html
|
||||
|
||||
Once finished, the documentation in HTML format will be ready at <kloudbuster-ws-root>/doc/build/html.
|
||||
|
||||
|
||||
Submit Review
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
6. Commit the code::
|
||||
|
||||
$ git commit -a
|
||||
|
||||
.. note:: For a feature commit, please supply a clear commit message indicating what the feature is; for a bugfix commit, please also containing a launchpad link to the bug you are working on.
|
||||
|
||||
7. Submit the review::
|
||||
|
||||
$ git review <TOPIC-BRANCH>
|
||||
|
||||
The members in the KloudBuster team will get notified once the Jenkin verification is passed. So watch your email from the review site, as it will contain the updates for your submission.
|
||||
|
||||
8. If the code is approved with a +2 review, Gerrit will automatically merge your code.
|
||||
|
||||
|
||||
File Bugs
|
||||
---------
|
||||
|
||||
Bugs should be filed on Launchpad, not GitHub:
|
||||
|
||||
https://bugs.launchpad.net/kloudbuster
|
||||
|
||||
|
||||
.. _developer_guide_of_openstack:
|
||||
|
||||
Developer's Guide of OpenStack
|
||||
------------------------------
|
||||
|
||||
Feedbacks and contributions to KloudBuster are welcome.
|
||||
|
||||
KloudBuster follows the same workflow as any other OpenStack project.
|
||||
|
||||
If you would like to contribute to the development of OpenStack, you must
|
||||
follow the steps in this page:
|
||||
|
||||
`<http://docs.openstack.org/infra/manual/developers.html>`_
|
||||
|
||||
If you already have a good understanding of how the system works and your
|
||||
OpenStack accounts are set up, you can skip to the development workflow
|
||||
section of this documentation to learn how changes to OpenStack should be
|
||||
submitted for review via the Gerrit tool:
|
||||
|
||||
`<http://docs.openstack.org/infra/manual/developers.html#development-workflow>`_
|
||||
|
||||
Pull requests submitted through GitHub will be ignored.
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
contain the root `toctree` directive.
|
||||
|
||||
Welcome to the KloudBuster documentation!
|
||||
========================================================
|
||||
=========================================
|
||||
|
||||
Contents:
|
||||
|
||||
|
||||
@ -2,153 +2,243 @@
|
||||
Installation
|
||||
============
|
||||
|
||||
Binary Installation
|
||||
-------------------
|
||||
There are two ways to install and run KloudBuster tool. Users of KloudBuster should use regular PyPI based installation, while developers of KloudBuster should use GitHub/OpenStack Repository based installation. Normally, PyPI based installation will satisfy most of use cases, and it is the recommended way for running KloudBuster under production environments, or through an automated or scheduled job. A git repository based installation gives more flexibility, and it is a must for developers of KloudBuster.
|
||||
|
||||
This is the recommended way to install KloudBuster for non-development use.
|
||||
KloudBuster is available in the Python Package Index (PyPI)::
|
||||
.. note:: Installation from PyPI will only have the latest stable version.
|
||||
|
||||
`KloudBuster PyPI <https://pypi.python.org/pypi/KloudBuster>`_
|
||||
PyPI based Installation
|
||||
-----------------------
|
||||
|
||||
You will need to have python 2.7 and pip installed before installing KloudBuster.
|
||||
This is the recommended way to install KloudBuster for non-development use, and KloudBuster is available in the Python Package Index (PyPI): `KloudBuster PyPI <https://pypi.python.org/pypi/KloudBuster>`_
|
||||
|
||||
At the command line::
|
||||
Step 1
|
||||
^^^^^^
|
||||
|
||||
$ pip install kloudbuster
|
||||
You will need to have python 2.7, pip, and some dependencies installed before installing KloudBuster, run the command based on your distro.
|
||||
|
||||
Or, if you have `virtualenv <https://pypi.python.org/pypi/virtualenv>`_ installed::
|
||||
Ubuntu/Debian based:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo apt-get install python-dev python-pip python-virtualenv
|
||||
$ sudo apt-get install libxml2-dev libxslt-dev libffi-dev libz-dev libyaml-dev libssl-dev
|
||||
|
||||
RHEL/Fedora/CentOS based:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo yum install python-devel python-pip python-virtualenv
|
||||
$ sudo yum install libxml2-devel libxslt-devel libffi-devel libyaml-devel openssl-devel
|
||||
|
||||
MacOSX:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ # Download the XCode command line tools from Apple App Store
|
||||
$ xcode-select --install
|
||||
$ sudo easy_install pip
|
||||
$ sudo pip install virtualenv
|
||||
|
||||
Step 2
|
||||
^^^^^^
|
||||
|
||||
Create a virtual environment for Python, and install KloudBuster:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ virtualenv vkb
|
||||
$ source vkb/bin/activate
|
||||
$ pip install kloudbuster
|
||||
|
||||
Or, if you have `virtualenvwrapper <https://virtualenvwrapper.readthedocs.org>`_ installed::
|
||||
Alternatively, if you have `virtualenvwrapper <https://virtualenvwrapper.readthedocs.org>`_ installed:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ mkvirtualenv kloudbuster
|
||||
$ pip install kloudbuster
|
||||
|
||||
.. note::
|
||||
"A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them." It is optional but recommended. We could use::
|
||||
|
||||
To verify kloudbuster is installed, just type
|
||||
$ sudo pip install kloudbuster
|
||||
|
||||
.. code::
|
||||
|
||||
kloudbuster --help
|
||||
instead if isolation among multiple Python projects is not needed.
|
||||
|
||||
|
||||
Source Installation
|
||||
-------------------
|
||||
For code development, clone the kloudbuster git repository::
|
||||
To verify kloudbuster is installed, just type::
|
||||
|
||||
git clone https://github.com/openstack/kloudbuster.git
|
||||
kloudbuster --help
|
||||
|
||||
Then install dependencies (after optionally creating and activating a virtual env)::
|
||||
.. _git_installation:
|
||||
|
||||
cd kloudbuster
|
||||
pip install -r requirements.txt
|
||||
pip install -r test-requirements.txt
|
||||
GitHub/OpenStack Repository based Installation
|
||||
----------------------------------------------
|
||||
|
||||
To verify kloudbuster is installed, just type
|
||||
|
||||
.. code::
|
||||
|
||||
python kloudbuster/kloudbuster.py --help
|
||||
It is recommended to run KloudBuster inside a virtual environment. However, it can be skipped if installed in a dedicated VM.
|
||||
|
||||
|
||||
VM Image Upload
|
||||
Super quick installation on Ubuntu/Debian
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo apt-get install python-dev python-virtualenv git git-review qemu-utils
|
||||
$ sudo apt-get install libxml2-dev libxslt-dev libffi-dev libz-dev libyaml-dev libssl-dev
|
||||
$ # create a virtual environment
|
||||
$ virtualenv ./vkb
|
||||
$ source ./vkb/bin/activate
|
||||
$ git clone https://github.com/openstack/kloudbuster.git
|
||||
$ cd kloudbuster
|
||||
$ pip install -r requirements-dev.txt
|
||||
|
||||
Super quick installation on RHEL/Fedora/CentOS
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ sudo yum install python-devel python-virtualenv git qemu-img
|
||||
$ sudo yum install libxml2-devel libxslt-devel libffi-devel libyaml-devel openssl-devel
|
||||
$ # create a virtual environment
|
||||
$ virtualenv ./vkb
|
||||
$ source ./vkb/bin/activate
|
||||
$ git clone https://github.com/openstack/kloudbuster.git
|
||||
$ cd kloudbuster
|
||||
$ pip install -r requirements-dev.txt
|
||||
|
||||
Super quick installation on MacOSX
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
KloudBuster can run natively on MacOSX. These instructions have been verified to work on MacOSX 10.10 (Yosemite).
|
||||
|
||||
First, download XCode from App Store, then execute below commands:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ # Download the XCode command line tools
|
||||
$ xcode-select --install
|
||||
$ # Install pip
|
||||
$ sudo easy_install pip
|
||||
$ # Install python virtualenv
|
||||
$ sudo pip install virtualenv
|
||||
$ # create a virtual environment
|
||||
$ virtualenv ./vkb
|
||||
$ source ./vkb/bin/activate
|
||||
$ git clone https://github.com/openstack/kloudbuster.git
|
||||
$ cd kloudbuster
|
||||
$ pip install -r requirements-dev.txt
|
||||
|
||||
Verify installation
|
||||
^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
To verify kloudbuster is installed, just type:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python kloudbuster/kloudbuster.py --help
|
||||
|
||||
|
||||
Upload VM Image
|
||||
---------------
|
||||
|
||||
.. note::
|
||||
|
||||
If your OpenStack Glance can access the Internet, you can skip this section and you are done with the installation.
|
||||
|
||||
KloudBuster needs one "universal" test VM image (referred to as "KloudBuster image") that contains the necessary test software. The KloudBuster image is then instantiated in potentially large number of VMs by the KloudBuster application using the appropriate role (HTTP server, HTTP traffic generator...).
|
||||
|
||||
Pre-built images are available for download from the `OpenStack App Catalog <http://apps.openstack.org>`_ (preferred method) or can be built from MacOSX using Vagrant or from any Linux server.
|
||||
|
||||
If your OpenStack Glance can access the Internet, you can skip the following section and **you are done with the installation**.
|
||||
Pre-built images are available for download from the `OpenStack App Catalog <http://apps.openstack.org>`_ (preferred method). For whatever reason the pre-built version doesn't work for you, the image can be re-built from MacOSX using Vagrant or from any Linux server. See :ref:`below section <build_vm_image>` for more details.
|
||||
|
||||
|
||||
Manual Upload of the KloudBuster VM image
|
||||
Manual upload of the KloudBuster VM image
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
If Glance does not have access to http://storage.apps.openstack.org on the Internet, the KloudBuster VM image must be downloaded from the OpenStack App Catalog to an intermediate location then uploaded to Glance using either a Glance CLI command or using the Horizon dashboard.
|
||||
The KloudBuster VM image can be downloaded from `<http://apps.openstack.org/#tab=glance-images>`_ and look for an image named with the "kloudbuster_v" prefix and download the one that has the latest version.
|
||||
If Glance does not have access to http://storage.apps.openstack.org on the Internet, the KloudBuster VM image must be downloaded from the OpenStack App Catalog to an intermediate location then uploaded to Glance using either a Glance CLI command or via Horizon dashboard.
|
||||
|
||||
KloudBuster VM images are qcow2 images named "kloudbuster_v<version>.qcow2" (e.g. "kloudbuster_v3.qcow2").
|
||||
The KloudBuster VM image can be downloaded from `<http://apps.openstack.org/#tab=glance-images>`_. Look for an image named with the "kloudbuster_v" prefix and download the one that has the latest version.
|
||||
|
||||
The name of the image in Glance must match exactly the image name in the App Catalog (without the .qcow2 extension), for example to upload the image from a local copy of that image:
|
||||
KloudBuster VM images are qcow2 images named "kloudbuster_v<version>.qcow2" (e.g. "kloudbuster_v3.qcow2"). The name of the image in Glance must match exactly the image name in the App Catalog (without the .qcow2 extension), for example to upload the image from a local copy of that image:
|
||||
|
||||
.. code::
|
||||
.. code-block:: bash
|
||||
|
||||
glance image-create --file kloudbuster_v3.qcow2 --disk-format qcow2 --container-format bare --name kloudbuster_v3
|
||||
$ glance image-create --file kloudbuster_v3.qcow2 --disk-format qcow2 --container-format bare --is-public True --name kloudbuster_v3
|
||||
|
||||
|
||||
Rebuild the Image
|
||||
^^^^^^^^^^^^^^^^^
|
||||
.. _build_vm_image:
|
||||
|
||||
Only if using the pre-built version does not work (for whatever reason).
|
||||
Build VM image
|
||||
--------------
|
||||
|
||||
The official pre-built image from OpenStack App Catalog should satisfy most of use cases. This option is just for development purposes, and for whatever reason the pre-built version does not work for you.
|
||||
|
||||
|
||||
MacOSX with Vagrant
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
Build on MacOSX
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
You need to install first:
|
||||
|
||||
* `Virtualbox <https://cisco.jiveon.com/external-link.jspa?url=https://www.virtualbox.org/wiki/Downloads>`_
|
||||
* `Vagrant <https://cisco.jiveon.com/external-link.jspa?url=https://www.vagrantup.com/downloads.html>`_
|
||||
* `Virtualbox <https://www.virtualbox.org/wiki/Downloads>`_
|
||||
* `Vagrant <https://www.vagrantup.com/downloads.html>`_
|
||||
|
||||
.. code::
|
||||
And build the image with below commands:
|
||||
|
||||
# clone the kloudbuster repository if you have not done so
|
||||
git clone https://github.com/openstack/kloudbuster.git
|
||||
# go to the dib directory
|
||||
cd kloudbuster/kloudbuster/dib
|
||||
# run vagrant and start building the image
|
||||
vagrant up
|
||||
.. code-block:: bash
|
||||
|
||||
$ # Clone the kloudbuster repository if you have not done so
|
||||
$ git clone https://github.com/openstack/kloudbuster.git
|
||||
$ # Go to the dib directory
|
||||
$ cd kloudbuster/kb_dib
|
||||
$ # Run vagrant and start building the image
|
||||
$ vagrant up
|
||||
|
||||
After a few minutes (depending on virtualbox overhead), the qcow2 image will be built and available in the same directory. You can then upload it to OpenStack using the glance CLI, destroy the vagrant VM ("vagrant destroy") and dispose of the kloudbuster directory (if no longer needed).
|
||||
|
||||
Buid on Linux
|
||||
~~~~~~~~~~~~~
|
||||
Build on Linux
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
A generally faster build method than with MacOSX/Vagrant.
|
||||
Your Linux server must have python, git and qemu utilities installed.
|
||||
A generally faster build method than with MacOSX/Vagrant. Your Linux server must have python, git and qemu utilities installed. You can skip the package installation if you've installed KloudBuster using the GitHub/OpenStack repository based installation.
|
||||
|
||||
Ubuntu/Debian::
|
||||
Ubuntu/Debian based:
|
||||
|
||||
$ sudo apt-get install python-dev git qemu-utils
|
||||
.. code-block:: bash
|
||||
|
||||
Redhat/Fedora/CentOS::
|
||||
$ sudo apt-get install python-dev git qemu-utils
|
||||
$ # Source the virtual environment if you have one
|
||||
$ pip install PyYAML
|
||||
|
||||
sudo yum install python-devel git qemu-img
|
||||
Redhat/Fedora/CentOS based:
|
||||
|
||||
Furthermore, the python PyYAML package must be installed (use "pip install PyYAML" in your virtual environment if you have one).
|
||||
.. code-block:: bash
|
||||
|
||||
Then build the image:
|
||||
|
||||
.. code::
|
||||
$ sudo yum install python-devel git qemu-img
|
||||
$ # Source the virtual environment if you have one
|
||||
$ pip install PyYAML
|
||||
|
||||
# clone the kloudbuster repository
|
||||
git clone https://github.com/openstack/kloudbuster.git
|
||||
Build the image with below commands:
|
||||
|
||||
# go to the dib directory
|
||||
cd kloudbuster/kloudbuster/dib
|
||||
.. code-block:: bash
|
||||
|
||||
# run the build image script, will install DIB and start the build
|
||||
sh build-image.sh
|
||||
|
||||
After a few minutes, the qcow2 image will be built and available in the same directory. You can then upload it to OpenStack using the glance CLI),
|
||||
$ # Clone the kloudbuster repository if you have not done so
|
||||
$ git clone https://github.com/openstack/kloudbuster.git
|
||||
$ # Go to the dib directory
|
||||
$ cd kloudbuster/kb_dib
|
||||
$ # Run the build image script, which will install DIB and start the build
|
||||
$ ./build-image.sh
|
||||
|
||||
If you get an error message saying that import yaml fails (seems to happen only on Ubuntu):
|
||||
After a few minutes, the qcow2 image will be built and available in the same directory. You can then upload it to OpenStack using the glance CLI.
|
||||
|
||||
.. code::
|
||||
Trouble-shooting
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
dib-run-parts Thu Jul 2 09:27:50 PDT 2015 Running /tmp/image.ewtpa5DW/hooks/extra-data.d/99-squash-package-install
|
||||
|
||||
"/tmp/image.ewtpa5DW/hooks/extra-data.d/../bin/package-installs-squash",
|
||||
line 26, in <module>
|
||||
import yaml
|
||||
ImportError: No module named yaml
|
||||
|
||||
You need to comment out the secure_path option in your /etc/sudoers file (use "sudo visudo" to edit that file):
|
||||
If you get an error message saying that import yaml fails (seems to happen only on Ubuntu)::
|
||||
|
||||
.. code::
|
||||
dib-run-parts Thu Jul 2 09:27:50 PDT 2015 Running /tmp/image.ewtpa5DW/hooks/extra-data.d/99-squash-package-install
|
||||
|
||||
#Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
"/tmp/image.ewtpa5DW/hooks/extra-data.d/../bin/package-installs-squash",
|
||||
line 26, in <module>
|
||||
import yaml
|
||||
ImportError: No module named yaml
|
||||
|
||||
You need to comment out the secure_path option in your /etc/sudoers file (use "sudo visudo" to edit that file)::
|
||||
|
||||
#Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
|
||||
|
||||
6
kb_dib/Vagrantfile
vendored
6
kb_dib/Vagrantfile
vendored
@ -17,9 +17,9 @@ git clone git://github.com/openstack/dib-utils.git
|
||||
# install kloudbuster
|
||||
git clone git://github.com/openstack/kloudbuster.git
|
||||
|
||||
kb_root=kloudbuster/kloudbuster
|
||||
kb_root=kloudbuster
|
||||
# Extract image version number '__version__ = 2.0' becomes '__version__=2_0'
|
||||
ver=`grep '^__version__' $kb_root/kb_vm_agent.py | tr -d ' ' | tr '.' '_'`
|
||||
ver=`grep '^__version__' $kb_root/kloudbuster/kb_vm_agent.py | tr -d ' ' | tr '.' '_'`
|
||||
eval $ver
|
||||
|
||||
kb_image_name=kloudbuster_v$__version__
|
||||
@ -30,7 +30,7 @@ echo "Building $kb_image_name.qcow2..."
|
||||
export PATH=$PATH:`pwd`/diskimage-builder/bin:`pwd`/dib-utils/bin
|
||||
|
||||
# Add the kloudbuster elements directory to the DIB elements path
|
||||
export ELEMENTS_PATH=`pwd`/$kb_root/dib/elements
|
||||
export ELEMENTS_PATH=`pwd`/$kb_root/kb_dib/elements
|
||||
time disk-image-create -o $kb_image_name ubuntu kloudbuster
|
||||
mv $kb_image_name.qcow2 /vagrant
|
||||
|
||||
|
||||
@ -18,7 +18,7 @@ export PATH=$PATH:`pwd`/diskimage-builder/bin:`pwd`/dib-utils/bin
|
||||
export ELEMENTS_PATH=`pwd`/elements
|
||||
|
||||
# Extract image version number '__version__ = 2.0' becomes '__version__=2_0'
|
||||
ver=`grep '^__version__' ../kb_vm_agent.py | tr -d ' ' | tr '.' '_'`
|
||||
ver=`grep '^__version__' ../kloudbuster/kb_vm_agent.py | tr -d ' ' | tr '.' '_'`
|
||||
eval $ver
|
||||
|
||||
kb_image_name=kloudbuster_v$__version__
|
||||
|
||||
@ -100,6 +100,7 @@ class KB_Instance(object):
|
||||
report_interval):
|
||||
if not rate_limit:
|
||||
rate_limit = 65535
|
||||
|
||||
cmd = '%s -t%d -c%d -R%d -d%ds -p%ds --timeout %ds -D2 -e %s' % \
|
||||
(dest_path, threads, connections, rate_limit, duration,
|
||||
report_interval, timeout, target_url)
|
||||
|
||||
@ -158,7 +158,7 @@ paths:
|
||||
404:
|
||||
description: The session_id is not found or invalid
|
||||
|
||||
/kloudbuster/console_log/{session_id}:
|
||||
/kloudbuster/log/{session_id}:
|
||||
get:
|
||||
description: |
|
||||
Get KloudBuster console log for a given session
|
||||
|
||||
@ -170,13 +170,18 @@ class SecGroup(object):
|
||||
Sometimes this maybe in use if instance is just deleted
|
||||
Add a retry mechanism
|
||||
"""
|
||||
if not self.secgroup:
|
||||
return True
|
||||
|
||||
for _ in range(10):
|
||||
try:
|
||||
self.novaclient.security_groups.delete(self.secgroup)
|
||||
break
|
||||
return True
|
||||
except Exception:
|
||||
time.sleep(2)
|
||||
|
||||
LOG.error('Failed while deleting security group %s.' % self.secgroup.id)
|
||||
return False
|
||||
|
||||
class KeyPair(object):
|
||||
|
||||
@ -208,7 +213,8 @@ class KeyPair(object):
|
||||
"""
|
||||
Remove the keypair created by KloudBuster
|
||||
"""
|
||||
self.novaclient.keypairs.delete(self.keypair)
|
||||
if self.keypair:
|
||||
self.novaclient.keypairs.delete(self.keypair)
|
||||
|
||||
|
||||
class Flavor(object):
|
||||
|
||||
@ -35,6 +35,7 @@ def create_floating_ip(neutron_client, ext_net):
|
||||
"""
|
||||
Function that creates a floating ip and returns it
|
||||
Accepts the neutron client and ext_net
|
||||
|
||||
Module level function since this is not associated with a
|
||||
specific network instance
|
||||
"""
|
||||
@ -51,8 +52,19 @@ def delete_floating_ip(neutron_client, fip):
|
||||
Deletes the floating ip
|
||||
Module level function since this operation
|
||||
is not associated with a network
|
||||
|
||||
Sometimes this will fail if instance is just deleted
|
||||
Add a retry mechanism
|
||||
"""
|
||||
neutron_client.delete_floatingip(fip)
|
||||
for _ in range(10):
|
||||
try:
|
||||
neutron_client.delete_floatingip(fip)
|
||||
return True
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
|
||||
LOG.error('Failed while deleting floating IP %s.' % fip['floatingip']['id'])
|
||||
return False
|
||||
|
||||
def find_external_network(neutron_client):
|
||||
"""
|
||||
@ -141,26 +153,20 @@ class BaseNetwork(object):
|
||||
Deletes the compute resources
|
||||
Security groups, keypairs and instances
|
||||
"""
|
||||
flag = True
|
||||
# Delete the instances first
|
||||
for instance in self.instance_list:
|
||||
instance.delete_server()
|
||||
if instance.fip:
|
||||
"""
|
||||
Delete the Floating IP
|
||||
Sometimes this will fail if instance is just deleted
|
||||
Add a retry mechanism
|
||||
"""
|
||||
for _ in range(10):
|
||||
try:
|
||||
delete_floating_ip(self.neutron_client, instance.fip['floatingip']['id'])
|
||||
break
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
# Delete the Floating IP
|
||||
flag = flag &\
|
||||
delete_floating_ip(self.neutron_client, instance.fip['floatingip']['id'])
|
||||
|
||||
# Delete all security groups
|
||||
for secgroup_instance in self.secgroup_list:
|
||||
secgroup_instance.delete_secgroup()
|
||||
flag = flag & secgroup_instance.delete_secgroup()
|
||||
|
||||
return flag
|
||||
|
||||
def create_network_and_subnet(self, network_name):
|
||||
"""
|
||||
@ -202,13 +208,18 @@ class BaseNetwork(object):
|
||||
Deletes the network and associated subnet
|
||||
retry the deletion since network may be in use
|
||||
"""
|
||||
for _ in range(1, 5):
|
||||
if not self.network:
|
||||
return True
|
||||
for _ in range(10):
|
||||
try:
|
||||
self.neutron_client.delete_network(self.network['id'])
|
||||
break
|
||||
return True
|
||||
except NetworkInUseClient:
|
||||
time.sleep(1)
|
||||
|
||||
LOG.error('Failed while deleting network %s.' % self.network['id'])
|
||||
return False
|
||||
|
||||
def get_all_instances(self):
|
||||
return self.instance_list
|
||||
|
||||
@ -269,21 +280,19 @@ class Router(object):
|
||||
associated with a router
|
||||
"""
|
||||
|
||||
flag = True
|
||||
for network in self.network_list:
|
||||
# Now delete the compute resources and the network resources
|
||||
network.delete_compute_resources()
|
||||
flag = flag & network.delete_compute_resources()
|
||||
if network.network:
|
||||
self.remove_router_interface(network)
|
||||
network.delete_network()
|
||||
flag = flag & self.remove_router_interface(network)
|
||||
flag = flag & network.delete_network()
|
||||
# Also delete the shared port and remove it from router interface
|
||||
if self.shared_network:
|
||||
for _ in range(10):
|
||||
try:
|
||||
self.remove_router_interface(self.shared_network, use_port=True)
|
||||
self.shared_network = None
|
||||
break
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
flag = flag & self.remove_router_interface(self.shared_network, use_port=True)
|
||||
self.shared_network = None
|
||||
|
||||
return flag
|
||||
|
||||
def create_router(self, router_name, ext_net):
|
||||
"""
|
||||
@ -317,20 +326,22 @@ class Router(object):
|
||||
Also delete the networks attached to this router
|
||||
"""
|
||||
# Delete the network resources first and than delete the router itself
|
||||
self.delete_network_resources()
|
||||
if not self.router:
|
||||
return True
|
||||
network_flag = self.delete_network_resources()
|
||||
router_flag = False
|
||||
for _ in range(10):
|
||||
try:
|
||||
self.neutron_client.remove_gateway_router(self.router['router']['id'])
|
||||
self.shared_network = None
|
||||
break
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
for _ in range(10):
|
||||
try:
|
||||
self.neutron_client.delete_router(self.router['router']['id'])
|
||||
router_flag = True
|
||||
break
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
if not router_flag:
|
||||
LOG.error('Failed while deleting router %s.' % self.router['router']['id'])
|
||||
|
||||
return network_flag & router_flag
|
||||
|
||||
def _port_create_neutron(self, network_instance):
|
||||
"""
|
||||
@ -377,8 +388,15 @@ class Router(object):
|
||||
body = {
|
||||
'subnet_id': network_instance.network['subnets'][0]
|
||||
}
|
||||
self.neutron_client.remove_interface_router(self.router['router']['id'], body)
|
||||
|
||||
for _ in range(10):
|
||||
try:
|
||||
self.neutron_client.remove_interface_router(self.router['router']['id'], body)
|
||||
return True
|
||||
except Exception:
|
||||
time.sleep(1)
|
||||
|
||||
return False
|
||||
|
||||
class NeutronQuota(object):
|
||||
|
||||
|
||||
@ -62,6 +62,7 @@ if [ "$1" == "--file" ] && [ -f "$2" ]; then
|
||||
INSTANCE_LIST=`grep "instances" $2 | cut -d'|' -f3`
|
||||
SEC_GROUP_LIST=`grep "sec_groups" $2 | cut -d'|' -f3`
|
||||
FLAVOR_LIST=`grep "flavors" $2 | cut -d'|' -f3`
|
||||
KEYPAIR_LIST=`grep "keypairs" $2 | cut -d'|' -f2`
|
||||
ROUTER_LIST=`grep "routers" $2 | cut -d'|' -f3`
|
||||
NETWORK_LIST=`grep "networks" $2 | cut -d'|' -f3`
|
||||
TENANT_LIST=`grep "tenants" $2 | cut -d'|' -f3`
|
||||
@ -72,6 +73,7 @@ else
|
||||
INSTANCE_LIST=`nova list --all-tenants | grep KB | cut -d'|' -f2`
|
||||
SEC_GROUP_LIST=`neutron security-group-list | grep KB | cut -d'|' -f2`
|
||||
FLAVOR_LIST=`nova flavor-list | grep kb | cut -d'|' -f3`
|
||||
KEYPAIR_LIST=`nova keypair-list | grep KB | cut -d'|' -f2`
|
||||
ROUTER_LIST=`neutron router-list | grep KB | cut -d'|' -f2`
|
||||
NETWORK_LIST=`neutron net-list | grep KB | cut -d'|' -f2`
|
||||
TENANT_LIST=`keystone tenant-list | grep KB | cut -d'|' -f2`
|
||||
@ -91,6 +93,10 @@ for line in $SEC_GROUP_LIST; do
|
||||
neutron security-group-delete $line &
|
||||
done;
|
||||
|
||||
for line in $KEYPAIR_LIST; do
|
||||
nova keypair-delete "$line"
|
||||
done;
|
||||
|
||||
if [ "$FLOATINGIP_LIST" == "" ]; then
|
||||
echo -e "`neutron floatingip-list | grep -E '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'`" | while read line; do
|
||||
fid=`echo $line | cut -d'|' -f2 | xargs`
|
||||
|
||||
@ -114,8 +114,9 @@ class Kloud(object):
|
||||
nova_client = self.tenant_list[0].user_list[0].nova_client
|
||||
except Exception:
|
||||
# NOVA Client is not yet initialized, so skip cleaning up...
|
||||
return
|
||||
return True
|
||||
|
||||
flag = True
|
||||
if not self.reusing_tenants:
|
||||
flavor_manager = base_compute.Flavor(nova_client)
|
||||
if self.testing_side:
|
||||
@ -125,7 +126,9 @@ class Kloud(object):
|
||||
flavor_manager.delete_flavor('kb.server')
|
||||
|
||||
for tnt in self.tenant_list:
|
||||
tnt.delete_resources()
|
||||
flag = flag & tnt.delete_resources()
|
||||
|
||||
return flag
|
||||
|
||||
def get_first_network(self):
|
||||
if self.tenant_list:
|
||||
@ -341,6 +344,7 @@ class KloudBuster(object):
|
||||
The runner for KloudBuster Tests
|
||||
"""
|
||||
vm_creation_concurrency = self.client_cfg.vm_creation_concurrency
|
||||
cleanup_flag = True
|
||||
try:
|
||||
tenant_quota = self.calc_tenant_quota()
|
||||
self.kloud.create_resources(tenant_quota['server'])
|
||||
@ -419,16 +423,22 @@ class KloudBuster(object):
|
||||
# then testing side last (order is important because of the shared network)
|
||||
if self.server_cfg['cleanup_resources']:
|
||||
try:
|
||||
self.kloud.delete_resources()
|
||||
cleanup_flag = self.kloud.delete_resources()
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
KBResLogger.dump_and_save('svr', self.kloud.res_logger.resource_list)
|
||||
if not cleanup_flag:
|
||||
LOG.warn('Some resources are not cleaned up properly.')
|
||||
KBResLogger.dump_and_save('svr', self.kloud.res_logger.resource_list)
|
||||
|
||||
if self.client_cfg['cleanup_resources']:
|
||||
try:
|
||||
self.testing_kloud.delete_resources()
|
||||
cleanup_flag = self.testing_kloud.delete_resources()
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
KBResLogger.dump_and_save('clt', self.testing_kloud.res_logger.resource_list)
|
||||
if not cleanup_flag:
|
||||
KBResLogger.dump_and_save('clt', self.testing_kloud.res_logger.resource_list)
|
||||
|
||||
def dump_logs(self, offset=0):
|
||||
if not self.fp_logfile:
|
||||
|
||||
@ -124,10 +124,13 @@ class Tenant(object):
|
||||
Delete all user resources and than
|
||||
deletes the tenant
|
||||
"""
|
||||
flag = True
|
||||
# Delete all the users in the tenant along with network and compute elements
|
||||
for user in self.user_list:
|
||||
user.delete_resources()
|
||||
flag = flag & user.delete_resources()
|
||||
|
||||
if not self.reusing_users:
|
||||
# Delete the tenant (self)
|
||||
self.kloud.keystone.tenants.delete(self.tenant_id)
|
||||
|
||||
return flag
|
||||
|
||||
@ -107,18 +107,21 @@ class User(object):
|
||||
def delete_resources(self):
|
||||
LOG.info("Deleting all user resources for user %s" % self.user_name)
|
||||
|
||||
flag = True
|
||||
# Delete key pair
|
||||
if self.key_pair:
|
||||
self.key_pair.remove_public_key()
|
||||
|
||||
# Delete all user routers
|
||||
for router in self.router_list:
|
||||
router.delete_router()
|
||||
flag = flag & router.delete_router()
|
||||
|
||||
if not self.tenant.reusing_users:
|
||||
# Finally delete the user
|
||||
self.tenant.kloud.keystone.users.delete(self.user.id)
|
||||
|
||||
return flag
|
||||
|
||||
def update_tenant_quota(self, tenant_quota):
|
||||
nova_quota = base_compute.NovaQuota(self.nova_client, self.tenant.tenant_id)
|
||||
nova_quota.update_quota(**tenant_quota['nova'])
|
||||
@ -209,6 +212,7 @@ class User(object):
|
||||
if config_scale.public_key_file:
|
||||
self.key_pair = base_compute.KeyPair(self.nova_client)
|
||||
self.key_name = self.user_name + '-K'
|
||||
self.res_logger.log('keypairs', self.key_name, "")
|
||||
self.key_pair.add_public_key(self.key_name, config_scale.public_key_file)
|
||||
|
||||
# Find the external network that routers need to attach to
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user