135 lines
3.7 KiB
ReStructuredText
135 lines
3.7 KiB
ReStructuredText
=======
|
|
Testing
|
|
=======
|
|
|
|
Kayobe has a number of test suites covering different areas of code. Many tests
|
|
are run in virtual environments using ``tox``.
|
|
|
|
Preparation
|
|
===========
|
|
|
|
System Prerequisites
|
|
--------------------
|
|
|
|
The following packages should be installed on the development system prior to
|
|
running kayobe's tests.
|
|
|
|
* Ubuntu/Debian::
|
|
|
|
sudo apt-get install build-essential python-dev libssl-dev python-pip git
|
|
|
|
* Fedora 21/RHEL7/CentOS7::
|
|
|
|
sudo yum install python-devel openssl-devel python-pip git gcc
|
|
|
|
* Fedora 22 or higher::
|
|
|
|
sudo dnf install python-devel openssl-devel python-pip git gcc
|
|
|
|
* OpenSUSE/SLE 12::
|
|
|
|
sudo zypper install python-devel python-pip libopenssl-devel git
|
|
|
|
Python Prerequisites
|
|
--------------------
|
|
|
|
If your distro has at least ``tox 1.8``, use your system package manager to
|
|
install the ``python-tox`` package. Otherwise install this on all distros::
|
|
|
|
sudo pip install -U tox
|
|
|
|
You may need to explicitly upgrade ``virtualenv`` if you've installed the one from
|
|
your OS distribution and it is too old (tox will complain). You can upgrade it
|
|
individually, if you need to::
|
|
|
|
sudo pip install -U virtualenv
|
|
|
|
Running Unit Tests Locally
|
|
==========================
|
|
|
|
If you haven't already, the kayobe source code should be pulled directly from
|
|
git::
|
|
|
|
# from your home or source directory
|
|
cd ~
|
|
git clone https://git.openstack.org/openstack/kayobe.git
|
|
cd kayobe
|
|
|
|
Running Unit and Style Tests
|
|
----------------------------
|
|
|
|
Kayobe defines a number of different tox environments in ``tox.ini``. The
|
|
default environments may be displayed::
|
|
|
|
tox -list
|
|
|
|
To run all default environments::
|
|
|
|
tox
|
|
|
|
To run one or more specific environments, including any of the non-default
|
|
environments::
|
|
|
|
tox -e <environment>[,<environment>]
|
|
|
|
Environments
|
|
------------
|
|
|
|
The following tox environments are provided:
|
|
|
|
alint
|
|
Run Ansible linter.
|
|
ansible
|
|
Run Ansible tests for some ansible roles using Ansible playbooks.
|
|
ansible-syntax
|
|
Run a syntax check for all Ansible files.
|
|
docs
|
|
Build Sphinx documentation.
|
|
molecule
|
|
Run Ansible tests for some Ansible roles using the molecule test framework.
|
|
pep8
|
|
Run style checks for all shell, python and documentation files.
|
|
py27,py34
|
|
Run python unit tests for kayobe python module.
|
|
|
|
Writing Tests
|
|
=============
|
|
|
|
Unit Tests
|
|
----------
|
|
|
|
Unit tests follow the lead of OpenStack, and use ``unittest``. One difference
|
|
is that tests are run using the discovery functionality built into
|
|
``unittest``, rather than ``ostestr``/``stestr``. Unit tests are found in
|
|
``kayobe/tests/unit/``, and should be added to cover all new python code.
|
|
|
|
Ansible Role Tests
|
|
------------------
|
|
|
|
Two types of test exist for Ansible roles - pure Ansible and molecule tests.
|
|
|
|
Pure Ansible Role Tests
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
These tests exist for the ``kolla-ansible`` role, and are found in
|
|
``ansible/<role>/tests/*.yml``. The role is exercised using an ansible
|
|
playbook.
|
|
|
|
Molecule Role Tests
|
|
^^^^^^^^^^^^^^^^^^^
|
|
|
|
`Molecule <https://molecule.readthedocs.io/en/latest/>`_ is an Ansible role
|
|
testing framework that allows roles to be tested in isolation, in a stable
|
|
environment, under multiple scenarios. Kayobe uses Docker engine to provide the
|
|
test environment, so this must be installed and running on the development
|
|
system.
|
|
|
|
Molecule scenarios are found in ``ansible/<role>/molecule/<scenario>``, and
|
|
defined by the config file ``ansible/<role>/molecule/<scenario>/molecule.yml``
|
|
Tests are written in python using the `pytest
|
|
<https://docs.pytest.org/en/latest/>`_ framework, and are found in
|
|
``ansible/<role>/molecule/<scenario>/tests/test_*.py``.
|
|
|
|
Molecule tests currently exist for the ``kolla-openstack`` role, and should be
|
|
added for all new roles where practical.
|