Merge "Revise all lint tests and add consolidated 'linters' tox environment"
This commit is contained in:
Jenkins
Gerrit Code Review
commit
e28d2dcadd
@ -14,30 +14,39 @@ these scripts from the root of the repository clone. For example:
|
||||
Bootstrapping
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
bootstrap-aio.sh
|
||||
----------------
|
||||
|
||||
The ``bootstrap-aio.sh`` script prepares a host for an *all-in-one* (AIO)
|
||||
deployment for the purposes of development and gating. Create the necessary
|
||||
partitions, directories, and configurations. Configurable via environment
|
||||
variables to work with Jenkins.
|
||||
|
||||
bootstrap-ansible.sh
|
||||
--------------------
|
||||
|
||||
The ``bootstrap-ansible.sh`` script installs Ansible including core and extras
|
||||
module repositories and Galaxy roles.
|
||||
The ``bootstrap-ansible.sh`` script installs Ansible including `core`_ and
|
||||
`extras`_ module repositories and Galaxy roles.
|
||||
|
||||
Configurable environment variables:
|
||||
While there are several configurable environment variables which this script
|
||||
uses, the following are commonly used:
|
||||
|
||||
* ``ANSIBLE_GIT_RELEASE`` - Version of Ansible to install.
|
||||
* ``ANSIBLE_ROLE_FILE`` - Galaxy roles to install. Defaults to
|
||||
contents of ``ansible-role-requirements.yml`` file.
|
||||
* ``ANSIBLE_GIT_RELEASE`` - The version of Ansible to install.
|
||||
|
||||
* ``ANSIBLE_ROLE_FILE`` - The location of a yaml file which ansible-galaxy can
|
||||
consume which specifies which roles to download and install. The default
|
||||
value for this is ``ansible-role-requirements.yml``.
|
||||
|
||||
The script also creates the ``openstack-ansible`` wrapper tool that provides
|
||||
the variable files to match ``/etc/openstack_deploy/user_*.yml`` as
|
||||
arguments to ``ansible-playbook`` as a convenience.
|
||||
|
||||
.. core: https://github.com/ansible/ansible-modules-core
|
||||
.. extras: https://github.com/ansible/ansible-modules-extras
|
||||
|
||||
bootstrap-aio.sh
|
||||
----------------
|
||||
|
||||
The ``bootstrap-aio.sh`` script prepares a host for an `All-In-One`_ (AIO)
|
||||
deployment for the purposes of development and gating. The script creates the
|
||||
necessary partitions, directories, and configurations. The script can be
|
||||
configured using environment variables - more details are provided on the
|
||||
`All-In-One`_ page.
|
||||
|
||||
.. All-In-One: quickstart-aio
|
||||
|
||||
Development and Testing
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
@ -83,8 +92,8 @@ Configurable environment variables:
|
||||
``openstack_tempest_gate.sh`` script, defined in the ``os_tempest`` role.
|
||||
Defaults to ``scenario heat_api cinder_backup``.
|
||||
|
||||
PEP8
|
||||
----
|
||||
Lint Tests
|
||||
----------
|
||||
|
||||
Python coding conventions are tested using `PEP8`_, with the following
|
||||
convention exceptions:
|
||||
@ -98,11 +107,6 @@ Testing may be done locally by executing:
|
||||
|
||||
tox -e pep8
|
||||
|
||||
.. PEP8: https://www.python.org/dev/peps/pep-0008/
|
||||
|
||||
Bashate
|
||||
-------
|
||||
|
||||
Bash coding conventions are tested using `Bashate`_, with the following
|
||||
convention exceptions:
|
||||
|
||||
@ -114,10 +118,34 @@ Testing may be done locally by executing:
|
||||
|
||||
tox -e bashate
|
||||
|
||||
.. Bashate: https://github.com/openstack-dev/bashate
|
||||
Ansible is lint tested using `ansible-lint`_.
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
Testing may be done locally by executing:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tox -e ansible-lint
|
||||
|
||||
Ansible playbook syntax is tested using ansible-playbook.
|
||||
|
||||
Testing may be done locally by executing:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tox -e ansible-syntax
|
||||
|
||||
A consolidated set of all lint tests may be done locally by executing:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
tox -e linters
|
||||
|
||||
.. PEP8: https://www.python.org/dev/peps/pep-0008/
|
||||
.. Bashate: https://github.com/openstack-dev/bashate
|
||||
.. ansible-lint: https://github.com/willthames/ansible-lint
|
||||
|
||||
Documentation Build
|
||||
-------------------
|
||||
|
||||
Documentation is developed in reStructuredText_ (RST) and compiled into
|
||||
HTML using Sphinx.
|
||||
@ -133,43 +161,25 @@ Documentation may be built locally by executing:
|
||||
Gating
|
||||
^^^^^^
|
||||
|
||||
gate-check-commit.sh
|
||||
--------------------
|
||||
Every commit to OpenStack-Ansible is verified by OpenStack-CI through the
|
||||
following jobs:
|
||||
|
||||
The ``gate-check-commit.sh`` script executes a suite of tests necessary for
|
||||
each commit to the repository. By default, the script runs the bootstrap
|
||||
scripts, builds an *all-in-one* deployment of OSA, and runs various Tempest
|
||||
tests on it.
|
||||
* ``gate-openstack-ansible-docs``: This job executes the
|
||||
`Documentation Build`_.
|
||||
|
||||
Configurable environment variables:
|
||||
* ``gate-openstack-ansible-linters``: This job executes the `Lint Tests`_.
|
||||
|
||||
* ``BOOTSTRAP_AIO`` - Boolean (yes/no) to run AIO bootstrap script. Defaults
|
||||
to ``yes``.
|
||||
* ``BOOTSTRAP_ANSIBLE`` - Boolean (yes/no) to run Ansible bootstrip script.
|
||||
Defaults to ``yes``.
|
||||
* ``RUN_TEMPEST`` - Boolean (yes/no) to run Tempest tests. Defaults to
|
||||
``yes``.
|
||||
* ``gate-openstack-ansible-dsvm-commit``: This job executes the
|
||||
``gate-check-commit.sh`` script which executes a convergence test and then a
|
||||
functional test.
|
||||
|
||||
gate-check-docs.sh
|
||||
------------------
|
||||
The convergence test is the execution of an AIO build
|
||||
which aims to test the primary code path for a functional environment. The
|
||||
functional test then executes OpenStack's Tempest testing suite to verify
|
||||
that the environment that has deployed successfully actually works.
|
||||
|
||||
The ``gate-check-docs.sh`` script invokes Sphinx to build the HTML
|
||||
documentation from RST source.
|
||||
|
||||
gate-check-lint.sh
|
||||
------------------
|
||||
|
||||
The ``gate-check-lint.sh`` script executes a suite of tests necessary for each
|
||||
commit to the repository to verify correct YAML and Python syntax.
|
||||
|
||||
All files that begin with a Python shebang pass through *flake8* which ignores
|
||||
the following rules due to Ansible conventions:
|
||||
|
||||
* F403 - 'from ansible.module_utils.basic import \*'
|
||||
* H303 - No wildcard imports
|
||||
|
||||
Ansible playbooks pass through ``ansible-playbook --syntax-check``
|
||||
and ``ansible-lint``.
|
||||
While this script is primarily developed and maintained for use in
|
||||
OpenStack-CI, it can be used in other environments.
|
||||
|
||||
--------------
|
||||
|
||||
|
||||
@ -1,38 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
## Shell Opts ----------------------------------------------------------------
|
||||
set -e -u -x
|
||||
|
||||
|
||||
## Library Check -------------------------------------------------------------
|
||||
info_block "Checking for required libraries." 2> /dev/null || source $(dirname ${0})/scripts-library.sh
|
||||
|
||||
|
||||
## Main ----------------------------------------------------------------------
|
||||
info_block "Running Basic Ansible Lint Check"
|
||||
|
||||
|
||||
# Install the development requirements.
|
||||
if [ -f "test-requirements.txt" ]; then
|
||||
pip2 install -r test-requirements.txt || pip install -r test-requirements.txt
|
||||
fi
|
||||
|
||||
pushd doc
|
||||
info_block "Building HTML docs"
|
||||
|
||||
# Treat warnings as errors.
|
||||
make html SPHINXOPTS="-W"
|
||||
popd
|
||||
@ -1,52 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# Copyright 2014, Rackspace US, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
## Shell Opts ----------------------------------------------------------------
|
||||
set -e -u -x
|
||||
|
||||
|
||||
## Library Check -------------------------------------------------------------
|
||||
info_block "Checking for required libraries." 2> /dev/null || source $(dirname ${0})/scripts-library.sh
|
||||
|
||||
|
||||
## Main ----------------------------------------------------------------------
|
||||
info_block "Running Basic Ansible Lint Check"
|
||||
|
||||
# next, bootstrap Ansible
|
||||
source $(dirname ${0})/bootstrap-ansible.sh
|
||||
|
||||
# Install the development requirements.
|
||||
if [ -f "test-requirements.txt" ]; then
|
||||
pip2 install -r test-requirements.txt || pip install -r test-requirements.txt
|
||||
else
|
||||
pip2 install ansible-lint || pip install ansible-lint
|
||||
fi
|
||||
|
||||
# Create keys if they don't already exist.
|
||||
ssh_key_create
|
||||
|
||||
# Perform our simple sanity checks.
|
||||
pushd playbooks
|
||||
echo -e '[all]\nlocalhost ansible_connection=local' | tee local_only_inventory
|
||||
|
||||
# Do a basic syntax check on all playbooks and roles.
|
||||
info_block "Running Syntax Check"
|
||||
ansible-playbook -i local_only_inventory --syntax-check *.yml --list-tasks
|
||||
|
||||
# Perform a lint check on all playbooks and roles.
|
||||
info_block "Running Lint Check"
|
||||
ansible-lint --version
|
||||
ansible-lint *.yml
|
||||
popd
|
||||
82
tox.ini
82
tox.ini
@ -1,7 +1,7 @@
|
||||
[tox]
|
||||
minversion = 1.6
|
||||
skipsdist = True
|
||||
envlist = docs,pep8,bashate,releasenotes
|
||||
envlist = docs,linters,releasenotes
|
||||
|
||||
[testenv]
|
||||
usedevelop = True
|
||||
@ -42,23 +42,75 @@ commands =
|
||||
|
||||
[flake8]
|
||||
# Ignores the following rules due to how ansible modules work in general
|
||||
# F403 'from ansible.module_utils.basic import *' used; unable to detect undefined names
|
||||
# F403 'from ansible.module_utils.basic import *' used;
|
||||
# unable to detect undefined names
|
||||
# H303 No wildcard (*) import.
|
||||
ignore=F403,H303
|
||||
|
||||
# Run bashate check for all bash scripts
|
||||
# Ignores the following rules:
|
||||
# E003: Indent not multiple of 4 (we prefer to use multiples of 2)
|
||||
[testenv:bashate]
|
||||
deps = bashate
|
||||
deps =
|
||||
bashate
|
||||
whitelist_externals = bash
|
||||
commands =
|
||||
bash -c "grep -Irl \
|
||||
-e '!/usr/bin/env bash' \
|
||||
-e '!/bin/bash' \
|
||||
-e '!/bin/sh' \
|
||||
--exclude-dir '.*' \
|
||||
--exclude-dir '*.egg' \
|
||||
--exclude-dir '*.egg-info' \
|
||||
--exclude 'tox.ini' \
|
||||
{toxinidir} | xargs bashate --verbose --ignore=E003"
|
||||
# Run bashate check for all bash scripts
|
||||
# Ignores the following rules:
|
||||
# E003: Indent not multiple of 4 (we prefer to use multiples of 2)
|
||||
# E005: File does not begin with #! or have .sh prefix
|
||||
# E006: Line longer than 79 columns
|
||||
bash -c "grep -Irl \
|
||||
-e '!/usr/bin/env bash' \
|
||||
-e '!/bin/bash' \
|
||||
-e '!/bin/sh' \
|
||||
--exclude-dir '.*' \
|
||||
--exclude-dir '*.egg' \
|
||||
--exclude-dir '*.egg-info' \
|
||||
--exclude 'tox.ini' \
|
||||
{toxinidir} | xargs bashate --verbose --ignore=E003,E005,E006"
|
||||
|
||||
[testenv:ansible-lint]
|
||||
deps =
|
||||
ansible>1.9,<2.0
|
||||
ansible-lint
|
||||
whitelist_externals = bash
|
||||
commands =
|
||||
# Perform an Ansible lint check
|
||||
bash -c "ansible-lint {toxinidir}/playbooks/*.yml"
|
||||
|
||||
[testenv:ansible-syntax]
|
||||
deps =
|
||||
ansible>1.9,<2.0
|
||||
whitelist_externals = bash
|
||||
commands =
|
||||
# Perform an Ansible syntax check
|
||||
bash -c "mkdir -p {envtmpdir}/ansible; \
|
||||
export ANSIBLE_CONFIG={envtmpdir}/ansible/ansible.cfg; \
|
||||
export ANSIBLE_ROLES_PATH={envtmpdir}/ansible/roles; \
|
||||
sed 's|/etc/ansible|{envtmpdir}/ansible|' \
|
||||
{toxinidir}/tests/ansible.cfg | \
|
||||
tee {envtmpdir}/ansible/ansible.cfg; \
|
||||
sed 's|path: /etc/ansible|path: {envtmpdir}/ansible|' \
|
||||
{toxinidir}/ansible-role-requirements.yml | \
|
||||
tee {envtmpdir}/ansible/ansible-role-requirements.yml; \
|
||||
ansible-galaxy install \
|
||||
--role-file={envtmpdir}/ansible/ansible-role-requirements.yml \
|
||||
--roles-path={envtmpdir}/ansible/roles \
|
||||
--ignore-errors \
|
||||
--force; \
|
||||
ansible-playbook -i 'localhost ansible-connection=local,' \
|
||||
--syntax-check \
|
||||
--list-tasks \
|
||||
{toxinidir}/playbooks/*.yml"
|
||||
|
||||
[testenv:linters]
|
||||
deps =
|
||||
{[testenv:pep8]deps}
|
||||
{[testenv:bashate]deps}
|
||||
{[testenv:ansible-lint]deps}
|
||||
# note that the ansible-syntax deps are omitted on purpose as the
|
||||
# ansible-lint env contains duplicate items in its dep list
|
||||
whitelist_externals = bash
|
||||
commands =
|
||||
{[testenv:pep8]commands}
|
||||
{[testenv:bashate]commands}
|
||||
{[testenv:ansible-lint]commands}
|
||||
{[testenv:ansible-syntax]commands}
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user