openstack/openstack-ansible
Code Issues Proposed changes

Merge "Developer docs refactor"

Browse Source
This commit is contained in:
Zuul 2018-04-04 20:53:23 +00:00 committed by Gerrit Code Review
commit 80905d232d
18 changed files with 1334 additions and 982 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

372
doc/source/contributor/additional-roles.rst
View File

@ -1,372 +0,0 @@
==================================
Contributing to Roles and Services
==================================
If you would like to contribute towards a role to introduce an OpenStack
or infrastructure service, or to improve an existing role, the
OpenStack-Ansible project would welcome that contribution and your assistance
in maintaining it.
Recommended procedure to develop a role
---------------------------------------
#. Deploy OpenStack-Ansible (possibly using
`an AIO`_
deploy) so that you have the rest of an OpenStack cluster to integrate with
in your testing.
#. Deploy your service on another VM, or possibly directly on the AIO host, by
hand. Configure the service to coordinate with the OpenStack cluster
appropriately. When all the related systems are communicating with each
other you can use the resulting configuration as a reference later.
#. Develop a role for your service. A recommended process is detailed below.
.. _an AIO: quickstart-aio.html
Writing a new role
------------------
Here are the steps to write the role:
#. You can review roles which may be currently in development by checking our
`specs repository`_ and `unmerged specs`_ on review.openstack.org. If you
do not find a spec for the role, propose a blueprint/spec `(see also the
spec template)`_ outlining the new Role. By proposing a draft spec you can
help the OpenStack-Ansible community keep track of what roles are being
developed and perhaps connect you with others who may be interested and
able to help you in the process.
#. Create a source repository (e.g. on Github) to start your work on the Role.
#. Generate the reference directory structure for an Ansible role which is
the necessary subset of the documented `Best Practice`_. You might use
Ansible Galaxy tools to do this for you (e.g. ``ansible-galaxy init``).
You may additionally want to include directories such as ``docs`` and
``examples`` and ``tests`` for your role.
#. Generate a meta/main.yml right away. This file is important to Ansible to
ensure your dependent roles are installed and available and provides others
with the information they will need to understand the purpose of your role.
#. Develop task files for each of the install stages in turn, creating any
handlers and templates as needed. Ensure that you notify handlers after any
task which impacts the way the service would run (such as configuration
file modifications). Also take care that file ownership and permissions are
appropriate.
.. HINT:: Fill in variable defaults, libraries, and prerequisites as you
discover a need for them. You can also develop documentation for your
role at the same time.
#. Add tests to the role.
#. Ensuring the role matches OpenStack-Ansible's latest standards.
#. Deploying the role on an AIO.
Writing tasks in a role
^^^^^^^^^^^^^^^^^^^^^^^
Most OpenStack services will follow a common series of stages to install or
update a service deployment. This is apparent when you review `tasks/main.yml`
for existing roles.
#. pre-install: prepare the service user group and filesystem directory paths
on the host or container
#. install: install system packages, prepare the (optional) service virtual
environment, install service and requirements (into a virtual environment)
#. post-install: apply all configuration files
#. service add: register the service (each of: service type, service project,
service user, and endpoints) within Keystone's service catalog.
#. service setup: install a service-startup script (init, upstart, systemd,
etc.) so that the service will start up when the container or host next
starts.
#. service init/startup: signal to the host or container to start the services,
make sure the service runs on boot.
There may be other specialized steps required by some services but most of the
roles will perform all of these at a minimum. Begin by reviewing a role for a
service that has something in common with your service and think about how you
can fit most of the common service setup and configuration steps into that
model.
.. HINT:: Following the patterns you find in other roles can help ensure your role
is easier to use and maintain.
.. _(see also the spec template): https://git.openstack.org/cgit/openstack/openstack-ansible-specs/tree/specs/templates/template.rst
.. _specs repository: https://git.openstack.org/cgit/openstack/openstack-ansible-specs
.. _unmerged specs: https://review.openstack.org/#/q/status:+open+project:openstack/openstack-ansible-specs
.. _Best Practice: https://docs.ansible.com/ansible/playbooks_best_practices.html#directory-layout
Keep in mind a role candidate for inclusion should respect our
`Ansible Style Guide`_.
.. _Ansible Style Guide: contribute.html#ansible-style-guide
Adding tests to a role
^^^^^^^^^^^^^^^^^^^^^^
Each of the role tests is in its tests/ folder.
This folder contains at least the following files:
#. ``test.yml`` ("super" playbook acting as test router to sub-playbooks)
#. ``<role name>-overrides.yml``. This var file is automatically loaded
by our shell script in our `tests repository`_.
#. ``inventory``. A static inventory for role testing.
It's possible some roles have multiple inventories. See for example the
neutron role with its ``lxb_inventory``, ``calico_inventory``.
#. ``group_vars`` and ``host_vars``. These folders will hold override the
necessary files for testing. For example, this is where you override
the IP addresses, IP ranges, and ansible connection details.
#. ``ansible-role-requirements.yml``. This should be fairly straightforward:
this file contains all the roles to clone before running your role.
The roles' relative playbooks will have to be listed in the ``test.yml``
file. However, keep in mind to NOT re-invent the wheel. For example,
if your role needs keystone, you don't need to create your own keystone
install playbook, because we have a generic keystone install playbook
in the `tests repository`.
.. _tests repository: https://git.openstack.org/cgit/openstack/openstack-ansible-tests
Ensuring the role matches OpenStack-Ansible's standards
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#. To facilitate the development and tests implemented across all
OpenStack-Ansible roles, a base set of folders and files need to be
implemented. A base set of configuration and test facilitation scripts must
include at least the following:
* ``tox.ini``:
The lint testing, documentation build, release note build and
functional build execution process for the role's gate tests are all
defined in this file.
* ``test-requirements.txt``:
The Python requirements that must be installed when executing the
tests.
* ``bindep.txt``:
The binary requirements that must be installed on the host the tests
are executed on for the Python requirements and the tox execution to
work. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``setup.cfg`` and ``setup.py``:
Information about the repository used when building artifacts.
* ``run_tests.sh``:
A script for developers to execute all standard tests on a
suitable host. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``Vagrantfile``:
A configuration file to allow a developer to easily create a
test virtual machine using `Vagrant`_. This must automatically execute
``run_tests.sh``. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``README.rst``, ``LICENSE``, ``CONTRIBUTING.rst``:
A set of standard files whose content is self-explanatory.
* ``.gitignore``:
A standard git configuration file for the repository which should be
pretty uniform across all the repositories. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``.gitreview``:
A standard file configured for the project to inform the ``git-review``
plugin where to find the upstream gerrit remote for the repository.
* ``tests/tests-repo-clone.sh`` needs to be copied from the
``openstack-ansible-tests`` repository.
Please have a look at a role like os_cinder, os_keystone, or os_neutron
for latest files.
#. The role development should initially be focused on implementing a set of
tasks and a test playbook which converge. The convergence must:
* Implement ``developer_mode`` to build from a git source into a Python
virtual environment.
* Deploy the applicable configuration files in the right places.
* Ensure that the service starts.
The convergence may involve consuming other OpenStack-Ansible roles (For
example: ``galera_server``, ``galera_client``, ``rabbitmq_server``) in
order to ensure that the appropriate infrastructure is in place. Re-using
existing roles in OpenStack-Ansible or Ansible Galaxy is strongly
encouraged.
#. The role *must* support Ubuntu 16.04 LTS. It should
ideally also support CentOS 7 and openSUSE 42.X but this is not required
at this time. The patterns to achieve this include:
* The separation of platform specific variables into role vars files.
* The detection and handling of different init systems (init.d, SystemD).
* The detection and handling of different package managers (apt, yum).
* The detection and handling of different network configuration methods.
There are several examples of these patterns implemented across many of
the OpenStack-Ansible roles. Developers are advised to inspect the
established patterns and either implement or improve upon them.
#. The role implementation should be done in such a way that it is agnostic
with regards to whether it is implemented in a container, or on a
physical host. The test infrastructure may make use of LXC containers for
the separation of services, but if a role is used by a playbook that
targets a host, it must work regardless of whether that host is a
container, a virtual server, or a physical server. The use of LXC
containers for role tests is not required but it may be useful in order
to simulate a multi-node build out as part of the testing infrastructure.
#. Any secrets (For example: passwords) should not be provided with default
values in the tasks, role vars, or role defaults. The tasks should be
implemented in such a way that any secrets required, but not provided,
should result in the task execution failure. It is important for a
secure-by-default implementation to ensure that an environment is not
vulnerable due to the production use of default secrets. Deployers
must be forced to properly provide their own secret variable values.
#. Once the initial convergence is working and the services are running,
the role development should focus on implementing some level of
functional testing. Ideally, the functional tests for an OpenStack role
should make use of Tempest to execute the functional tests. The ideal
tests to execute are scenario tests as they test the functions that
the service is expected to do in a production deployment. In the absence
of any scenario tests for the service a fallback option is to implement
the smoke tests instead.
#. The role must include documentation. The `Documentation and Release Note
Guidelines`_ provide specific guidelines with regards to style and
conventions. The documentation must include a description of the
mandatory infrastructure (For example: a database and a message queue are
required), variables (For example: the database name and credentials) and
group names (For example: The role expects a group named ``foo_all`` to
be present and it expects the host to be a member of it) for the role's
execution to succeed.
.. _Documentation and Release Note Guidelines: contribute.html#documentation-and-release-note-guidelines
.. _Vagrant: https://www.vagrantup.com/
Deploying the role
^^^^^^^^^^^^^^^^^^
#. Include your role on the deploy host. See also `Adding Galaxy roles`_.
#. Perform any other host preparation (such as the tasks performed by the
``bootstrap-aio.yml`` playbook). This includes any preparation tasks that
are particular to your service.
#. Generate files to include your service in the Ansible inventory
using `env.d`_ and `conf.d`_ files for use on your deploy host.
.. HINT:: You can follow examples from other roles, making the appropriate
modifications being sure that group labels in ``env.d`` and ``conf.d``
files are consistent.
.. HINT:: A description of how these work can be
found in :deploy_guide:`Appendix C <app-custom-layouts.html>`
of the Deployment Guide.
#. Generate secrets, if any, as described in the :deploy_guide:`Configure
Service Credentials <configure.html#configuring-service-credentials>`.
You can append your keys to an existing ``user_secrets.yml`` file or add a
new file to the ``openstack_deploy`` directory to contain them. Provide
overrides for any other variables you will need at this time as well, either
in ``user_variables.yml`` or another file. This is explained in more depth
under `Extending OpenStack-Ansible`_.
Any secrets required for the role to work must be noted in the
``etc/openstack_deploy/user_secrets.yml`` file for reuse by other users.
#. If your service is installed from source or relies on python packages which
need to be installed from source, specify a repository for the source
code of each requirement by adding a file to your deploy host under
``playbooks/defaults/repo_packages`` in the OpenStack-Ansible source
repository and following the pattern of files currently in that directory.
You could also simply add an entry to an existing file there. Be sure to
run the ``repo-build.yml`` play later so that wheels for your packages will
be included in the repository infrastructure.
#. Make any required adjustments to the load balancer configuration
(e.g. modify ``inventory/group_vars/all/haproxy.yml`` in the
OpenStack-Ansible source repository on your deploy host) so that your
service can be reached through a load balancer, if appropriate, and be sure
to run the ``haproxy-install.yml`` play later so your changes will be
applied. Please note, you can also use ``haproxy_extra_services`` variable
if you don't want to provide your service as default for everyone.
#. Put together a service install playbook file for your role. This can also
be modeled from any existing service playbook that has similar
dependencies to your service (database, messaging, storage drivers,
container mount points, etc.). A common place to keep playbook files in a
Galaxy role is in an ``examples`` directory off the root of the role.
If the playbook is meant for installing an OpenStack service, name it
``os-<service>-install.yml`` and target it at the appropriate
group defined in the service ``env.d`` file.
It is crucial that the implementation of the service is optional and
that the deployer must opt-in to the deployment through the population
of a host in the applicable host group. If the host group has no
hosts, Ansible skips the playbook's tasks automatically.
#. Any variables needed by other roles to connect to the new role, or by the
new role to connect to other roles, should be implemented in
``inventory/group_vars``. The group vars are essentially the
glue which playbooks use to ensure that all roles are given the
appropriate information. When group vars are implemented it should be a
minimum set to achieve the goal of integrating the new role into the
integrated build.
#. Documentation must be added in the role to describe how to implement
the new service in an integrated environement. This content must
adhere to the `Documentation and Release Note Guidelines`_. Until the
role has integrated functional testing implemented (see also the
Role development maturity paragraph), the documentation
must make it clear that the service inclusion in OpenStack-Ansible is
experimental and is not fully tested by OpenStack-Ansible in an
integrated build.
#. A feature release note must be added to announce the new service
availability and to refer to the role documentation for further
details. This content must adhere to the
`Documentation and Release Note Guidelines`_.
#. It must be possible to execute a functional, integrated test which
executes a deployment in the same way as a production environment. The
test must execute a set of functional tests using Tempest. This is the
required last step before a service can remove the experimental warning
from the documentation.
.. HINT:: If you adhere to the pattern of isolating your role's extra
deployment requirements (secrets and var files, HAProxy yml fragments,
repo_package files, etc.) in their own files it makes it easy for you to
automate these additional steps when testing your role.
.. _Adding Galaxy roles: extending.html#adding-galaxy-roles
.. _env.d: extending.html#env-d
.. _conf.d: extending.html#conf-d
.. _Extending OpenStack-Ansible: extending.html#user-yml-files
Role development maturity
-------------------------
A role may be fully mature, even if it is not integrated in the
``openstack-ansible`` repository.
A role can be in one of the four maturity levels:
* ``Complete``
* ``Incubated``
* ``Unmaintained``
* ``Retired``
Here are a series of rules that define maturity levels:
* A role can be retired at any time if it is not relevant anymore.
* A role can be ``Incubated`` for maximum 2 cycles.
* An ``Incubated`` role that passes functional testing will be upgraded
to the ``Complete`` status, and cannot return in ``Incubated`` status.
* An ``Incubated`` role that didn't implement functional testing in
the six month timeframe will become ``Unmaintained``.
* A role in ``Complete`` status can be downgraded to ``Unmaintained``.
status, according to the maturity downgrade procedure.
Maturity downgrade procedure
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If a role has failed periodics or gate test for two weeks, a bug
should be filed, and a message to the mailing list will be sent,
referencing the bug.
The next community meeting should discuss about role deprecation,
and if no contributor comes forward to fix the role, periodic
testing will be turned off, and the role will move to an
``unmaintained`` state.
Maturity Matrix
^^^^^^^^^^^^^^^
View the following role maturity table to see each role's status.
.. _role maturity table: role-maturity.html

67
doc/source/contributor/bug-triage.rst → doc/source/contributor/bugs.rst
View File

@ -1,9 +1,60 @@
============================
OpenStack-Ansible Bug Triage
============================
==============================
OpenStack-Ansible Bug Handling
==============================
.. _bug_reporting:
Bug Reporting
=============
Bugs should be filed on the `OpenStack-Ansible Launchpad project`_.
When submitting a bug, or working on a bug, please ensure the following
criteria are met:
* The description clearly states or describes the original problem or root
cause of the problem.
* The description clearly states the expected outcome of the user action.
* Include historical information on how the problem was identified.
* Any relevant logs or user configuration are included, either directly
or through a pastebin.
* If the issue is a bug that needs fixing in a branch other than master,
please note the associated branch within the launchpad issue.
* The provided information should be totally self-contained. External access
to web services/sites should not be needed.
* Steps to reproduce the problem if possible.
.. _OpenStack-Ansible Launchpad project: https://bugs.launchpad.net/openstack-ansible
Bug Tags
^^^^^^^^
If the reported needs fixing in a branch in addition to master, add a
'\<release\>-backport-potential' tag (e.g. ``liberty-backport-potential``).
There are predefined tags that will auto-complete.
Status
^^^^^^
Please leave the **status** of an issue alone until someone confirms it or
a member of the bugs team triages it. While waiting for the issue to be
confirmed or triaged the status should remain as **New**.
Importance
^^^^^^^^^^
Should only be touched if it is a Blocker/Gating issue. If it is, please
set to **High**, and only use **Critical** if you have found a bug that
can take down whole infrastructures. Once the importance has been changed
the status should be changed to **Triaged** by someone other than the bug
creator.
The triaging process is explained here below.
.. _bug_triage:
Bug triage
==========
What is a bug triage
~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^
"Bug triage is a process where tracker issues are screened and
prioritised. Triage should help ensure we appropriately manage all
@ -24,7 +75,7 @@ The bug triage practices in OpenStack-Ansible are based on the
.. _OpenStack Bug Triage Documentation: https://docs.openstack.org/infra/manual/developers.html#working-on-bugs
Bug classification
~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^
The **Status** of a bug is one of the following:
@ -53,7 +104,7 @@ The **Importance** of a bug is one of the following:
* *Undecided* - Not decided yet. It might need more discussion.
Bug triage meeting duties
~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^
If the bug description is incomplete, or the report is lacking the
information necessary to reproduce the issue, ask the reporter to
@ -75,7 +126,7 @@ supervisors rights to also prioritize bugs per importance (on top of
classifying them on status).
Bug skimming duty
~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^
To help triaging bugs, one person of the bug team can be on "bug
skimming duty".
@ -116,7 +167,7 @@ skimming duty".
an importance set.
Bug skimming duty weekly checklist
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Prioritize or reprioritize OpenStack-Ansible `confirmed bugs`_.

382
doc/source/contributor/code-rules.rst Normal file
View File

@ -0,0 +1,382 @@
.. _code_rules:
==========
Code rules
==========
.. _codeguidelines:
General Guidelines for Submitting Code
======================================
* Write good commit messages. We follow the OpenStack
"`Git Commit Good Practice`_" guide. if you have any questions regarding how
to write good commit messages please review the upstream OpenStack
documentation.
* Changes to the project should be submitted for review via the Gerrit tool,
following the `workflow documented here`_.
* Pull requests submitted through GitHub will be ignored and closed without
regard.
* Patches should be focused on solving one problem at a time. If the review is
overly complex or generally large the initial commit will receive a "**-2**"
and the contributor will be asked to split the patch up across multiple
reviews. In the case of complex feature additions the design and
implementation of the feature should be done in such a way that it can be
submitted in multiple patches using dependencies. Using dependent changes
should always aim to result in a working build throughout the dependency
chain. Documentation is available for `advanced gerrit usage`_ too.
* All patch sets should adhere to the :ref:`Ansible Style Guide` listed here as
well as adhere to the `Ansible best practices`_ when possible.
* All changes should be clearly listed in the commit message, with an
associated bug id/blueprint along with any extra information where
applicable.
* Refactoring work should never include additional "rider" features. Features
that may pertain to something that was re-factored should be raised as an
issue and submitted in prior or subsequent patches.
* New features, breaking changes and other patches of note must include a
release note generated using `the reno tool`_. Please see the
:ref:`Documentation and Release Note Guidelines<documentation>`
for more information.
* All patches including code, documentation and release notes should be built
and tested locally with the appropriate test suite before submitting for
review. See :ref:`Development and Testing<devel_and_testing>`
for more information.
.. _Git Commit Good Practice: https://wiki.openstack.org/wiki/GitCommitMessages
.. _workflow documented here: https://docs.openstack.org/infra/manual/developers.html#development-workflow
.. _advanced gerrit usage: http://www.mediawiki.org/wiki/Gerrit/Advanced_usage
.. _Ansible best practices: http://docs.ansible.com/playbooks_best_practices.html
.. _the reno tool: https://docs.openstack.org/developer/reno/usage.html
.. _documentation:
Documentation and Release Note Guidelines
=========================================
Documentation is a critical part of ensuring that the deployers of
OpenStack-Ansible are appropriately informed about:
* How to use the project's tooling effectively to deploy OpenStack.
* How to implement the right configuration to meet the needs of their specific
use-case.
* Changes in the project over time which may affect an existing deployment.
To meet these needs developers must submit
:ref:`code comments<codecomments>`, documentation (see also the
:ref:`documentation locations section<docslocations>`) and
:ref:`release notes<reno>` with any code submissions.
All forms of documentation should comply with the guidelines provided
in the `OpenStack Documentation Contributor
Guide`_, with particular reference to the following sections:
* Writing style
* RST formatting conventions
.. _OpenStack Documentation Contributor Guide: https://docs.openstack.org/contributor-guide/
.. _codecomments:
Code Comments
-------------
Code comments for variables should be used to explain the purpose of the
variable. This is particularly important for the role defaults file as the file
is included verbatim in the role's documentation. Where there is an optional
variable, the variable and an explanation of what it is used for should be
added to the defaults file.
Code comments for bash/python scripts should give guidance to the purpose of
the code. This is important to provide context for reviewers before the patch
has merged, and for later modifications to remind the contributors what the
purpose was and why it was done that way.
.. _docslocations:
Documentation locations
-----------------------
OpenStack-Ansible has multiple forms of documentation with different intent.
The :deploy_guide:`Deployment Guide <index.html>`
intends to help deployers deploy OpenStack-Ansible for the first time.
The :dev_docs:`User Guide <user/index.html>` intends to provide user
stories on how to do specific things with OpenStack-Ansible.
The :dev_docs:`Operations Guide <admin/index.html>` provide help
on how to manage and operate OpenStack-Ansible.
The in-depth technical information is located in the
:dev_docs:`OpenStack-Ansible Reference <reference/index.html>`.
The role documentation (for example, the `keystone role documentation`_)
intends to explain all the options available for the role and how to implement
more advanced requirements. To reduce duplication, the role documentation
directly includes the role's default variables file which includes the
comments explaining the purpose of the variables. The long hand documentation
for the roles should focus less on explaining variables and more on explaining
how to implement advanced use cases.
The role documentation must include a description of the mandatory
infrastructure (For example: a database and a message queue are required),
variables (For example: the database name and credentials) and group names
(For example: The role expects a group named ``foo_all`` to
be present and it expects the host to be a member of it) for the role's
execution to succeed.
Where possible the documentation in OpenStack-Ansible should steer clear of
trying to explain OpenStack concepts. Those explanations belong in the
OpenStack Manuals or service documentation and OpenStack-Ansible documentation
should link to those documents when available, rather than duplicate their
content.
.. _keystone role documentation: https://docs.openstack.org/developer/openstack-ansible-os_keystone/
.. _reno:
Release Notes
-------------
Release notes are generated using `the reno tool`_. Release notes must be
written with the following guidelines in mind:
* Each list item must make sense to read without the context of the patch or
the repository the patch is being submitted into. The reason for this is that
all release notes are consolidated and presented in a long list without
reference to the source patch or the context of the repository.
* Each note should be brief and to the point. Try to avoid multi-paragraph
notes. For features the note should typically refer to documentation for more
details. For bug fixes the note can refer to a registered bug for more
details.
In most cases only the following sections should be used for new release notes
submitted with patches:
* ``features``: This should inform the deployer briefly about a new feature and
should describe how to use it either by referencing the variables to set or
by referring to documentation.
* ``issues``: This should inform the deployer about known issues. This may be
used when fixing an issue and wanting to inform deployers about a workaround
that can be used for versions prior to that which contains the patch that
fixes the issue. Issue notes should specifically make mention of what
versions of OpenStack-Ansible are affected by the issue.
* ``upgrade``: This should inform the deployer about changes which may affect
them when upgrading from a previous major or minor version. Typically, these
notes would describe changes to default variable values or variables that
have been removed.
* ``deprecations``: If a variable has been deprecated (ideally using the
deprecation filter), then it should be communicated through notes in this
section. Note that if a variable has been removed entirely then it has not
been deprecated and the removal should be noted in the ``upgrade`` section.
.. _specs:
Submitting a specification
==========================
By proposing a draft spec you can help the OpenStack-Ansible
community keep track of what roles or large changes are being developed,
and perhaps connect you with others who may be interested and able
to help you in the process.
Our specifications repository follows the usual OpenStack and
OpenStack-Ansible guidelines for submitting code.
However, to help you in the writing of the specification, we have a
`specification template`_ that can be copied into the latest release
name folder. Rename and edit it for your needs.
.. _specification template: https://git.openstack.org/cgit/openstack/openstack-ansible-specs/tree/specs/templates/template.rst
.. _Ansible Style Guide:
Ansible Style Guide
===================
YAML formatting
---------------
When creating tasks and other roles for use in Ansible please create them
using the YAML dictionary format.
Example YAML dictionary format:
.. code-block:: yaml
- name: The name of the tasks
module_name:
thing1: "some-stuff"
thing2: "some-other-stuff"
tags:
- some-tag
- some-other-tag
Example what **NOT** to do:
.. code-block:: yaml
- name: The name of the tasks
module_name: thing1="some-stuff" thing2="some-other-stuff"
tags: some-tag
.. code-block:: yaml
- name: The name of the tasks
module_name: >
thing1="some-stuff"
thing2="some-other-stuff"
tags: some-tag
Usage of the ">" and "|" operators should be limited to Ansible conditionals
and command modules such as the Ansible ``shell`` or ``command``.
Tags and tags conventions
-------------------------
Tags are assigned based on the relevance of each individual item.
Higher level includes (for example in the ``tasks/main.yml``) need high
level tags. For example, ``*-config`` or ``*-install``.
Included tasks can have more detailed tags.
The following convention is used:
* A tag including the word ``install`` handles software installation tasks.
Running a playbook with ``--tags <role>-install`` only deploys the
necessary software on the target, and will not configure it to your
needs or run any service.
* A tag including the word ``config`` prepares the configuration of the
software (adapted to your needs), and all the components necessary
to run the service(s) configured in the role. Running a playbook with
``--tags <role>-config`` is only possible if the target already ran
the tags ``<role>-install``.
* A tag including the word ``upgrade`` handles all the upgrade tasks.
Variable files conventions
--------------------------
The variables files in a role are split in 3 locations:
#. The `defaults/main.yml` file
#. The `vars/main.yml` file
#. The `vars/<platform specific>.yml` file
The variables with lower priority should be in the `defaults/main.yml`.
This allows their overriding with group variables or host variables.
A good example for this are default database connection details, default
queues connection details, or debug mode.
In other words, `defaults/main.yml` contains variables that are meant to
be overridable by a deployer or a continuous integration system.
These variables should be limited as much as possible, to avoid
increasing the test matrix.
The `vars/main.yml` is always included. It contains generic
variables that aren't meant to be changed by a deployer. This includes
for example static information that aren't distribution specific (like
aggregation of role internal variables for example).
The `vars/<platform specific>.yml` is the place where distribution
specific content will be stored. For example, this file will hold
the package names, repositories urls and keys, file paths, service
names/init scripts.
Secrets
^^^^^^^
Any secrets (For example: passwords) should not be provided with default
values in the tasks, role vars, or role defaults. The tasks should be
implemented in such a way that any secrets required, but not provided,
should result in the task execution failure. It is important for a
secure-by-default implementation to ensure that an environment is not
vulnerable due to the production use of default secrets. Deployers
must be forced to properly provide their own secret variable values.
Task files conventions
----------------------
Most OpenStack services will follow a common series of stages to
install, configure, or update a service deployment. This is apparent
when you review `tasks/main.yml` for existing roles.
If developing a new role, please follow the conventions set by
existing roles.
Tests conventions
-----------------
The conventions for writing tests are described in the
:ref:`tests` page.
Other OpenStack-Ansible conventions
-----------------------------------
To facilitate the development and tests implemented across all
OpenStack-Ansible roles, a base set of folders and files need to be
implemented. A base set of configuration and test facilitation scripts must
include at least the following:
* ``tox.ini``:
The lint testing, documentation build, release note build and
functional build execution process for the role's gate tests are all
defined in this file.
* ``test-requirements.txt``:
The Python requirements that must be installed when executing the
tests.
* ``bindep.txt``:
The binary requirements that must be installed on the host the tests
are executed on for the Python requirements and the tox execution to
work. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``setup.cfg`` and ``setup.py``:
Information about the repository used when building artifacts.
* ``run_tests.sh``:
A script for developers to execute all standard tests on a
suitable host. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``Vagrantfile``:
A configuration file to allow a developer to easily create a
test virtual machine using `Vagrant`_. This must automatically execute
``run_tests.sh``. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``README.rst``, ``LICENSE``, ``CONTRIBUTING.rst``:
A set of standard files whose content is self-explanatory.
* ``.gitignore``:
A standard git configuration file for the repository which should be
pretty uniform across all the repositories. This must be copied from the
``openstack-ansible-tests`` repository and will be automatically
be overriden by our proposal bot should any change happen.
* ``.gitreview``:
A standard file configured for the project to inform the ``git-review``
plugin where to find the upstream gerrit remote for the repository.
* ``docs/`` and ``releasenotes/`` folders need to be exist and be
properly configured.
Please have a look at a role like os_cinder, os_keystone, or os_neutron
for latest files.
.. _Vagrant: https://www.vagrantup.com/
Container technology independence
---------------------------------
The role implementation should be done in such a way that it is agnostic
with regards to whether it is implemented in a container, or on a
physical host. The test infrastructure may make use of containers for
the separation of services, but if a role is used by a playbook that
targets a host, it must work regardless of whether that host is a
container, a virtual server, or a physical server. The use of
containers for role tests is not required but it may be useful in order
to simulate a multi-node build out as part of the testing infrastructure.
Minimum supported distributions
-------------------------------
See our :ref:`supported-distros` page.

485
doc/source/contributor/contribute.rst
View File

@ -1,97 +1,83 @@
.. _contributing:
======================
Contributor Guidelines
======================
Reporting Bugs
~~~~~~~~~~~~~~
Before submitting code
======================
Bugs should be filed on `Bug Launchpad`_ for OpenStack-Ansible.
Before jumping ahead and working on code, a series of steps should
be taken:
When submitting a bug, or working on a bug, please ensure the following
criteria are met:
* Is there a bug for it? Can your track if someone else has seen
the same bug?
* Are you sure nobody is working on this problem at the moment?
Could there be a review pending fixing the same issue?
* Have you checked if your issue/feature request
hasn't been solved in another branch?
* The description clearly states or describes the original problem or root
cause of the problem.
* Include historical information on how the problem was identified.
* Any relevant logs are included.
* If the issue is a bug that needs fixing in a branch other than master,
please note the associated branch within the launchpad issue.
* The provided information should be totally self-contained. External access
to web services/sites should not be needed.
* Steps to reproduce the problem if possible.
If you're willing to submit code, please remember the following rules:
Tags
----
If it's a bug that needs fixing in a branch in addition to master, add a
'\<release\>-backport-potential' tag (e.g. ``liberty-backport-potential``).
There are predefined tags that will auto-complete.
* All code should match our
:ref:`codeguidelines`.
* All code requires to go through our :ref:`reviews`.
* Documentation should be provided with the
code directly. See also :ref:`documentation`.
* Fixing bugs and increasing test coverage have priority to new features.
See also the section :ref:`bugfixing`.
* New features are following a process, explained in the section
:ref:`newfeatures`.
New features are less likely to be :ref:`backported<backport>`
to previous branches.
Status
------
Please leave the **status** of an issue alone until someone confirms it or
a member of the bugs team triages it. While waiting for the issue to be
confirmed or triaged the status should remain as **New**.
.. _reviews:
Importance
----------
Should only be touched if it is a Blocker/Gating issue. If it is, please
set to **High**, and only use **Critical** if you have found a bug that
can take down whole infrastructures. Once the importance has been changed
the status should be changed to *Triaged* by someone other than the bug
creator.
Review process
==============
The triaging process is explained on the `bug triage documentation`_ page.
Any new code will be reviewed before merging into our repositories.
.. _Bug Launchpad: https://bugs.launchpad.net/openstack-ansible
.. _bug triage documentation: bug-triage.html
We follow openstack guidelines for the code reviewing process.
General Guidelines for Submitting Code
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Please be aware that any patch can be refused by the community if they
don't match the :ref:`codeguidelines`.
* Write good commit messages. We follow the OpenStack
"`Git Commit Good Practice`_" guide. if you have any questions regarding how
to write good commit messages please review the upstream OpenStack
documentation.
* Changes to the project should be submitted for review via the Gerrit tool,
following the `workflow documented here`_.
* Pull requests submitted through GitHub will be ignored and closed without
regard.
* Patches should be focused on solving one problem at a time. If the review is
overly complex or generally large the initial commit will receive a "**-2**"
and the contributor will be asked to split the patch up across multiple
reviews. In the case of complex feature additions the design and
implementation of the feature should be done in such a way that it can be
submitted in multiple patches using dependencies. Using dependent changes
should always aim to result in a working build throughout the dependency
chain. Documentation is available for `advanced gerrit usage`_ too.
* All patch sets should adhere to the `Ansible Style Guide`_ listed here as
well as adhere to the `Ansible best practices`_ when possible.
* All changes should be clearly listed in the commit message, with an
associated bug id/blueprint along with any extra information where
applicable.
* Refactoring work should never include additional "rider" features. Features
that may pertain to something that was re-factored should be raised as an
issue and submitted in prior or subsequent patches.
* New features, breaking changes and other patches of note must include a
release note generated using `the reno tool`_. Please see the
`Documentation and Release Note Guidelines`_ for more information.
* All patches including code, documentation and release notes should be built
and tested locally with the appropriate test suite before submitting for
review. See `Development and Testing`_ for more information.
.. _bugfixing:
.. _Git Commit Good Practice: https://wiki.openstack.org/wiki/GitCommitMessages
.. _workflow documented here: https://docs.openstack.org/infra/manual/developers.html#development-workflow
.. _advanced gerrit usage: http://www.mediawiki.org/wiki/Gerrit/Advanced_usage
.. _Ansible best practices: http://docs.ansible.com/playbooks_best_practices.html
.. _the reno tool: https://docs.openstack.org/developer/reno/usage.html
.. _Development and Testing: scripts.html#development-and-testing
Working on bug fixes
====================
Working on Features
~~~~~~~~~~~~~~~~~~~
Any bug fix should have, in its commit message:
* All feature additions/deletions should be accompanied by a blueprint/spec.
e.g. adding additional active agents to neutron, developing a new service
role, etc...
Closes-Bug: #bugnumber
or
Related-Bug: #bugnumber
where #bugnumber refers to a Launchpad issue.
See also the `working on bugs`_ section of the openstack documentation.
.. _working on bugs: https://docs.openstack.org/infra/manual/developers.html#working-on-bugs
.. _newfeatures:
Working on new features
=======================
If you would like to contribute towards a role to introduce an OpenStack
or infrastructure service, or to improve an existing role, the
OpenStack-Ansible project would welcome that contribution and your assistance
in maintaining it.
Here are a few rules to get started:
* All large feature additions/deletions should be accompanied by a
blueprint/spec. e.g. adding additional active agents to neutron,
developing a new service role, etc... See also
:ref:`specs`.
* Before creating blueprint/spec an associated 'Wishlist Bug' can be raised on
launchpad. This issue will be triaged and a determination will be made on
how large the change is and whether or not the change warrants a
@ -101,12 +87,13 @@ Working on Features
* All blueprints/specs should be voted on and approved by core reviewers
before any associated code will be merged. For more information on
blueprints/specs please review the OpenStack documentation regarding
`Working on Specifications and Blueprints`_.
`Working on Specifications and Blueprints`_ and our own
:ref:`specs`.
* Once the blueprint work is completed the author(s) can request a backport
of the blueprint work into a stable branch. Each backport will be evaluated
on a case by case basis with cautious consideration based on how the
backport affects any existing deployments. See the `Backporting`_ section
for more information.
backport affects any existing deployments. See the
:ref:`backport` section for more information.
* Any new OpenStack services implemented which have `Tempest`_ tests
available must be implemented along with suitable functional tests enabled
as part of the feature development in order to ensure that any changes
@ -115,12 +102,110 @@ Working on Features
OpenStack documentation about what the feature is and how it works. The
documentation should then describe how it is implemented in
OpenStack-Ansible and what configuration options there are.
See also the :ref:`documentation` section.
.. _Working on Specifications and Blueprints: https://docs.openstack.org/infra/manual/developers.html#working-on-specifications-and-blueprints
.. _Tempest: https://docs.openstack.org/developer/tempest/
Example process to develop a new role
-------------------------------------
Here are the steps to write the role:
#. You can review roles which may be currently in development by checking our
`specs repository`_ and `unmerged specs`_ on review.openstack.org. If you
do not find a spec for the role, propose a blueprint/spec.
See also :ref:`specs`.
#. Create a source repository (e.g. on Github) to start your work on the Role.
#. Generate the reference directory structure for an Ansible role which is
the necessary subset of the documented `Best Practice`_. You might use
Ansible Galaxy tools to do this for you (e.g. ``ansible-galaxy init``).
You may additionally want to include directories such as ``docs`` and
``examples`` and ``tests`` for your role.
#. Generate a meta/main.yml right away. This file is important to Ansible to
ensure your dependent roles are installed and available and provides others
with the information they will need to understand the purpose of your role.
#. Develop task files for each of the install stages in turn, creating any
handlers and templates as needed. Ensure that you notify handlers after any
task which impacts the way the service would run (such as configuration
file modifications). Also take care that file ownership and permissions are
appropriate.
.. HINT:: Fill in variable defaults, libraries, and prerequisites as you
discover a need for them. You can also develop documentation for your
role at the same time.
#. Add tests to the role. See also our :ref:`tests` page.
#. Ensuring the role matches OpenStack-Ansible's latest standards.
See also our :ref:`code_rules` page.
#. Ensure the role converges:
* Implement **developer_mode** to build from a git source into
a Python virtual environment.
* Deploy the applicable configuration files in the right places.
* Ensure that the service starts.
The convergence may involve consuming other OpenStack-Ansible roles
(For example: **galera_server, galera_client, rabbitmq_server**)
in order to ensure that the appropriate infrastructure is in place.
Re-using existing roles in OpenStack-Ansible or Ansible Galaxy is
strongly encouraged.
#. Once the initial convergence is working and the services are running,
the role development should focus on implementing some level of
functional testing. See also :ref:`tempest-testing`.
#. Test the role on a new machine, using our provided scripts.
#. Submit your role for review.
#. If required, ask the OpenStack-Ansible PTL to import the github
role into the openstack-ansible namespace (This can only be done
early in the development cycle, and may be postponed to next
cycle).
#. If necessary, work on the integration within the
openstack-ansible integrated repository, and deploy
the role on an AIO. See also :ref:`integrate-new-role-with-aio`.
.. _specs repository: https://git.openstack.org/cgit/openstack/openstack-ansible-specs
.. _unmerged specs: https://review.openstack.org/#/q/status:+open+project:openstack/openstack-ansible-specs
.. _Best Practice: https://docs.ansible.com/ansible/playbooks_best_practices.html#directory-layout
Example process for adding a feature to an existing role
--------------------------------------------------------
#. Search for in the `OpenStack-Ansible Launchpad project`_ for
the feature request.
#. If no "Wishlist" item exist in Launchpad for your feature, create
a bug for it. Don't hesitate to ask if a spec is required in
the bug.
#. The :ref:`bug_triage` will classify if this new feature requires
a spec or not.
#. Work on the role files, following our :ref:`code_rules`.
#. Add an extra role test scenario, to ensure your code path is
tested and working.
#. Test your new scenario with a new machine.
See also the :ref:`devel_and_testing` page.
#. Submit your code for review, with its necessary documentation and
release notes.
.. _OpenStack-Ansible Launchpad project: https://bugs.launchpad.net/openstack-ansible
Example process to incubate a new "ops" project
-----------------------------------------------
A new project in "openstack-ansible-ops" can be started at any time,
with no constraint like writing a specification, or creating a bug.
Instead, the new code has to be isolated on a separate folder of the
`openstack-ansible-ops repo`_.
.. _openstack-ansible-ops repo: https://git.openstack.org/cgit/openstack/openstack-ansible-ops
.. _backport:
Backporting
~~~~~~~~~~~
===========
* Backporting is defined as the act of reproducing a change from another
branch. Unclean/squashed/modified cherry-picks and complete
@ -135,7 +220,7 @@ Backporting
commit message. This can be done with ``cherry-pick -x``. Here's more
information on `Submitting a change to a branch for review`_.
* Every backport commit must still only solve one problem, as per the
guidelines in `General Guidelines for Submitting Code`_.
guidelines in :ref:`codeguidelines`.
* If a backport is a squashed set of cherry-picked commits, the original SHAs
should be referenced in the commit message and the reason for squashing the
commits should be clearly explained.
@ -148,239 +233,3 @@ Backporting
.. _Submitting a change to a branch for review: http://www.mediawiki.org/wiki/Gerrit/Advanced_usage#Submitting_a_change_to_a_branch_for_review_.28.22backporting.22.29
.. _OpenStack Guidelines for stable branches: https://docs.openstack.org/project-team-guide/stable-branches.html
Documentation and Release Note Guidelines
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Documentation is a critical part of ensuring that the deployers of
OpenStack-Ansible are appropriately informed about:
* How to use the project's tooling effectively to deploy OpenStack.
* How to implement the right configuration to meet the needs of their specific
use-case.
* Changes in the project over time which may affect an existing deployment.
To meet these needs developers must submit code comments, documentation and
release notes with any code submissions. All forms of documentation should
comply with the guidelines provided in the `OpenStack Documentation Contributor
Guide`_, with particular reference to the following sections:
* Writing style
* RST formatting conventions
.. _OpenStack Documentation Contributor Guide: https://docs.openstack.org/contributor-guide/
Code Comments
-------------
Code comments for variables should be used to explain the purpose of the
variable. This is particularly important for the role defaults file as the file
is included verbatim in the role's documentation. Where there is an optional
variable, the variable and an explanation of what it is used for should be
added to the defaults file.
Code comments for bash/python scripts should give guidance to the purpose of
the code. This is important to provide context for reviewers before the patch
has merged, and for later modifications to remind the contributors what the
purpose was and why it was done that way.
Documentation
-------------
OpenStack-Ansible has multiple forms of documentation with different intent.
.. note::
The statements below regarding the Install Guide and Role Documentation are
statements of intent. The work to fulfill the intent is ongoing. Any new
documentation submissions should try to help this intent where possible.
The `Deployment Guide <https://docs.openstack.org/project-deploy-guide/openstack-ansible>`_
intends to help deployers deploy OpenStack-Ansible for the first time.
The role documentation (for example, the `keystone role documentation`_)
intends to explain all the options available for the role and how to implement
more advanced requirements. To reduce duplication, the role documentation
directly includes the role's default variables file which includes the
comments explaining the purpose of the variables. The long hand documentation
for the roles should focus less on explaining variables and more on explaining
how to implement advanced use cases.
Where possible the documentation in OpenStack-Ansible should steer clear of
trying to explain OpenStack concepts. Those explanations belong in the
OpenStack Manuals or service documentation and OpenStack-Ansible documentation
should link to those documents when available, rather than duplicate their
content.
.. _keystone role documentation: https://docs.openstack.org/developer/openstack-ansible-os_keystone/
Release Notes
-------------
Release notes are generated using `the reno tool`_. Release notes must be
written with the following guidelines in mind:
* Each list item must make sense to read without the context of the patch or
the repository the patch is being submitted into. The reason for this is that
all release notes are consolidated and presented in a long list without
reference to the source patch or the context of the repository.
* Each note should be brief and to the point. Try to avoid multi-paragraph
notes. For features the note should typically refer to documentation for more
details. For bug fixes the note can refer to a registered bug for more
details.
In most cases only the following sections should be used for new release notes
submitted with patches:
* ``features``: This should inform the deployer briefly about a new feature and
should describe how to use it either by referencing the variables to set or
by referring to documentation.
* ``issues``: This should inform the deployer about known issues. This may be
used when fixing an issue and wanting to inform deployers about a workaround
that can be used for versions prior to that which contains the patch that
fixes the issue. Issue notes should specifically make mention of what
versions of OpenStack-Ansible are affected by the issue.
* ``upgrade``: This should inform the deployer about changes which may affect
them when upgrading from a previous major or minor version. Typically, these
notes would describe changes to default variable values or variables that
have been removed.
* ``deprecations``: If a variable has been deprecated (ideally using the
deprecation filter), then it should be communicated through notes in this
section. Note that if a variable has been removed entirely then it has not
been deprecated and the removal should be noted in the ``upgrade`` section.
Ansible Style Guide
~~~~~~~~~~~~~~~~~~~
YAML formatting
---------------
When creating tasks and other roles for use in Ansible please create them
using the YAML dictionary format.
Example YAML dictionary format:
.. code-block:: yaml
- name: The name of the tasks
module_name:
thing1: "some-stuff"
thing2: "some-other-stuff"
tags:
- some-tag
- some-other-tag
Example what **NOT** to do:
.. code-block:: yaml
- name: The name of the tasks
module_name: thing1="some-stuff" thing2="some-other-stuff"
tags: some-tag
.. code-block:: yaml
- name: The name of the tasks
module_name: >
thing1="some-stuff"
thing2="some-other-stuff"
tags: some-tag
Usage of the ">" and "|" operators should be limited to Ansible conditionals
and command modules such as the Ansible ``shell`` or ``command``.
Tags and tags conventions
-------------------------
Tags are assigned based on the relevance of each individual item.
Higher level includes (for example in the ``tasks/main.yml``) need high
level tags. For example, ``*-config`` or ``*-install``.
Included tasks can have more detailed tags.
The following convention is used:
* A tag including the word ``install`` handles software installation tasks.
Running a playbook with ``--tags <role>-install`` only deploys the
necessary software on the target, and will not configure it to your
needs or run any service.
* A tag including the word ``config`` prepares the configuration of the
software (adapted to your needs), and all the components necessary
to run the service(s) configured in the role. Running a playbook with
``--tags <role>-config`` is only possible if the target already ran
the tags ``<role>-install``.
* A tag including the word ``upgrade`` handles all the upgrade tasks.
Variable files conventions
--------------------------
The variables files in a role are split in 3 locations:
#. The `defaults/main.yml` file
#. The `vars/main.yml` file
#. The `vars/<platform specific>.yml` file
The variables with lower priority should be in the `defaults/main.yml`.
This allows their overriding with group variables or host variables.
A good example for this are default database connection details, default
queues connection details, or debug mode.
In other words, `defaults/main.yml` contains variables that are meant to
be overridable by a deployer or a continuous integration system.
These variables should be limited as much as possible, to avoid
increasing the test matrix.
The `vars/main.yml` is always included. It contains generic
variables that aren't meant to be changed by a deployer. This includes
for example static information that aren't distribution specific (like
aggregation of role internal variables for example).
The `vars/<platform specific>.yml` is the place where distribution
specific content will be stored. For example, this file will hold
the package names, repositories urls and keys, file paths, service
names/init scripts.
Development cycle checklist
~~~~~~~~~~~~~~~~~~~~~~~~~~~
On top of the normal cycle goals, a contributor can help the OpenStack-Ansible
development team by performing one of the following recurring tasks:
* By milestone 1:
* Community goal acknowledgement.
* By milestone 2:
* Handle deprecations from upstream project's previous cycle.
* Handle OpenStack-Ansible roles deprecations from the previous cycle.
* Refresh static elements in roles. For example, update a specific version of
the software packages.
* Bump ``ceph_stable_release`` to latest Ceph LTS release in the integrated
OpenStack-Ansible repo, and inside the ``ceph_client`` role defaults.
* Check and bump galera versions if required.
* Check and bump rabbitmq versions if required.
* Check outstanding reviews and move to merge or abandon them if no longer
valid.
* By milestone 3:
* Implement features
* After milestone 3:
* Feature freeze, bug fixes, and testing improvements
* After official project release, before official OpenStack-Ansible release:
* Bump RDO, Ubuntu Cloud Archive and openSUSE OBS OpenStack Cloud
repositories if they are ready on time.

10
doc/source/developer-docs/distributions.rst → doc/source/contributor/distributions.rst
View File

@ -2,14 +2,16 @@
Distribution support
====================
.. _supported-distros:
Supported distributions
~~~~~~~~~~~~~~~~~~~~~~~
=======================
The list of supported distributions can be found in the
:deploy_guide:`Deployment Guide <deploymenthost.html>`
Minimum requirements for OpenStack-Ansible roles
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
================================================
Existing and new distributions are expected to meet the following requirements
in order for them to be accepted in the individual OpenStack-Ansible roles:
@ -17,7 +19,7 @@ in order for them to be accepted in the individual OpenStack-Ansible roles:
* Pass functional tests
Graduation
~~~~~~~~~~
==========
For a distribution to be considered supported by the OpenStack-Ansible
project, it has to meet the following minimum requirements:
@ -28,7 +30,7 @@ project, it has to meet the following minimum requirements:
Temptest testing framework.
Voting
~~~~~~
======
Distributions can be promoted to voting jobs on individual roles once they move
to the `Graduation` phase and their stability has been confirmed by the core

9
doc/source/contributor/index.rst
View File

@ -24,11 +24,14 @@ Contents:
.. toctree::
:maxdepth: 2
project-onboarding
bugs
contribute
code-rules
testing
periodic-work
inventory-and-vars
scripts
contribute
bug-triage
core-reviewers
role-services
distributions

112
doc/source/contributor/periodic-work.rst Normal file
View File

@ -0,0 +1,112 @@
.. _periodicwork:
=============
Periodic Work
=============
Releasing
=========
Our release schedule for stable branches is around every two weeks:
* We request an OpenStack release on the second Friday of the month
* We request an OpenStack release on the last Friday of the month
The releases take generally a few days to get "tagged" and be publicly
available.
Dependency Updates
------------------
The dependencies for OpenStack-Ansible are updated
through the use of ``scripts/sources-branch-updater.sh``. This script
updates all pinned SHA's for OpenStack services, OpenStack-Ansible roles,
and other python dependencies which are not handled by the OpenStack global
requirements management process. This script also updates the statically
held templates/files in each role to ensure that they are always up to date.
Finally, it also increments the patch version of the
``openstack_release`` variable.
The update script is used as follows:
.. parsed-literal::
# change directory to the openstack-ansible checkout
cd ~/code/openstack-ansible
# ensure that the correct branch is checked out
git checkout |current_release_git_branch_name|
# ensure that the branch is up to date
git pull
# create the local branch for the update
git checkout -b sha-update
# execute the script for all openstack services
./scripts/sources-branch-updater.sh -b |current_release_git_branch_name| -o |current_release_git_branch_name|
# execute the script for gnocchi
./scripts/sources-branch-updater.sh -s playbooks/defaults/repo_packages/gnocchi.yml -b |current_release_gnocchi_git_branch_name| -o |current_release_git_branch_name|
# the console code should only be updated when necessary for a security fix, or for the OSA master branch
./scripts/sources-branch-updater.sh -s playbooks/defaults/repo_packages/nova_consoles.yml -b master
# the testing repositories should not be updated for stable branches as the new tests
# or other changes introduced may not work for older branches
./scripts/sources-branch-updater.sh -s playbooks/defaults/repo_packages/openstack_testing.yml -b master
# commit the changes
new_version=$(awk '/^openstack_release/ {print $2}' inventory/group_vars/all/all.yml)
git add --all
git commit -a -m "Update all SHAs for ${new_version}" \
-m "This patch updates all the roles to the latest available stable
SHA's, copies the release notes from the updated roles into the
integrated repo, updates all the OpenStack Service SHA's, and
updates the appropriate python requirements pins.
# push the changes up to gerrit
git review
Development cycle checklist
===========================
On top of the normal cycle goals, a contributor can help the OpenStack-Ansible
development team by performing one of the following recurring tasks:
* By milestone 1:
* Community goal acknowledgement.
* By milestone 2:
* Handle deprecations from upstream project's previous cycle.
* Handle OpenStack-Ansible roles deprecations from the previous cycle.
* Refresh static elements in roles. For example, update a specific version of
the software packages.
* Bump ``ceph_stable_release`` to latest Ceph LTS release in the integrated
OpenStack-Ansible repo, and inside the ``ceph_client`` role defaults.
* Check and bump galera versions if required.
* Check and bump rabbitmq versions if required.
* Check outstanding reviews and move to merge or abandon them if no longer
valid.
* By milestone 3:
* Implement features
* After milestone 3:
* Feature freeze, bug fixes, and testing improvements
* After official project release, before official OpenStack-Ansible release:
* Bump RDO, Ubuntu Cloud Archive and openSUSE OBS OpenStack Cloud
repositories if they are ready on time.

118
doc/source/contributor/project-onboarding.rst Normal file
View File

@ -0,0 +1,118 @@
==================
Project Onboarding
==================
This document should help you understand how to contribute to
OpenStack-Ansible.
Project repositories
====================
The OpenStack-Ansible project has different kinds of git repositories,
each of them with specific use cases, and different sets of practices.
.. list-table::
:header-rows: 1
* - Repository type or name
- Code location
- Repository purpose
* - | **OpenStack-Ansible**
| Also called *integrated repository*
- * https://github.com/openstack/openstack-ansible
- Our main repository, used by deployers.
Uses the other repositories.
* - | The **OpenStack-Ansible roles** repositories
- * https://github.com/openstack/openstack-ansible-os_nova
* https://github.com/openstack/openstack-ansible-os_glance
* https://github.com/openstack/ansible-role-systemd_mount
* https://github.com/openstack/ansible-config_template
* https://github.com/openstack/ansible-hardening
* ...
- Each role is in charge of deploying **exactly one**
component of an OpenStack-Ansible deployment.
* - | The **tests repository**
- * https://github.com/openstack/openstack-ansible-tests
- | The tests repository is the location for common code used in
the integrated repo and role repos tests.
| It allows us to not repeat ourselves: it is the location of
common playbooks, common tasks and scripts.
* - | The **specs** repository
- * https://github.com/openstack/openstack-ansible-specs
- This repository contains all the information concerning
large bodies of work done in OpenStack-Ansible,
split by cycle.
* - | The **ops** repository
- * https://github.com/openstack/openstack-ansible-ops
- This repository is an incubator for new projects, each project
solving a particular operational problem. Each project has its
own folder in this repository.
* - | External repositories
- * https://github.com/ceph/ceph-ansible
* https://github.com/logan2211/ansible-resolvconf
* https://github.com/willshersystems/ansible-sshd
* https://github.com/opendaylight/integration-packaging-ansible-opendaylight
* https://github.com/evrardjp/ansible-keepalived
* ...
- OpenStack-Ansible is not re-inventing the wheel, and tries to
reuse as much as possible existing roles. A bugfix for one of
those repositories must be handled to these repositories'
maintainers.
How to contribute on code or issues
===================================
* For contributing code and documentation, you must follow the
OpenStack practices. Nothing special is required for OpenStack-Ansible.
See also the `OpenStack developers getting started page`_.
and our :ref:`contributor guidelines<contributing>` before hacking.
* For helping on or submitting bugs, you must have an account on
ubuntu Launchpad.
All our repositories share the same `Launchpad project`_.
Please check our :ref:`bug report<bug_reporting>` and
:ref:`bug triage<bug_triage>` processes.
Easy to fix bugs are marked with the tag *low hanging fruit*, and
should be the target of first time contributors.
* For sharing your user experience, stories, and helping other users,
please join us in our :ref:`IRC channel<irc>`.
* The OpenStack-Ansible project has recurring tasks that need
attention, like releasing, or other code duties.
See our page :ref:`Periodic work<periodicwork>`.
.. _OpenStack developers getting started page: https://docs.openstack.org/infra/manual/developers.html#getting-started
.. _Launchpad project: https://bugs.launchpad.net/openstack-ansible
Community communication channels
================================
.. _irc:
IRC channel
^^^^^^^^^^^
The OpenStack-Ansible community communicates a lot through IRC, in
the #openstack-ansible channel, on freenode. This channel is
logged, and its logs are published
`here <http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/>`_.
Weekly meetings are held in our IRC channel. The schedule and
logs can be found `here<http://eavesdrop.openstack.org/#OpenStack_Ansible_Deployment_Meeting>`_.
Next meeting agenda can be found on our
`Meetings wiki page <https://wiki.openstack.org/wiki/Meetings/openstack-ansible>`_.
Mailing lists
^^^^^^^^^^^^^
A member of the OpenStack-Ansible community should monitor the
**OpenStack-dev** and **OpenStack-operators** `mailing lists`_.
.. _mailing lists: http://lists.openstack.org/cgi-bin/mailman/listinfo
All our communications should be prefixed with **[openstack-ansible]**.

11
doc/source/contributor/role-maturity.rst
View File

@ -1,11 +0,0 @@
=============
Role maturity
=============
All of the OpenStack-Ansible roles do not have the same level of maturity and
testing.
Here is a dashboard of the current status of the roles:
.. raw:: html
:file: role-maturity-matrix.html

14
doc/source/contributor/role-services.rst
View File

@ -1,14 +0,0 @@
==================
Roles and services
==================
If you would like to contribute towards a role to introduce an OpenStack
service and/or an infrastructure service to support an OpenStack deployment,
the OpenStack-Ansible project would welcome that contribution and your
assistance in maintaining it.
.. toctree::
:maxdepth: 2
additional-roles
role-maturity

244
doc/source/contributor/scripts.rst
View File

@ -1,244 +0,0 @@
================
Included Scripts
================
The repository contains several helper scripts to manage gate jobs,
install base requirements, and update repository information. Execute
these scripts from the root of the repository clone. For example:
.. code:: bash
$ scripts/<script_name>.sh
Bootstrapping
^^^^^^^^^^^^^
bootstrap-ansible.sh
--------------------
The ``bootstrap-ansible.sh`` script installs Ansible, including the `core`_ and
`extras`_ module repositories and Galaxy roles.
While there are several configurable environment variables which this script
uses, the following are commonly used:
* ``ANSIBLE_PACKAGE`` - The version of Ansible to install.
For example:
.. code:: bash
$ export ANSIBLE_PACKAGE="ansible==2.1.0"
Installing directly from git is also supported. For example, from the tip of
Ansible development branch:
.. code:: bash
$ export ANSIBLE_PACKAGE="git+https://github.com/ansible/ansible@devel#egg=ansible"
* ``ANSIBLE_ROLE_FILE`` - The location of a YAML file which ansible-galaxy can
consume which specifies which roles to download and install. The default
value for this is ``ansible-role-requirements.yml``.
The script also creates the ``openstack-ansible`` wrapper tool that provides
the variable files to match ``/etc/openstack_deploy/user_*.yml`` as
arguments to ``ansible-playbook`` as a convenience.
.. _core: https://github.com/ansible/ansible-modules-core
.. _extras: https://github.com/ansible/ansible-modules-extras
bootstrap-aio.sh
----------------
The ``bootstrap-aio.sh`` script prepares a host for an
:ref:`All-In-One <quickstart-aio>` (AIO)
deployment for the purposes of development and gating. The script creates the
necessary partitions, directories, and configurations. The script can be
configured using environment variables - more details are provided on the
:ref:`All-In-One <quickstart-aio>` page.
Development and Testing
^^^^^^^^^^^^^^^^^^^^^^^
Lint Tests
----------
Python coding conventions are tested using `PEP8`_, with the following
convention exceptions:
* F403 - 'from ansible.module_utils.basic import \*'
Testing may be done locally by executing:
.. code-block:: bash
tox -e pep8
Bash coding conventions are tested using `Bashate`_, with the following
convention exceptions:
* E003: Indent not multiple of 4. We prefer to use multiples of 2 instead.
* E006: Line longer than 79 columns. As many scripts are deployed as templates
and use jinja templating, this is very difficult to achieve. It is
still considered a preference and should be a goal to improve
readability, within reason.
* E040: Syntax error determined using `bash -n`. As many scripts are deployed
as templates and use jinja templating, this will often fail. This
test is reasonably safely ignored as the syntax error will be
identified when executing the resulting script.
Testing may be done locally by executing:
.. code-block:: bash
tox -e bashate
Ansible is lint tested using `ansible-lint`_.
Testing may be done locally by executing:
.. code-block:: bash
tox -e ansible-lint
Ansible playbook syntax is tested using ansible-playbook.
Testing may be done locally by executing:
.. code-block:: bash
tox -e ansible-syntax
A consolidated set of all lint tests may be done locally by executing:
.. code-block:: bash
tox -e linters
.. _PEP8: https://www.python.org/dev/peps/pep-0008/
.. _Bashate: https://git.openstack.org/cgit/openstack-dev/bashate
.. _ansible-lint: https://github.com/willthames/ansible-lint
Documentation Build
-------------------
Documentation is developed in reStructuredText_ (RST) and compiled into
HTML using Sphinx.
Documentation may be built locally by executing:
.. code-block:: bash
tox -e docs
tox -e deploy-guide
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
Release Notes Build
-------------------
Release notes are generated using the `the reno tool`_ and compiled into
HTML using Sphinx.
Release notes may be built locally by executing:
.. code-block:: bash
tox -e releasenotes
.. _the reno tool: https://docs.openstack.org/developer/reno/usage.html
.. note::
The ``releasenotes`` build argument only tests committed changes.
Ensure your local changes are committed before running the
``releasenotes`` build.
Gating
^^^^^^
Every commit to the OpenStack-Ansible integrated build is verified by
OpenStack-CI through the following jobs:
* ``gate-openstack-ansible-releasenotes``: This job executes the
`Release Notes Build`_.
* ``gate-openstack-ansible-docs-ubuntu-xenial``: This job executes the
`Documentation Build`_.
* ``gate-openstack-ansible-linters-ubuntu-xenial``: This job executes
the `Lint Tests`_.
* ``gate-openstack-ansible-openstack-ansible-aio-ubuntu-xenial``: where
``aio`` is the scenario, ``ubuntu`` is the distribution, and ``xenial``
is the version of the distribution.
The same test is executed against multiple distribution versions, and
may be executed against multiple distributions and multiple scenarios
too.
This job executes the ``gate-check-commit.sh`` script which executes a
convergence test and then a functional test.
The convergence test is the execution of an AIO build which aims to test
the primary code path for a functional environment. The functional test
then executes OpenStack's Tempest testing suite to verify that the
environment that has deployed successfully actually works.
While this script is primarily developed and maintained for use in
OpenStack-CI, it can be used in other environments.
Dependency Updates
^^^^^^^^^^^^^^^^^^
The dependencies for OpenStack-Ansible are updated approximately every two
weeks through the use of ``scripts/sources-branch-updater.sh``. This script
updates all pinned SHA's for OpenStack services, OpenStack-Ansible roles,
and other python dependencies which are not handled by the OpenStack global
requirements management process. This script also updates the statically
held templates/files in each role to ensure that they are always up to date.
Finally, it also does a minor version increment of the value for
``openstack_release``.
The update script is used as follows:
.. parsed-literal::
# change directory to the openstack-ansible checkout
cd ~/code/openstack-ansible
# ensure that the correct branch is checked out
git checkout |current_release_git_branch_name|
# ensure that the branch is up to date
git pull
# create the local branch for the update
git checkout -b sha-update
# execute the script for all openstack services
./scripts/sources-branch-updater.sh -b |current_release_git_branch_name| -o |current_release_git_branch_name|
# execute the script for gnocchi
./scripts/sources-branch-updater.sh -s playbooks/defaults/repo_packages/gnocchi.yml -b |current_release_gnocchi_git_branch_name| -o |current_release_git_branch_name|
# the console code should only be updated when necessary for a security fix, or for the OSA master branch
./scripts/sources-branch-updater.sh -s playbooks/defaults/repo_packages/nova_consoles.yml -b master
# the testing repositories should not be updated for stable branches as the new tests
# or other changes introduced may not work for older branches
./scripts/sources-branch-updater.sh -s playbooks/defaults/repo_packages/openstack_testing.yml -b master
# commit the changes
new_version=$(awk '/^openstack_release/ {print $2}' inventory/group_vars/all/all.yml)
git add --all
git commit -a -m "Update all SHAs for ${new_version}" \
-m "This patch updates all the roles to the latest available stable
SHA's, copies the release notes from the updated roles into the
integrated repo, updates all the OpenStack Service SHA's, and
updates the appropriate python requirements pins.
# push the changes up to gerrit
git review

368
doc/source/contributor/testing.rst Normal file
View File

@ -0,0 +1,368 @@
.. _tests:
=======
Testing
=======
Adding tests to a new role
==========================
Each of the role tests is in its tests/ folder.
This folder contains at least the following files:
* ``test.yml`` ("super" playbook acting as test router to sub-playbooks)
* ``<role name>-overrides.yml``. This var file is automatically loaded
by our shell script in our `tests repository`_.
* ``inventory``. A static inventory for role testing.
It's possible some roles have multiple inventories. See for example the
neutron role with its ``lxb_inventory``, ``calico_inventory``.
* ``group_vars`` and ``host_vars``. These folders will hold override the
necessary files for testing. For example, this is where you override
the IP addresses, IP ranges, and ansible connection details.
* ``ansible-role-requirements.yml``. This should be fairly straightforward:
this file contains all the roles to clone before running your role.
The roles' relative playbooks will have to be listed in the ``test.yml``
file. However, keep in mind to NOT re-invent the wheel. For example,
if your role needs keystone, you don't need to create your own keystone
install playbook, because we have a generic keystone install playbook
in the `tests repository`.
* Only add a ``zuul.d`` folder when your role is imported into the
openstack-ansible namespace.
.. _tests repository: https://git.openstack.org/cgit/openstack/openstack-ansible-tests
Extending tests of an existing role
===================================
#. Modify the tox.ini to add your new scenario. If required, you can
override the inventory, and/or the variable files.
#. Add a new non-voting job in ``zuul.d/jobs.yaml``, and wire it in
the project tests file ``zuul.d/project.yaml``.
.. _tempest-testing:
Improve testing with tempest
============================
Once the initial convergence is working and the services are running,
the role development should focus on implementing some level of
functional testing.
Ideally, the functional tests for an OpenStack role
should make use of Tempest to execute the functional tests. The ideal
tests to execute are scenario tests as they test the functions that
the service is expected to do in a production deployment. In the absence
of any scenario tests for the service a fallback option is to implement
the smoke tests instead.
If no tempest is provided, some other functional testing should be done.
For APIs, you can probably check the HTTP response codes, with
specially crafted requests.
.. _integrate-new-role-with-aio:
Testing a new role with an AIO
==============================
#. Include your role on the deploy host.
See also :ref:`extend_osa_roles`.
#. Perform any other host preparation (such as the tasks performed by the
``bootstrap-aio.yml`` playbook). This includes any preparation tasks that
are particular to your service.
#. Generate files to include your service in the Ansible inventory
using ``env.d`` and ``conf.d`` files for use on your deploy host.
.. HINT:: You can follow examples from other roles, making the appropriate
modifications being sure that group labels in ``env.d`` and ``conf.d``
files are consistent.
.. HINT:: A description of how these work can be
found in :ref:`inventory-confd` and :ref:`inventory-envd`.
#. Generate secrets, if any, as described in the :deploy_guide:`Configure
Service Credentials <configure.html#configuring-service-credentials>`.
You can append your keys to an existing ``user_secrets.yml`` file or add a
new file to the ``openstack_deploy`` directory to contain them. Provide
overrides for any other variables you will need at this time as well, either
in ``user_variables.yml`` or another file.
See also our :ref:`user-overrides` page.
Any secrets required for the role to work must be noted in the
``etc/openstack_deploy/user_secrets.yml`` file for reuse by other users.
#. If your service is installed from source or relies on python packages which
need to be installed from source, specify a repository for the source
code of each requirement by adding a file to your deploy host under
``playbooks/defaults/repo_packages`` in the OpenStack-Ansible source
repository and following the pattern of files currently in that directory.
You could also simply add an entry to an existing file there. Be sure to
run the ``repo-build.yml`` play later so that wheels for your packages will
be included in the repository infrastructure.
#. Make any required adjustments to the load balancer configuration
(e.g. modify ``inventory/group_vars/all/haproxy.yml`` in the
OpenStack-Ansible source repository on your deploy host) so that your
service can be reached through a load balancer, if appropriate, and be sure
to run the ``haproxy-install.yml`` play later so your changes will be
applied. Please note, you can also use ``haproxy_extra_services`` variable
if you don't want to provide your service as default for everyone.
#. Put together a service install playbook file for your role. This can also
be modeled from any existing service playbook that has similar
dependencies to your service (database, messaging, storage drivers,
container mount points, etc.). A common place to keep playbook files in a
Galaxy role is in an ``examples`` directory off the root of the role.
If the playbook is meant for installing an OpenStack service, name it
``os-<service>-install.yml`` and target it at the appropriate
group defined in the service ``env.d`` file.
It is crucial that the implementation of the service is optional and
that the deployer must opt-in to the deployment through the population
of a host in the applicable host group. If the host group has no
hosts, Ansible skips the playbook's tasks automatically.
#. Any variables needed by other roles to connect to the new role, or by the
new role to connect to other roles, should be implemented in
``inventory/group_vars``. The group vars are essentially the
glue which playbooks use to ensure that all roles are given the
appropriate information. When group vars are implemented it should be a
minimum set to achieve the goal of integrating the new role into the
integrated build.
#. Documentation must be added in the role to describe how to implement
the new service in an integrated environement. This content must
adhere to the :ref:`documentation`. Until the
role has integrated functional testing implemented (see also the
Role development maturity paragraph), the documentation
must make it clear that the service inclusion in OpenStack-Ansible is
experimental and is not fully tested by OpenStack-Ansible in an
integrated build. Alternatively, an user story can be created.
#. A feature release note must be added to announce the new service
availability and to refer to the role documentation for further
details. This content must adhere to the
:ref:`documentation`.
#. It must be possible to execute a functional, integrated test which
executes a deployment in the same way as a production environment. The
test must execute a set of functional tests using Tempest. This is the
required last step before a service can remove the experimental warning
from the documentation.
.. HINT:: If you adhere to the pattern of isolating your role's extra
deployment requirements (secrets and var files, HAProxy yml fragments,
repo_package files, etc.) in their own files it makes it easy for you to
automate these additional steps when testing your role.
.. _devel_and_testing:
Running tests locally
=====================
Linting
-------
Python coding conventions are tested using `PEP8`_, with the following
convention exceptions:
* F403 - 'from ansible.module_utils.basic import \*'
Testing may be done locally by executing:
.. code-block:: bash
./run_tests.sh pep8
Bash coding conventions are tested using `Bashate`_, with the following
convention exceptions:
* E003: Indent not multiple of 4. We prefer to use multiples of 2 instead.
* E006: Line longer than 79 columns. As many scripts are deployed as templates
and use jinja templating, this is very difficult to achieve. It is
still considered a preference and should be a goal to improve
readability, within reason.
* E040: Syntax error determined using `bash -n`. As many scripts are deployed
as templates and use jinja templating, this will often fail. This
test is reasonably safely ignored as the syntax error will be
identified when executing the resulting script.
Testing may be done locally by executing:
.. code-block:: bash
./run_tests.sh bashate
Ansible is lint tested using `ansible-lint`_.
Testing may be done locally by executing:
.. code-block:: bash
./run_tests.sh ansible-lint
Ansible playbook syntax is tested using ansible-playbook.
Testing may be done locally by executing:
.. code-block:: bash
./run_tests.sh ansible-syntax
A consolidated set of all lint tests may be done locally by executing:
.. code-block:: bash
./run_tests.sh linters
.. _PEP8: https://www.python.org/dev/peps/pep-0008/
.. _Bashate: https://git.openstack.org/cgit/openstack-dev/bashate
.. _ansible-lint: https://github.com/willthames/ansible-lint
Documentation building
----------------------
Documentation is developed in reStructuredText_ (RST) and compiled into
HTML using Sphinx.
Documentation may be built locally by executing:
.. code-block:: bash
./run_tests.sh docs
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
Deployment guide building
^^^^^^^^^^^^^^^^^^^^^^^^^
The OpenStack-Ansible integrated repo also has an extra documentation
building process, to build the deployment guide.
This guide may be built locally by executing:
.. code-block:: bash
./run_tests.sh deploy-guide
Release notes building
----------------------
Release notes are generated using the `the reno tool`_ and compiled into
HTML using Sphinx.
Release notes may be built locally by executing:
.. code-block:: bash
./run_tests.sh releasenotes
.. _the reno tool: https://docs.openstack.org/developer/reno/usage.html
.. note::
The ``releasenotes`` build argument only tests committed changes.
Ensure your local changes are committed before running the
``releasenotes`` build.
Roles functional or scenario testing
------------------------------------
To run a functional test of the role, execute:
.. code-block:: bash
./run_tests.sh functional
Some roles have extra tests, like neutron, defined in ``tox.ini``.
To run a functional test named "calico", execute:
.. code-block:: bash
./run_tests.sh calico
Integrated repo functional or scenario testing
----------------------------------------------
To test the integrated repo, follow the
:deploy_guide:`Deployment Guide <index.html>`
Alternatively, you can check the :ref:`aio guide<quickstart-aio>`,
or even run the gate wrapper script,
named ``scripts/gate-check-commit.sh``, described below.
The OpenStack Infrastructure automated tests
============================================
There should be no difference between running tests in the openstack
infrastructure, versus running locally.
The tests in the openstack infrastructure are triggered by jobs
defined in each repo ``zuul.d`` folder.
See also the `zuul user guide`_.
However, for reliability purposes, a few variables are defined
to point to the OpenStack infra pypi and packages mirrors.
.. _zuul user guide: https://docs.openstack.org/infra/zuul/user/index.html
The integrated repo functional test is using the
``scripts/gate-check-commit.sh`` script, which receives arguments
from the zuul run playbook definition.
While this script is primarily developed and maintained for use in
OpenStack-CI, it can be used in other environments.
.. _role-maturity:
Role development maturity
=========================
A role may be fully mature, even if it is not integrated in the
``openstack-ansible`` repository. The maturity depends on its
testing levels.
A role can be in one of the four maturity levels:
* ``Complete``
* ``Incubated``
* ``Unmaintained``
* ``Retired``
Here are a series of rules that define maturity levels:
* A role can be retired at any time if it is not relevant anymore.
* A role can be ``Incubated`` for maximum 2 cycles.
* An ``Incubated`` role that passes functional testing will be upgraded
to the ``Complete`` status, and cannot return in ``Incubated`` status.
* An ``Incubated`` role that didn't implement functional testing in
the six month timeframe will become ``Unmaintained``.
* A role in ``Complete`` status can be downgraded to ``Unmaintained``.
status, according to the maturity downgrade procedure.
Maturity downgrade procedure
----------------------------
If a role has failed periodics or gate test for two weeks, a bug
should be filed, and a message to the mailing list will be sent,
referencing the bug.
The next community meeting should discuss about role deprecation,
and if no contributor comes forward to fix the role, periodic
testing will be turned off, and the role will move to an
``unmaintained`` state.
.. _role-maturity-matrix:
Maturity Matrix
---------------
All of the OpenStack-Ansible roles do not have the same level of maturity and
testing.
Here is a dashboard of the current status of the roles:
.. raw:: html
:file: role-maturity-matrix.html

35
doc/source/reference/configuration/extending-osa.rst
View File

@ -2,7 +2,7 @@ Extending OpenStack-Ansible
===========================
Including OpenStack-Ansible in your project
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-------------------------------------------
Including the openstack-ansible repository within another project can be
done in several ways:
@ -18,22 +18,21 @@ Also note that copying files into directories such as ``env.d`` or
project.
Including OpenStack-Ansible with your Ansible structure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-------------------------------------------------------
You can create your own playbook, variable, and role structure while still
including the OpenStack-Ansible roles and libraries by setting environment
variables or by adjusting ``/usr/local/bin/openstack-ansible.rc``.
The relevant environment variables for Ansible 1.9 (included in
OpenStack-Ansible) are as follows:
The relevant environment variables for OpenStack-Ansible are as follows:
``ANSIBLE_LIBRARY``
This variable should point to
``openstack-ansible/playbooks/library``. Doing so allows roles and
``/etc/ansible/plugins/library``. Doing so allows roles and
playbooks to access OpenStack-Ansible's included Ansible modules.
``ANSIBLE_ROLES_PATH``
This variable should point to
``openstack-ansible/playbooks/roles``. This allows Ansible to
``/etc/ansible/roles`` by default. This allows Ansible to
properly look up any OpenStack-Ansible roles that extension roles
may reference.
``ANSIBLE_INVENTORY``
@ -58,3 +57,27 @@ Consider this directory structure::
The environment variables set would use
``../openstack-ansible/playbooks/<directory>``.
.. _extend_osa_roles:
Adding new or overriding roles in your OpenStack-Ansible installation
---------------------------------------------------------------------
By default OpenStack-Ansible uses its `ansible-role-requirements`_ file
to fetch the roles it requires for the installation process.
The roles will be fetched into the standard ``ANSIBLE_ROLES_PATH``,
which defaults to ``/etc/ansible/roles``.
``ANSIBLE_ROLE_FILE`` is an environment variable pointing to
the location of a YAML file which ansible-galaxy can consume,
specifying which roles to download and install.
The default value for this is ``ansible-role-requirements.yml``.
You can override the ansible-role-requirement file used by defining
the environment variable ``ANSIBLE_ROLE_FILE`` before running the
``bootstrap-ansible.sh`` script.
.. _ansible-role-requirements: https://git.openstack.org/cgit/openstack/openstack-ansible/tree/ansible-role-requirements.yml

4
doc/source/reference/inventory/understanding-inventory.rst
View File

@ -22,6 +22,8 @@ To customize the layout of the components for your deployment, modify the
host groups and container groups appropriately before running the installation
playbooks.
.. _inventory-confd:
Understanding host groups (conf.d structure)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@ -51,6 +53,8 @@ variables to any component containers on the specific host.
particularly for new services, by using a new file in the
``conf.d/`` directory.
.. _inventory-envd:
Understanding container groups (env.d structure)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

6
doc/source/user/aio/quickstart.rst
View File

@ -191,8 +191,10 @@ Notes:
# TMPDIR=/var/tmp scripts/bootstrap-ansible.sh
In order for all the services to run, the host must be prepared with the
appropriate disks, packages, network configuration and a base configuration
for the OpenStack Deployment. For the default AIO scenario, this preparation
appropriate disks partitioning, packages, network configuration and
configurations for the OpenStack Deployment.
For the default AIO scenario, this preparation
is completed by executing:
.. code-block:: shell-session

1
doc/source/user/aio/scenario-table-gen.html
View File

@ -0,0 +1 @@
<table border="1"><thead valign="bottom"><tr><th style="padding-left:5px;padding-right:5px;" class="head"></th><th style="padding-left:5px;padding-right:5px;" class="head">aio_basekit</th><th style="padding-left:5px;padding-right:5px;" class="head">aio_lxc</th><th style="padding-left:5px;padding-right:5px;" class="head">aio_metal</th><th style="padding-left:5px;padding-right:5px;" class="head">aio_nspawn</th><th style="padding-left:5px;padding-right:5px;" class="head">ceph</th><th style="padding-left:5px;padding-right:5px;" class="head">congress</th><th style="padding-left:5px;padding-right:5px;" class="head">octavia</th><th style="padding-left:5px;padding-right:5px;" class="head">tacker</th><th style="padding-left:5px;padding-right:5px;" class="head">translations</th></tr></thead><tbody valign="top"><tr><td align="left">ceph</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td></tr><tr><td align="left">cinder</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">congress</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td>&#160;</td><td>&#160;</td></tr><tr><td align="left">designate</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">glance</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">haproxy</td><td align="center">X</td><td align="center">X</td><td>&#160;</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td></tr><tr><td align="left">heat</td><td>&#160;</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td><td align="center">X</td></tr><tr><td align="left">horizon</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">keystone</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td></tr><tr><td align="left">magnum</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">neutron</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">nova</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">octavia</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">sahara</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">swift</td><td>&#160;</td><td align="center">X</td><td align="center">X</td><td align="center">X</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td></tr><tr><td align="left">tacker</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td><td>&#160;</td></tr><tr><td align="left">trove</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td>&#160;</td><td align="center">X</td></tr></tbody></table>

1
doc/source/user/index.rst
View File

@ -29,3 +29,4 @@ For in-depth technical information, see the
l3pods/example.rst
ceph/full-deploy.rst
security/index.rst
source-overrides/index.rst

77
doc/source/user/source-overrides/index.rst Normal file
View File

@ -0,0 +1,77 @@
==========================
Source overriding examples
==========================
There are situations where a deployer want to override sources with
its own fork.
This chapter gives case-by-case examples on how to override the
default sources.
Overriding Ansible version
==========================
Overriding the default Ansible version is not recommended, as
each branch of OpenStack-Ansible has been built with the a specific
Ansible version in mind, and many Ansible changes are neither backwards
nor forward compatible.
The ``bootstrap-ansible.sh`` script installs Ansible, and uses
a variable ``ANSIBLE_PACKAGE`` to describe which version to install.
For example to install ansible version 2.5.0:
.. code:: bash
$ export ANSIBLE_PACKAGE="ansible==2.5.0"
Installing directly from git is also supported. For example, from the tip of
Ansible development branch:
.. code:: bash
$ export ANSIBLE_PACKAGE="git+https://github.com/ansible/ansible@devel#egg=ansible"
Overriding the roles
====================
Overriding the role file has been explained in the reference guide,
on the :ref:`extend_osa_roles` section.
Overriding other upstream projects source code
==============================================
All the upstream repositories used are defined in the
``openstack-ansible`` integrated repository, in the
``playbooks/defaults/repo_packages`` folder.
For example, if you want to override ``glance`` repository with your
own, you need to define the following:
::
glance_git_repo: https://<your git repo>
glance_git_install_branch: <your git branch or commit SHA>
glance_git_project_group: glance_all
Please note, for this glance example, that you do not need to edit the
``playbooks/defaults/repo_packages/openstack_services.yml`` file.
Instead, the usual overrides mechanism can take place, and you
can define these 3 variables in a ``user_*.yml`` file.
See also the :ref:`user-overrides` page.
.. note::
These variables behave a little differently than standard ansible
precedence, because they are also consumed by a custom lookup plugin.
The ``py_pkgs lookup`` will ignore all _git_ variables unless the
``_git_repo`` variable is present.
So even if you only want to override the ``_git_install_branch`` for
a repository, you should also define the ``_git_repo`` variable
in your user variables.