diff --git a/doc/source/contributor/CONTRIBUTING.rst b/doc/source/contributor/CONTRIBUTING.rst index 27c0e36af7..68e1ec425a 100644 --- a/doc/source/contributor/CONTRIBUTING.rst +++ b/doc/source/contributor/CONTRIBUTING.rst @@ -7,37 +7,37 @@ How To Contribute Basics ====== -#. Our source code is hosted on `OpenStack Kolla-Ansible Git`_. Bugs should be - filed on launchpad_. +#. Our source code is hosted on `OpenStack Kolla-Ansible Git + `_. Bugs should be + filed on `launchpad `_. -#. Please follow OpenStack `Gerrit Workflow`_ to contribute to Kolla. +#. Please follow OpenStack `Gerrit Workflow + `__ + to contribute to Kolla-ansible. #. Note the branch you're proposing changes to. ``master`` is the current focus of development. Kolla project has a strict policy of only allowing backports in ``stable/branch``, unless when not applicable. A bug in a ``stable/branch`` will first have to be fixed in ``master``. -#. Please file a launchpad_ blueprint for any significant code change and a bug - for any significant bug fix or add a TrivialFix tag for simple changes. - See how to reference a bug or a blueprint in the commit message here_ +#. Please file a `blueprint of kolla-ansible `__ + for any significant code change and a bug for any significant bug fix, + or add a ``TrivialFix`` tag to commit message for simple changes. + See how to reference a bug or a blueprint in the `commit message + `_. #. TrivialFix tags or bugs are not required for documentation changes. -.. _OpenStack Kolla-Ansible Git: https://git.openstack.org/cgit/openstack/kolla-ansible/ -.. _launchpad: https://bugs.launchpad.net/kolla-ansible -.. _here: https://wiki.openstack.org/wiki/GitCommitMessages - Development Environment ======================= Please follow our :doc:`/user/quickstart` to deploy your environment and test your changes. -Please use the existing sandbox repository, available at -https://git.openstack.org/cgit/openstack-dev/sandbox, for learning, understanding -and testing the `Gerrit Workflow`_. - -.. _Gerrit Workflow: https://docs.openstack.org/infra/manual/developers.html#development-workflow +Please use the existing sandbox repository, available at `sandbox +`_, for learning, understanding +and testing the `Gerrit Workflow +`_. Adding a new service ==================== @@ -86,12 +86,15 @@ that Kolla uses throughout that should be followed. template file in ``ansible/roles/common/templates`` with the following content: - .. code:: + .. path ansible/roles/common/templates/cron-logrotate-PROJECT.conf.j2 + .. code-block:: none "/var/log/kolla/PROJECT/*.log" { } + .. end + - For OpenStack services there should be an entry in the ``services`` list in the ``cron.json.j2`` template file in ``ansible/roles/common/templates``. @@ -114,18 +117,18 @@ that Kolla uses throughout that should be followed. - All YAML data files should start with three dashes (``---``). -Other than the above, most roles follow the following pattern: +Other than the above, most service roles abide by the following pattern: - - ``Register``: Involves registering the service with Keystone, creating - endpoints, roles, users, etc. +- ``Register``: Involves registering the service with Keystone, creating + endpoints, roles, users, etc. - - ``Config``: Distributes the config files to the nodes to be pulled into - the container on startup. +- ``Config``: Distributes the config files to the nodes to be pulled into + the container on startup. - - ``Bootstrap``: Creating the database (but not tables), database user for - the service, permissions, etc. +- ``Bootstrap``: Creating the database (but not tables), database user for + the service, permissions, etc. - - ``Bootstrap Service``: Starts a one shot container on the host to create - the database tables, and other initial run time config. +- ``Bootstrap Service``: Starts a one shot container on the host to create + the database tables, and other initial run time config. - - ``Start``: Start the service(s). +- ``Start``: Start the service(s). diff --git a/doc/source/contributor/kolla-for-openstack-development.rst b/doc/source/contributor/kolla-for-openstack-development.rst index af2d1f9f1c..7087076f45 100644 --- a/doc/source/contributor/kolla-for-openstack-development.rst +++ b/doc/source/contributor/kolla-for-openstack-development.rst @@ -7,17 +7,17 @@ development on OpenStack services. .. note:: - This functionality is new in the Pike release. + This functionality is new in the Pike release. Heat was the first service to be supported, and so the following will use submitting a patch to Heat using Kolla as an example. Only source containers are supported. -.. WARNING:: +.. warning:: - Kolla dev mode is intended for OpenStack hacking/development only. Do not - use this in production! + Kolla dev mode is intended for OpenStack hacking or development only. + Do not use this in production! Enabling Kolla "dev mode" ------------------------- @@ -25,15 +25,21 @@ Enabling Kolla "dev mode" To enable dev mode for all supported services, set in ``/etc/kolla/globals.yml``: -:: +.. path /etc/kolla/globals.yml +.. code-block:: none - kolla_dev_mode: true + kolla_dev_mode: true + +.. end To enable it just for heat, set: -:: +.. path /etc/kolla/globals.yml +.. code-block:: none - heat_dev_mode: true + heat_dev_mode: true + +.. end Usage ----- @@ -44,9 +50,11 @@ container's virtualenv under the location expected by the service on startup. After making code changes, simply restart the container to pick them up: -:: +.. code-block:: console - docker restart heat_api + docker restart heat_api + +.. end Debugging --------- @@ -54,23 +62,29 @@ Debugging ``remote_pdb`` can be used to perform debugging with Kolla containers. First, make sure it is installed in the container in question: -:: +.. code-block:: console - docker exec -it -u root heat_api pip install remote_pdb + docker exec -it -u root heat_api pip install remote_pdb + +.. end Then, set your breakpoint as follows: -:: +.. code-block:: none - from remote_pdb import RemotePdb - RemotePdb('127.0.0.1', 4444).set_trace() + from remote_pdb import RemotePdb + RemotePdb('127.0.0.1', 4444).set_trace() + +.. end Once you run the code(restart the container), pdb can be accessed using ``socat``: -:: +.. code-block:: console - socat readline tcp:127.0.0.1:4444 + socat readline tcp:127.0.0.1:4444 -For more information on remote_pdb, see -https://pypi.python.org/pypi/remote-pdb. +.. end + +Learn more information about `remote_pdb +`_. diff --git a/doc/source/contributor/running-tests.rst b/doc/source/contributor/running-tests.rst index 317f2d9483..2f897493c8 100644 --- a/doc/source/contributor/running-tests.rst +++ b/doc/source/contributor/running-tests.rst @@ -6,8 +6,9 @@ Running tests Kolla-ansible contains a suit of tests in the ``tests`` directory. -Any proposed code change in gerrit is automatically rejected by the OpenStack -Jenkins server [#f1]_ if the change causes test failures. +Any proposed code change in gerrit is automatically rejected by the +`OpenStack Jenkins server `__ +if the change causes test failures. It is recommended for developers to run the test suite before submitting patch for review. This allows to catch errors as early as possible. @@ -22,30 +23,36 @@ so the only package you install is ``tox`` itself: .. code-block:: console - $ pip install tox + pip install tox -See `the unit testing section of the Testing wiki page`_ for more information. -Following are some simple examples. +.. end + +For more information, see `the unit testing section of the Testing wiki page +`_. For example: To run the Python 2.7 tests: .. code-block:: console - $ tox -e py27 + tox -e py27 + +.. end To run the style tests: .. code-block:: console - $ tox -e pep8 + tox -e pep8 + +.. end To run multiple tests separate items by commas: .. code-block:: console - $ tox -e py27,py34,pep8 + tox -e py27,py35,pep8 -.. _the unit testing section of the Testing wiki page: https://wiki.openstack.org/wiki/Testing#Unit_Tests +.. end Running a subset of tests ------------------------- @@ -59,48 +66,58 @@ directory use: .. code-block:: console - $ tox -e py27 kolla-ansible.tests + tox -e py27 kolla-ansible.tests + +.. end To run the tests of a specific file -say ``kolla-ansible/tests/test_kolla_docker.py``: +``kolla-ansible/tests/test_kolla_docker.py``: .. code-block:: console - $ tox -e py27 test_kolla_docker + tox -e py27 test_kolla_docker + +.. end To run the tests in the ``ModuleArgsTest`` class in the ``kolla-ansible/tests/test_kolla_docker.py`` file: .. code-block:: console - $ tox -e py27 test_kolla_docker.ModuleArgsTest + tox -e py27 test_kolla_docker.ModuleArgsTest + +.. end To run the ``ModuleArgsTest.test_module_args`` test method in the ``kolla-ansible/tests/test_kolla_docker.py`` file: .. code-block:: console - $ tox -e py27 test_kolla_docker.ModuleArgsTest.test_module_args + tox -e py27 test_kolla_docker.ModuleArgsTest.test_module_args + +.. end Debugging unit tests ------------------------- +-------------------- In order to break into the debugger from a unit test we need to insert a breaking point to the code: .. code-block:: python - import pdb; pdb.set_trace() + import pdb; pdb.set_trace() -Then run ``tox`` with the debug environment as one of the following:: +.. end - tox -e debug - tox -e debug test_file_name.TestClass.test_name +Then run ``tox`` with the debug environment as one of the following: -For more information see the `oslotest documentation +.. code-block:: console + + tox -e debug + tox -e debug test_file_name.TestClass.test_name + +.. end + +For more information, see the `oslotest documentation `_. - -.. rubric:: Footnotes - -.. [#f1] See https://docs.openstack.org/infra/system-config/jjb.html diff --git a/doc/source/contributor/vagrant-dev-env.rst b/doc/source/contributor/vagrant-dev-env.rst index e50f8ad206..7443414e59 100644 --- a/doc/source/contributor/vagrant-dev-env.rst +++ b/doc/source/contributor/vagrant-dev-env.rst @@ -42,87 +42,147 @@ choice. Various downloads can be found at the `Vagrant downloads Install required dependencies as follows: -On CentOS:: +For CentOS 7 or later: - sudo yum install ruby-devel libvirt-devel zlib-devel libpng-devel gcc \ - qemu-kvm qemu-img libvirt libvirt-python libvirt-client virt-install \ - bridge-utils git +.. code-block:: console -On Ubuntu 16.04 or later:: + sudo yum install ruby-devel libvirt-devel zlib-devel libpng-devel gcc \ + qemu-kvm qemu-img libvirt libvirt-python libvirt-client virt-install \ + bridge-utils git - sudo apt-get install vagrant ruby-dev ruby-libvirt python-libvirt libvirt-dev nfs-kernel-server zlib1g-dev libpng12-dev gcc git +.. end -.. note:: Many distros ship outdated versions of Vagrant by default. When in - doubt, always install the latest from the downloads page above. +For Ubuntu 16.04 or later: + +.. code-block:: console + + sudo apt-get install vagrant ruby-dev ruby-libvirt python-libvirt \ + libvirt-dev nfs-kernel-server zlib1g-dev libpng12-dev gcc git + +.. end + +.. note:: + + Many distros ship outdated versions of Vagrant by default. When in + doubt, always install the latest from the downloads page above. Next install the hostmanager plugin so all hosts are recorded in ``/etc/hosts`` -(inside each vm):: +(inside each vm): - vagrant plugin install vagrant-hostmanager +.. code-block:: console -If you are going to use VirtualBox, then install vagrant-vbguest:: + vagrant plugin install vagrant-hostmanager - vagrant plugin install vagrant-vbguest +.. end + +If you are going to use VirtualBox, then install vagrant-vbguest: + +.. code-block:: console + + vagrant plugin install vagrant-vbguest + +.. end Vagrant supports a wide range of virtualization technologies. If VirtualBox is used, the vbguest plugin will be required to install the VirtualBox Guest -Additions in the virtual machine:: +Additions in the virtual machine: - vagrant plugin install vagrant-vbguest +.. code-block:: console + + vagrant plugin install vagrant-vbguest + +.. end This documentation focuses on libvirt specifics. To install vagrant-libvirt -plugin:: +plugin: - vagrant plugin install --plugin-version ">= 0.0.31" vagrant-libvirt +.. code-block:: console + + vagrant plugin install --plugin-version ">= 0.0.31" vagrant-libvirt + +.. end Some Linux distributions offer vagrant-libvirt packages, but the version they provide tends to be too old to run Kolla. A version of >= 0.0.31 is required. To use libvirt from Vagrant with a low privileges user without being asked for -a password, add the user to the libvirt group:: +a password, add the user to the libvirt group: - sudo gpasswd -a ${USER} libvirt - newgrp libvirt +.. code-block:: console + + sudo gpasswd -a ${USER} libvirt + newgrp libvirt + +.. end Setup NFS to permit file sharing between host and VMs. Contrary to the rsync method, NFS allows both way synchronization and offers much better performance -than VirtualBox shared folders. On CentOS:: +than VirtualBox shared folders. For CentOS: - # Add the virtual interfaces to the internal zone - sudo firewall-cmd --zone=internal --add-interface=virbr0 - sudo firewall-cmd --zone=internal --add-interface=virbr1 - # Enable nfs, rpc-bind and mountd services for firewalld - sudo firewall-cmd --permanent --zone=internal --add-service=nfs - sudo firewall-cmd --permanent --zone=internal --add-service=rpc-bind - sudo firewall-cmd --permanent --zone=internal --add-service=mountd - sudo firewall-cmd --permanent --zone=internal --add-port=2049/udp - sudo firewall-cmd --permanent --add-port=2049/tcp - sudo firewall-cmd --permanent --add-port=111/udp - sudo firewall-cmd --permanent --add-port=111/tcp - sudo firewall-cmd --reload - # Start required services for NFS - sudo systemctl restart firewalld - sudo systemctl start nfs-server - sudo systemctl start rpcbind.service +#. Add the virtual interfaces to the internal zone: + +.. code-block:: console + + sudo firewall-cmd --zone=internal --add-interface=virbr0 + sudo firewall-cmd --zone=internal --add-interface=virbr1 + +.. end + +#. Enable nfs, rpc-bind and mountd services for firewalld: + +.. code-block:: console + + sudo firewall-cmd --permanent --zone=internal --add-service=nfs + sudo firewall-cmd --permanent --zone=internal --add-service=rpc-bind + sudo firewall-cmd --permanent --zone=internal --add-service=mountd + sudo firewall-cmd --permanent --zone=internal --add-port=2049/udp + sudo firewall-cmd --permanent --add-port=2049/tcp + sudo firewall-cmd --permanent --add-port=111/udp + sudo firewall-cmd --permanent --add-port=111/tcp + sudo firewall-cmd --reload + +.. end + +#. Start required services for NFS: + +.. code-block:: console + + sudo systemctl restart firewalld + sudo systemctl start nfs-server + sudo systemctl start rpcbind.service + +.. end Ensure your system has libvirt and associated software installed and setup -correctly. On CentOS:: +correctly. For CentOS: - sudo systemctl start libvirtd - sudo systemctl enable libvirtd +.. code-block:: console -Find a location in the system's home directory and checkout Kolla repos:: + sudo systemctl start libvirtd + sudo systemctl enable libvirtd - git clone https://git.openstack.org/openstack/kolla-ansible - git clone https://git.openstack.org/openstack/kolla +.. end + +Find a location in the system's home directory and checkout Kolla repos: + +.. code-block:: console + + git clone https://git.openstack.org/openstack/kolla-ansible + git clone https://git.openstack.org/openstack/kolla + +.. end Both repos must share the same parent directory so the bootstrap code can locate them. Developers can now tweak the Vagrantfile or bring up the default **all-in-one** -CentOS 7-based environment:: +CentOS 7-based environment: - cd kolla-ansible/contrib/dev/vagrant && vagrant up +.. code-block:: console + + cd kolla-ansible/contrib/dev/vagrant && vagrant up + +.. end The command ``vagrant status`` provides a quick overview of the VMs composing the environment. @@ -131,9 +191,13 @@ Vagrant Up ========== Once Vagrant has completed deploying all nodes, the next step is to launch -Kolla. First, connect with the **operator** node:: +Kolla. First, connect with the **operator** node: - vagrant ssh operator +.. code-block:: console + + vagrant ssh operator + +.. end To speed things up, there is a local registry running on the operator. All nodes are configured so they can use this insecure repo to pull from, and use @@ -150,42 +214,63 @@ like ``vagrant destroy``. Building images --------------- -Once logged on the **operator** VM call the ``kolla-build`` utility:: +Once logged on the **operator** VM call the ``kolla-build`` utility: - kolla-build +.. code-block:: console -``kolla-build`` accept arguments as documented in `Building Container Images`_. + kolla-build + +.. end + +``kolla-build`` accept arguments as documented in `Building Container Images +`_. It builds Docker images and pushes them to the local registry if the **push** option is enabled (in Vagrant this is the default behaviour). Deploying OpenStack with Kolla ------------------------------ -Deploy **all-in-one** with:: +To deploy **all-in-one**: - sudo kolla-ansible deploy +.. code-block:: console -Deploy multinode -On Centos 7:: + sudo kolla-ansible deploy - sudo kolla-ansible deploy -i /usr/share/kolla-ansible/ansible/inventory/multinode +.. end -On Ubuntu 16.04 or later:: +To deploy multinode: - sudo kolla-ansible deploy -i /usr/local/share/kolla-ansible/ansible/inventory/multinode +For Centos 7: -Validate OpenStack is operational:: +.. code-block:: console - kolla-ansible post-deploy - . /etc/kolla/admin-openrc.sh - openstack user list + sudo kolla-ansible deploy -i /usr/share/kolla-ansible/ansible/inventory/multinode -Or navigate to http://172.28.128.254/ with a web browser. +.. end + +For Ubuntu 16.04 or later: + +.. code-block:: console + + sudo kolla-ansible deploy -i /usr/local/share/kolla-ansible/ansible/inventory/multinode + +.. end + +Validate OpenStack is operational: + +.. code-block:: console + + kolla-ansible post-deploy + . /etc/kolla/admin-openrc.sh + openstack user list + +.. end + +Or navigate to ``http://172.28.128.254/`` with a web browser. Further Reading =============== All Vagrant documentation can be found at -`Vagrant documentation` `__. +`Vagrant documentation `_. -.. _Building Container Images: https://docs.openstack.org/kolla/latest/admin/image-building.html