diff --git a/doc/source/developer/tht_walkthrough/service_template_sections.rst b/doc/source/developer/tht_walkthrough/service_template_sections.rst index 215291a0..fe3d6f01 100644 --- a/doc/source/developer/tht_walkthrough/service_template_sections.rst +++ b/doc/source/developer/tht_walkthrough/service_template_sections.rst @@ -324,8 +324,187 @@ such as the IPSec service deployment for TripleO:: This type of deployment applies for services that are better tied to TripleO's Ansible inventory or that don't require a specific inventory to run. +Container-related parameters +---------------------------- + +This covers the sections that allow you to write a containerized service for +TripleO. + +Containerized services brought a big change to TripleO. From packaging puppet +manifests and relying on them for configuration, we now have to package +containers, make sure the configuration ends up in the container somehow, then +run the containers. Here I won't describe the whole workflow of how we +containerized OpenStack services, but instead I'll describe what you need to +know to deploy a containerized service with TripleO. + +``puppet_config`` section +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Before getting into the deployment steps where TripleO starts running services +and containers, there is a step where puppet is ran in containers and all the +needed configurations are created. The ``puppet_config`` section controls this +step. + +There are several options we can pass here: + +* ``puppet_tags``: This describes the puppet resources that will be allowed to + run in puppet when generating the configuration files. Note that deeper + knowledge of your manifests and what runs in puppet is required for this. + Else, it might be better to generate the configuration files with Ansible + with the mechanisms described in previous sections of this document. + Any service that specifies tags will have the default tags of + ``'file,concat,file_line,augeas,cron'`` appended to the setting. + To know what settings to set here, as mentioned, you need to know your puppet + manifests. But, for instance, for keystone, an appropriate setting would be: + ``keystone_config``. For our etcd example, no tags are needed, since the + default tags we set here are enough. + +* ``config_volume``: The name of the directory where configuration files + will be generated for this service. You'll eventually use this to know what + location to bind-mount into the container to get the configuration. So, the + configuration will be persisted in: + ``/var/lib/config-data/puppet-generated/`` + +* ``config_image``: The name of the container image that will be used for + generating configuration files. This is often the same container + that the runtime service uses. Some services share a common set of + config files which are generated in a common base container. Typically + you'll get this from a parameter you pass to the template, e.g. + ``Image`` or ``ConfigImage``. Dealing with these + images requires dealing with the `container image prepare workflow`_. + The parameter should point to the specific image to be used, and it'll be + pulled from the registry as part of the + deployment. + +* ``step_config``: Similarly to the ``step_config`` that's described earlier in + this document, this setting controls the puppet manifest that is ran for this + service. The aforementioned puppet tags are used along with this manifest to + generate a config directory for this container. + +One important thing to note is that, if you're creating a containerized +service, you don't need to output a ``step_config`` section from the +``roles_data`` output. TripleO figured out if you're creating a containerized +service by checking for the existence of the ``docker_config`` section in the +``roles_data`` output. + +``kolla_config`` section +^^^^^^^^^^^^^^^^^^^^^^^^ + +As you might know, TripleO uses kolla to build the container images. Kolla, +however, not only provides the container definitions, but provides a rich +framework to extend and configure your containers. Part of this is the fact +that it provides an entry point that receives a configuration file, with which +you can modify several things from the container on start-up. We take advantage +of this in TripleO, and it's exactly what the ``kolla_config`` represents. + +For each container we create, we have a relevant ``kolla_config`` entry, with a +mapping key that has the following format:: + + /var/lib/kolla/config_files/.json + +This, contains YAML that represents how to map config files into the container. +In the container, this typically ends up mapped as +``/var/lib/kolla/config_files/config.json`` which kolla will end up reading. + +The typical configuration settings we use with this setting are the following: + +* ``command``: This defines the command we'll be running on the container. + Typically it'll be the command that runs the "server". So, in the example you + see ``/usr/bin/etcd ...``, which will be the main process running. + +* ``config_files``: This tells kolla where to read the configuration files + from, and where to persist them to. Typically what this is used for is that + the configuration generated by puppet is read from the host as "read-only", + and mounted on ``/var/lib/kolla/config_files/src``. Subsequently, it is + copied on to the right location by the kolla mechanisms. This way we make + sure that the container has the right permissions for the right user, given + we'll typically be in another user namespace in the container. + +* ``permissions``: As you would expect, this sets up the appropriate + permissions for a file or set of files in the container. + +``docker_config`` section +^^^^^^^^^^^^^^^^^^^^^^^^^ + +This is the section where we tell TripleO what containers to start. Here, we +explicitly write on which step to start which container. Steps are set as keys +with the ``step_`` format. Inside these, we should set up keys +with the specific container names. In our example, we're running only the etcd +container, so we use a key called ``etcd`` to give it such a name. A tool +called `paunch`_ will read these parameters, and start the containers +with those settings. + +Here's an example of the container definition:: + + step_2: + etcd: + image: {get_param: DockerEtcdImage} + net: host + privileged: false + restart: always + healthcheck: + test: /openstack/healthcheck + volumes: + - /var/lib/etcd:/var/lib/etcd + - /etc/localtime:/etc/localtime:ro + - /var/lib/kolla/config_files/etcd.json:/var/lib/kolla/config_files/config.json:ro + - /var/lib/config-data/puppet-generated/etcd/:/var/lib/kolla/config_files/src:ro + environment: + - KOLLA_CONFIG_STRATEGY=COPY_ALWAYS + +This is what we're telling TripleO to do: + +* Start the container on step 2 + +* Use the container image coming from the ``DockerEtcdImage`` heat parameter. + +* For the container, use the host's network. + +* The container is not `privileged`_. + +* Docker will use the ``/openstack/healthcheck`` endpoint for healthchecking + +* We tell it what volumes to mount + + - Aside from the necessary mounts, note that we're bind-mounting the + file ``/var/lib/kolla/config_files/etcd.json`` on to + ``/var/lib/kolla/config_files/config.json``. This will be read by kolla + in order for the container to execute the actions we configured in the + ``kolla_config`` section. + + - We also bind-mount ``/var/lib/config-data/puppet-generated/etcd/``, which + is where the puppet ran (which was ran inside a container) persisted the + needed configuration files. We bind-mounted this to + ``/var/lib/kolla/config_files/src`` since we told kolla to copy this to + the correct location inside the container on the ``config_files`` section + that's part of ``kolla_config``. + +* Environment tells the container engine which environment variables to set + + - We set ``KOLLA_CONFIG_STRATEGY=COPY_ALWAYS`` in the example, since this + tells kolla to always execute the ``config_files`` and ``permissions`` + directives as part of the kolla entry point. If we don't set this, it + will only be executed the first time we run the container. + +``docker_puppet_tasks`` section +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +These are containerized puppet executions that are meant as bootstrapping +tasks. They typically run on a "bootstrap node", meaning, they only run on one +relevant node in the cluster. And are meant for actions that you should only +execute once. Examples of this are: creating keystone endpoints, creating +keystone domains, creating the database users, etc. + +The format for this is quite similar to the one described in ``puppet_config`` +section, except for the fact that you can set several of these, and they also +run as part of the steps (you can specify several of these, divided by the +``step_`` keys). + .. References .. _heat resource chain object: https://docs.openstack.org/heat/pike/template_guide/openstack.html#OS::Heat::ResourceChain .. _common/services.yaml: https://github.com/openstack/tripleo-heat-templates/blob/stable/queens/common/services.yaml#L44 +.. _container image prepare workflow: https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/overcloud.html#preparing-overcloud-images +.. _paunch: https://docs.openstack.org/developer/paunch/readme.html +.. _privileged: https://www.linux.com/blog/learn/sysadmin/2017/5/lazy-privileged-docker-containers