Fixups for containers documentation

Update documentation to match the latest architecture. Also fixed
several issues and documented how to populate a local docker registry.

Change-Id: If646497fc2f1795f13e116b54cbd36803940de7b

doc/source/containers_deployment/architecture.rst

@@ -1,5 +1,5 @@
 TripleO Containers Architecture
-================================
+===============================
 
 .. Warning::
 
@@ -16,7 +16,7 @@ of different projects to play together. The next section will cover each of the
 parts that allow for deploying OpenStack on containers using TripleO.
 
 Kolla Build
-------------
+-----------
 
 Kolla is an OpenStack team that aims to create tools to allow for deploying
 OpenStack on container technologies. Kolla (or Kolla Build) is one of the tools
@@ -40,12 +40,16 @@ of the services, like mariadb::
 .. note:: `parent_template` is the literal string to include. No need to replace
    it.
 
-Use the following commnand to build an image using kolla-build and the template
+Use the following command to build an image using kolla-build and the template
 above (`template-overrides.j2`)::
 
   $ kolla-build --base centos --template-override template-overrides.j2
 
+TripleO maintains its complete list of kolla customization in the
+`tripleo-common`_ project.
+
 .. _Kolla: https://docs.openstack.org/developer/kolla/image-building.html#dockerfile-customisation
+.. _tripleo-common: https://github.com/openstack/tripleo-common/blob/master/contrib/tripleo_kolla_template_overrides.j2
 
 heat-config-docker-cmd
 ----------------------
@@ -57,7 +61,8 @@ the keys are:
 
 * **net**: To specify what network to use. This is commonly set to host.
 
-* **privileged**: Whether to run the container with superuser privileges.
+* **privileged**: Whether to give full access to the host's devices to the
+  container, similar to what happens when the service runs directly on the host.
 
 * **volumes**: List of host path volumes, named volumes, or dynamic volumes to
   bind on the container.
@@ -148,7 +153,6 @@ are re-asserted when applying latter ones.
         - get_attr: [MongodbPuppetBase, role_data, config_settings]
         - mongodb::server::fork: false
 
-
 * **step_config**: This setting controls the manifest that is used to create
   docker config files via puppet. The puppet tags below are used along with
   this manifest to generate a config directory for this container.
@@ -176,7 +180,7 @@ are re-asserted when applying latter ones.
   steps below and the related docker-cmd hook documentation in the heat-agents
   project.
 
-* puppet_config
+* **puppet_config**:
 
   * **step_config**: Usually a reference to the one defined outside this section.
 
@@ -220,7 +224,6 @@ are re-asserted when applying latter ones.
   etc. See docker-puppet.py for formatting. Here's an example of Keystone's
   `docker_puppet_tasks`::
 
-
       docker_puppet_tasks:
         # Keystone endpoint creation occurs only on single node
         step_4:
@@ -244,7 +247,7 @@ Docker steps
 
 Similar to baremetal, docker containers are brought up in a stepwise manner. The
 current architecture supports bringing up baremetal services alongside of
-containers. Thefore, baremetal steps may be required depending on the service
+containers. Therefore, baremetal steps may be required depending on the service
 and they are always executed before the corresponding container step.
 
 The list below represents the correlation between the baremetal and the

doc/source/containers_deployment/overcloud.rst

@@ -30,7 +30,7 @@ layer.
 Manual overcloud deployment
 ----------------------------
 
-This section explains how to deploy a containerized undercloud manually. For an
+This section explains how to deploy a containerized overcloud manually. For an
 automated overcloud deployment, please follow the steps in the
 `Using TripleO Quickstart`_ section below.
 
@@ -41,6 +41,28 @@ To prepare your environment, you must follow all the steps described in the
 :ref:`basic-deployment-cli` documentation. Stop right at the
 :ref:`deploy-the-overcloud` section.
 
+Populate local docker registry
+..............................
+
+It's useful to run a local docker registry on the undercloud to speed up the
+overcloud deployment. A docker registry is normally already setup to listen on
+port 8787 as part of the undercloud install.
+
+To use the pre-built images coming from the `tripleoupstream` registry on the
+dockerhub, use the following command::
+
+    openstack overcloud container image upload
+
+Or use `kolla-build` to build the images yourself::
+
+    kolla-build --base centos --type binary --namespace tripleoupstream --registry 192.168.24.1:8787 --tag latest --template-override /usr/share/tripleo-common/contrib/tripleo_kolla_template_overrides.j2 --push
+
+Finally, point the heat templates to your local registry, for example in
+a `$HOME/docker_registry.yaml` file::
+
+    parameter_defaults:
+      DockerNamespace: 192.168.24.1:8787/tripleoupstream
+      DockerNamespaceIsRegistry: true
 
 Deploying the containerized Overcloud
 -------------------------------------
@@ -52,6 +74,10 @@ deploy` command::
 
   -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml
 
+In case of a local docker registry, also add the path to the override file::
+
+  -e $HOME/docker_registry.yaml
+
 
 Using TripleO Quickstart
 ------------------------

doc/source/containers_deployment/tips_tricks.rst

@@ -29,13 +29,13 @@ The following commands are useful for debugging containers.
 
   There's no shortcut for *rebuilding* the command that was used to run the
   container but, it's possible to do so by using the `docker inspect` command
-  and the format parameter:::
+  and the format parameter::
 
    $ docker inspect --format='{{range .Config.Env}} -e "{{.}}" {{end}} {{range .Mounts}} -v {{.Source}}:{{.Destination}}{{if .Mode}}:{{.Mode}}{{end}}{{end}} -ti {{.Config.Image}}' $CONTAINER_ID_OR_NAME
 
   Copy the output from the command above and append it to the one below, which
   will run the same container with a random name and remove it as soon as the
-  execution exits:::
+  execution exits::
 
     $ docker run --rm $OUTPUT_FROM_PREVIOUS_COMMAND /bin/bash
 
@@ -86,12 +86,12 @@ debugging purposes.  For example::
 
     ./docker-toool -c swift-proxy -r -e /bin/bash -u root -i -n test
 
-will run the swift proxy container with all the volumes, permissions
+The above command will run the swift proxy container with all the volumes,
-etc as used at runtime but as the root user, executing /bin/bash, named
+permissions etc as used at runtime but as the root user, executing /bin/bash,
-'test', and will run interactively (eg -ti).  This allows you to enter
+named 'test', and will run interactively (eg -ti).  This allows you to enter
-the container and run commands to see what is failing, perhaps install
+the container and run commands to see what is failing, perhaps install strace
-strace and strace the command etc.  You can also verify configurations
+and strace the command etc.  You can also verify configurations or any other
-or any other debugging task you may have.
+debugging task you may have.
 
 Debugging docker-puppet.py
 --------------------------
@@ -102,55 +102,48 @@ of tripleo-heat-templates.  When writing these tasks, it's useful to be
 able to run them manually instead of running them as part of the entire
 stack. To do so, one can run the script as shown below::
 
-  CONFG=/path/to/task.json $THT_ROOT/docker/docker-puppet.py
+  CONFIG=/path/to/task.json /path/to/docker-puppet.py
 
 The json file must follow the following form::
 
     [
-        [
+        {
-            $CONFIG_VOLUME,
+            "config_image": ...,
-            $PUPPET_TAGS,
+            "config_volume": ...,
-            $PUPPET_MANIFEST,
+            "puppet_tags": ...,
-            $DOCKER_IMAGE,
+            "step_config": ...
-            $DOCKER_VOLUMES
+        }
-        ]
     ]
 
 
-Using a more realistic example. Given a `docker_puppet_task` section like this::
+Using a more realistic example. Given a `puppet_config` section like this::
 
-      docker_puppet_tasks:
+      puppet_config:
-        step_2:
+        config_volume: glance_api
-          - 'mongodb_init_tasks'
+        puppet_tags: glance_api_config,glance_api_paste_ini,glance_swift_config,glance_cache_config
-          - 'mongodb_database,mongodb_user,mongodb_replset'
+        step_config: {get_attr: [GlanceApiPuppetBase, role_data, step_config]}
-          - 'include ::tripleo::profile::base::database::mongodb'
+        config_image:
-          - list_join:
+          list_join:
             - '/'
-            - [ {get_param: DockerNamespace}, {get_param: DockerMongodbImage} ]
+            - [ {get_param: DockerNamespace}, {get_param: DockerGlanceApiImage} ]
-          - - "mongodb:/var/lib/mongodb"
-            - "logs:/var/log/kolla:ro"
 
 
 Would generated a json file called `/var/lib/docker-puppet-tasks2.json` that looks like::
 
     [
-        [
+        {
-            mongodb_init_tasks,
+            "config_image": "tripleoupstream/centos-binary-glance-api:latest",
-            "mongodb_database,mongodb_user,mongodb_replset",
+            "config_volume": "glance_api",
-            "include ::tripleo::profile::base::database::mongodb",
+            "puppet_tags": "glance_api_config,glance_api_paste_ini,glance_swift_config,glance_cache_config",
-            "tripleoupstream/centos-binary-mongodb:latest",
+            "step_config": "include ::tripleo::profile::base::glance::api\n"
-            [
+        }
-                "mongodb:/var/lib/mongodb",
-                "logs:/var/log/kolla:ro"
-            ]
-        ]
     ]
 
 
 Setting the path to the above json file as value to the `CONFIG` var passed to
 `docker-puppet.py` will create a container using the
-`centos-binary-mongodb:latest` image and it'll run the puppet puppet tags listed
+`centos-binary-glance-api:latest` image and it and run puppet on a catalog
-in the second item of the array.
+restricted to the given puppet `puppet_tags`.
 
 As mentioned above, it's possible to create custom json files and call
 `docker-puppet.py` manually, which makes developing and debugging puppet steps

doc/source/containers_deployment/undercloud.rst

@@ -47,10 +47,7 @@ installation:
 
 * python-tripleoclient >= Pike
 * python-openstackclient >= Pike
-* python-heat-agent-hiera >= Pike
+* openstack-heat-agents >= Pike
-* python-heat-agent-apply-config >= Pike
-* python-heat-agent-puppet >= Pike
-* python-heat-agent-docker-cmd >= Pike
 * docker >= 1.12.5
 * openvswitch (minimum version supported by neutron)
 
@@ -102,11 +99,12 @@ The following command will install an undercloud with ironic, mistral and zaqar
       --templates=$THT_ROOT \
       --local-ip=$YOUR_SERVER_IP \
       --keep-running \
-      -e $THT_ROOT/environments/services/ironic.yaml \
+      -e $THT_ROOT/environments/services-docker/ironic.yaml \
-      -e $THT_ROOT/environments/services/mistral.yaml \
+      -e $THT_ROOT/environments/services-docker/mistral.yaml \
-      -e $THT_ROOT/environments/services/zaqar.yaml \
+      -e $THT_ROOT/environments/services-docker/zaqar.yaml \
       -e $THT_ROOT/environments/docker.yaml \
-      -e /home/stack/custom.yaml
+      -e $THT_ROOT/environments/mongodb-nojournal.yaml \
+      -e $HOME/custom.yaml
 
 
 The `keep-running` flag will keep the `openstack undercloud deploy` process
@@ -128,14 +126,12 @@ Cleaning up
 The following commands will help cleaning up your undercloud environment to
 start the deployment from scratch:
 
-To stop and remove all running containers (this will remove non-openstack
+To stop and remove all running containers::
-containers too)::
 
-    $ sudo docker stop $(sudo docker ps -a -q)
+    $ sudo docker ps -qa --filter label=managed_by=docker-cmd | xargs sudo docker rm -f
-    $ sudo docker rm $(sudo docker ps -a -q)
 
-To remove the existing volumes (bear in mind this will remove your database
+To remove the existing named volumes (bear in mind this will remove your
-files too)::
+database files too)::
 
     $ sudo docker volume rm $(sudo docker volume ls -q)