openstack/tripleo-docs
Code Issues Proposed changes

Composable services within roles Tutorial

Browse Source 
Composable services walkthrough
This is a documentation update to add a some guidelines and references for THT
and composable services within roles.

Change-Id: Ic8f0ddde3f19d41ffe13abb97f5e38d5d63df2a8
Partially-implements: blueprint composable-services-within-roles
This commit is contained in:
Carlos Camacho 2016-04-30 10:14:56 +02:00 committed by Steven Hardy
parent 3d2893402c
commit 16263bd480
7 changed files with 469 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/source/index.rst
View File

@ -15,6 +15,7 @@ Contents:
advanced_deployment/advanced_deployment
post_deployment/post_deployment
troubleshooting/troubleshooting
tht_walkthrough/tht_walkthrough
contributions/contributions

54
doc/source/tht_walkthrough/changes-puppet-tripleo.rst Normal file
View File

@ -0,0 +1,54 @@
Updating puppet-tripleo
-----------------------
The puppet manifests that currently define overcloud node configuration are
moved from the tripleo-heat-templates to new puppet-tripleo class definitions
as part of the composable services approach. In next iterations, all service
configuration should be moved also to puppet-tripleo.
This section considers the addition of the ntp definition to puppet-tripleo.
Folder structure convention
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Services should be defined in the services folder, depending on the service
purpose.
::
manifests
profile/base ---> To host all services not using pacemaker.
time ---> Specific folder for time services (NTP, timezone, Chrony among others).
ntp.pp ---> Puppet manifest to configure the service.
.. note::
For further information related to the current folders manifests structure
refer to the `puppet-tripleo repository`_.
Adding the puppet manifest
~~~~~~~~~~~~~~~~~~~~~~~~~~
This step will reference how the puppet logic should be organized in
puppet-tripleo.
Inside the manifests folder, add the service manifest following the folder
structure (``manifests/profile/base/time/ntp.pp``) as:
::
class tripleo::profile::base::time::ntp (
#We get the configuration step in which we can choose which steps to execute
$step = hiera('step'),
) {
#step assigned for core modules.
#(Check for further referencies about the configuration steps)
#http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/puppet/services/README.rst
if ($step >= 2){
#We will call the NTP puppet class and assign our configuration values.
#If needed additional Puppet packages can be added/installed by using the repo tripleo-puppet-elements
if count($ntpservers) > 0 {
include ::ntp
}
}
}
If users have followed all the previous steps, they should be able to configure
their services using the composable services within roles guidelines.

199
doc/source/tht_walkthrough/changes-tht.rst Normal file
View File

@ -0,0 +1,199 @@
Updating tripleo-heat-templates
---------------------------------
This section will describe the changes needed for tripleo-heat-templates.
Folder structure convention for tripleo-heat-templates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Services should be defined in the services folder, depending on the service
purpose.
::
puppet
services ---> To host all services.
<service type> ---> Folder to store a specific type services (If time, will store time based services like: NTP, timezone, Chrony among others).
<service name>.yaml ---> Heat template defining per-service configuration.
<service name>-base.yaml ---> Heat template defining common service configuration.
.. note::
No puppet manifests may be defined in the `THT repository`_, they
should go to the `puppet-tripleo repository`_ instead.
.. note::
The use of a base heat template (<service>-base.yaml) is necessary in cases where
a given 'service' (e.g. "heat") is comprised of a number of individual
component services (e.g. heat-api, heat-engine) which need to share some
of the base configuration (such as rabbit credentials).
Using a base template in those cases means we don't need to
duplicate that configuration.
Refer to: https://review.openstack.org/#/c/313577/ for further details.
Also, refer to :ref:`duplicated-parameters` for an use-case description.
Changes list
~~~~~~~~~~~~
The list of changes in THT are:
- If there is any configuration of the given feature/service
in any of the ``tripleo-heat-templates/puppet/manifests/*.pp``
files, then this will need to be removed and migrated to the
puppet-tripleo repository.
- Create a service type specific folder in the root services folder
(``puppet/services/time``).
- Create a heat template for the service inside the puppet/services folder
(``puppet/services/time/ntp.yaml``).
- Optionally, create a common heat template to reuse common configuration
data, which is referenced from each per-service heat template
(``puppet/services/time/ntp-base.yaml``).
Step 1 - Updating puppet references
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Remove all puppet references for the composable service from the current
manifests (*.pp). All the puppet logic will live in the puppet-tripleo
repository based on a configuration step, so it is mandatory to remove all the
puppet references from tripleo-heat-templates.
The updated .pp files for the NTP example were:
- ``puppet/manifests/overcloud_cephstorage.pp``
- ``puppet/manifests/overcloud_compute.pp``
- ``puppet/manifests/overcloud_controller.pp``
- ``puppet/manifests/overcloud_controller_pacemaker.pp``
- ``puppet/manifests/overcloud_object.pp``
- ``puppet/manifests/overcloud_volume.pp``
Step 2 - overcloud-resource-registry-puppet.yaml resource registry changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The resource ``OS::TripleO::Services::Ntp`` must be defined in the resource
registry (``overcloud-resource-registry-puppet.yaml``)
Create a new resource type alias which references the per-service
heat template file, as described above.
By updating the resource registry we are forcing to use a nested template to
configure our resources. In the example case the created resource
(OS::TripleO::Services::Ntp), will point to the corresponding service yaml file
(puppet/services/time/ntp.yaml).
Step 3 - overcloud.yaml initial changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Mandatory services should be added to the ControllerServices parameter default value,
which defines all the services enabled by default in the controller(s).
From ``overcloud.yaml`` in the parameters section find:
::
ControllerServices:
default:
- OS::TripleO::Services::Keystone
- OS::TripleO::Services::GlanceApi
- OS::TripleO::Services::GlanceRegistry
- OS::TripleO::Services::Ntp ---> New service deployed in the controller overcloud
description: A list of service resources (configured in the Heat
resource_registry) which represent nested stacks
for each service that should get installed on the Controllers.
type: comma_delimited_list
Update this section with your new service to be deployed to the controllers in
the overcloud.
The parameter ControllerServices will be used by the ControllerServiceChain
resource as follows:
::
ControllerServiceChain:
type: OS::TripleO::Services
properties:
Services: {get_param: ControllerServices}
EndpointMap: {get_attr: [EndpointMap, endpoint_map]}
THT changes for all the different roles are covered in:
- https://review.openstack.org/#/c/310421/ (tripleo-heat-templates controller)
- https://review.openstack.org/#/c/330916/ (tripleo-heat-templates compute)
- https://review.openstack.org/#/c/330921/ (tripleo-heat-templates cephstorage)
- https://review.openstack.org/#/c/330923/ (tripleo-heat-templates objectstorage)
.. note::
In the case of the controller services, they are defined as part of the ControllerServiceChain
resource. If it is needed to add optional services, they need to be appended to the current
services list defined by the default value of the ControllerServices parameter.
Step 4 - Create the services yaml files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Create: ``puppet/services/time/ntp.yaml``
This file will have all the configuration details for the service to be
configured.
::
heat_template_version: 2016-04-08
description: >
NTP service deployment using puppet, this YAML file
creates the interface between the HOT template
and the puppet manifest that actually installs
and configure NTP.
parameters:
EndpointMap:
default: {}
description: Mapping of service endpoint -> protocol. Typically set
via parameter_defaults in the resource registry.
type: json
NtpServers:
default: ['0.pool.ntp.org', '1.pool.ntp.org']
description: NTP servers
type: comma_delimited_list
NtpInterfaces:
default: ['0.0.0.0']
description: Listening interfaces
type: comma_delimited_list
outputs:
role_data:
description: Role ntp using composable services.
value:
config_settings:
ntp::ntpservers: {get_param: NtpServers}
ntp::ntpinterfaces: {get_param: NtpInterfaces}
step_config: |
include ::tripleo::profile::base::time::ntp
.. note::
It is required for all service templates to accept the EndpointMap parameter,
all other parameters are optional and may be defined per-service. Care should
be taken to avoid naming collisions between service parameters, e.g via using
the service name as a prefix, "Ntp" in this example.
Service templates should output a role_data value, which is a mapping containing
"config_settings" which is a mapping of hiera key/value pairs required to configure
the service, and "step_config", which is a puppet manifest fragment that references
the puppet-tripleo profile that configures the service.
If it is needed, the templates can be decomposed to remove
duplicated parameters among different deployment environments
(i.e. using pacemaker). To do this see
section :ref:`duplicated-parameters`.

104
doc/source/tht_walkthrough/design-patterns.rst Normal file
View File

@ -0,0 +1,104 @@
THT design patterns
-------------------
.. _duplicated-parameters:
Duplicated parameters
~~~~~~~~~~~~~~~~~~~~~
Problem: When defining multiple related services, it can be necessary
to define the same parameters (such as rabbit or DB credentials) in
multiple service templates. To avoid this, it is possible to define a
"base" heat template that contains the common parameters and config_settings
mapping for those services that require it.
This pattern will describe how to avoid duplicated parameters in the THT yaml
files.
``mongodb-base.yaml``: This file should have all the common parameters between
the different environments (With pacemaker and without pacemaker).
::
heat_template_version: 2016-04-08
description: >
Configuration details for MongoDB service using composable roles
parameters:
MongoDbNoJournal:
default: false
description: Should MongoDb journaling be disabled
type: boolean
MongoDbIPv6:
default: false
description: Enable IPv6 if MongoDB VIP is IPv6
type: boolean
MongoDbReplset:
type: string
default: "tripleo"
outputs:
role_data:
description: Role data for the MongoDB base service.
value:
config_settings:
mongodb::server::nojournal: {get_param: MongoDbNoJournal}
mongodb::server::ipv6: {get_param: MongoDbIPv6}
mongodb::server::replset: {get_param: MongoDbReplset}
In this way we will be able to reuse the common parameter among all the
template files requiring it.
Referencing the common parameter:
``mongodb.yaml``: Will have specific parameters to deploy mongodb without
pacemaker.
::
heat_template_version: 2016-04-08
description: >
MongoDb service deployment using puppet
parameters:
#Parameters not used EndpointMap
EndpointMap:
default: {}
description: Mapping of service endpoint -> protocol. Typically set
via parameter_defaults in the resource registry.
type: json
resources:
MongoDbBase:
type: ./mongodb-base.yaml
outputs:
role_data:
description: Service mongodb using composable services.
value:
config_settings:
map_merge:
- get_attr: [MongoDbBase, role_data, config_settings]
- mongodb::server::service_manage: True
step_config: |
include ::tripleo::profile::base::database::mongodb
In this case mongodb.yaml is using all the common parameter added in the
MongoDbBase resource.
If using the parameter 'EndpointMap' in the base template, you must the pass it from from the service file,
and even if it is not used in the service template, it must still be defined.
In the service file:
::
parameters:
EndpointMap:
default: {}
description: Mapping of service endpoint -> protocol. Typically set
via parameter_defaults in the resource registry.
type: json
resources:
<ServiceName>ServiceBase:
type: ./<ServiceName>-base.yaml
properties:
EndpointMap: {get_param: EndpointMap}
This will pass the endpoint information to the base config file.
.. note::
Even if the EndpointMap parameter is optional in the base template,
for consistency is advised always using it in all service templates.

72
doc/source/tht_walkthrough/introduction.rst Normal file
View File

@ -0,0 +1,72 @@
Introduction
------------
The initial scope of this tutorial is to create a brief walkthrough with some
guidelines and naming conventions for future modules and features aligned with
the composable services architecture. Regarding the example described in this
tutorial, which leads to align an _existing_ 'non-composable' service implementation
with the composable roles approach, it is important to notice that a similar approach would be
followed if a user needed to add an entirely new service to a tripleo deployment.
.. _puppet/manifests: https://github.com/openstack/tripleo-heat-templates/tree/3d01f650f18b9e4f1892a6d9aa17f1bfc99b5091/puppet/manifests
The puppet manifests used to configure services on overcloud nodes currently
reside in the tripleo-heat-templates repository, in the folder `puppet/manifests`_.
In order to properly organize and structure the code, all
manifests will be re-defined in the puppet-tripleo repository, and adapted to
the `composable services architecture`_
The use case for this example uses NTP as a service installed by default among
the OpenStack deployment. In this particular case, NTP is installed and
configured based on the code from puppet manifests:
``puppet/manifests/overcloud_cephstorage.pp``,
``puppet/manifests/overcloud_volume.pp``,
``puppet/manifests/overcloud_object.pp``,
``puppet/manifests/overcloud_compute.pp``,
``puppet/manifests/overcloud_controller.pp`` and
``puppet/manifests/overcloud_controller_pacemaker.pp``
Which means that NTP will be installed everywhere in the overcloud, so the
tutorial will describe the process of refactoring the code from those files
in order move it to the puppet-tripleo repository.
This tutorial is divided into several steps, according to different changes
that need to be added to the structure of tripleo-heat-templates and
puppet-tripleo.
Relevant repositories in this guide
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- tripleo-heat-templates: All the tripleo-heat-templates (aka THT) logic.
- puppet-tripleo: TripleO puppet manifests used to deploy the overcloud services.
- tripleo-puppet-elements: References puppet modules used by TripleO to deploy the overcloud services.
(Not used in this tutorial)
Gerrit patches used in this example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The gerrit patches used to describe this walkthrough are:
- https://review.openstack.org/#/c/310725/ (puppet-tripleo)
- https://review.openstack.org/#/c/310421/ (tripleo-heat-templates controller)
- https://review.openstack.org/#/c/330916/ (tripleo-heat-templates compute)
- https://review.openstack.org/#/c/330921/ (tripleo-heat-templates cephstorage)
- https://review.openstack.org/#/c/330923/ (tripleo-heat-templates objectstorage)
Change prerequisites
~~~~~~~~~~~~~~~~~~~~~
The controller services are defined and configured via Heat resource chains. In
the proposed patch (https://review.openstack.org/#/c/259568) controller
services will be wired to a new Heat feature that allows it to dynamically include
a set of nested stacks representing individual services via a Heat resource
chain. The current example will use this interface to decompose the controller
role into isolated services.

18
doc/source/tht_walkthrough/summary.rst Normal file
View File

@ -0,0 +1,18 @@
Summary
------
References:
#. https://etherpad.openstack.org/p/tripleo-composable-roles-work
#. https://review.openstack.org/#/c/245804/2/specs/mitaka/composable-services-within-roles.rst
#. https://review.openstack.org/#/q/topic:composable_service
#. http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/template_deploy.html
#. http://hardysteven.blogspot.com.es/2015/05/tripleo-heat-templates-part-1-roles-and.html
#. http://hardysteven.blogspot.com.es/2015/05/tripleo-heat-templates-part-2-node.html
#. http://hardysteven.blogspot.com.es/2015/05/tripleo-heat-templates-part-3-cluster.html

21
doc/source/tht_walkthrough/tht_walkthrough.rst Normal file
View File

@ -0,0 +1,21 @@
Composable services tutorial
============================
.. _composable services architecture: https://blueprints.launchpad.net/tripleo/+spec/composable-services-within-roles
.. _THT repository: https://github.com/openstack/tripleo-heat-templates/tree/master/puppet/services
.. _puppet-tripleo repository: https://github.com/openstack/puppet-tripleo/tree/master/manifests/profile
This guide will be a walkthrough related to how to add new services to a TripleO
deployment through additions to the tripleo-heat-templates and puppet-tripleo
repositories, using part of the architecture defined in the `composable services architecture`_
.. note::
No puppet manifests may be defined in the `THT repository`_, they
should go to the `puppet-tripleo repository`_ instead.
.. include:: introduction.rst
.. include:: changes-tht.rst
.. include:: changes-puppet-tripleo.rst
.. include:: design-patterns.rst
.. include:: summary.rst