Merge "Use puppet apply instead of puppet agent"
This commit is contained in:
Jenkins
Gerrit Code Review
commit
1dd4a9d02e
@ -5,10 +5,11 @@
|
||||
Puppet Master
|
||||
#############
|
||||
|
||||
Puppet agent is a mechanism use to pull puppet manifests and configuration
|
||||
from a centralized master. This means there is only one place that needs to
|
||||
hold secure information such as passwords, and only one location for the git
|
||||
repo holding the modules.
|
||||
The puppetmaster server is named puppetmaster for historical reasons - it
|
||||
no longer runs a puppetmaster process. There is a centralized 'hiera'
|
||||
database that contains secure information such as passwords. The puppetmaster
|
||||
server contains all of the ansible playbooks to run puppet apply
|
||||
as well as the scripts to create new servers.
|
||||
|
||||
At a Glance
|
||||
===========
|
||||
@ -25,50 +26,59 @@ At a Glance
|
||||
:Resources:
|
||||
* `Puppet Language Reference <https://docs.puppetlabs.com/references/latest/type.html>`_
|
||||
|
||||
Puppet Master
|
||||
-------------
|
||||
Puppet Driving Ansible Driving Puppet
|
||||
-------------------------------------
|
||||
|
||||
The puppet master is setup using a combination of Apache and mod passenger to
|
||||
ship the data to the clients.
|
||||
In OpenStack Infra, there are ansible playbooks that drive the running of
|
||||
``puppet apply`` on all of the hosts in the inventory. That process first
|
||||
copies appropriate ``hiera`` data files to each host, and when it is done
|
||||
it copies back the JSON report of the puppet run and submits it to
|
||||
``puppetdb``.
|
||||
|
||||
The cron jobs, current configuration files and more can be done with ``puppet
|
||||
apply`` but first some bootstrapping needs to be done.
|
||||
|
||||
You want to install these from puppetlabs' apt repo. There is a script in the
|
||||
root of the system-config repository that will setup and install the
|
||||
puppet client. After that you must install the puppetmaster and hiera (used to
|
||||
maintain secrets on the puppet master).
|
||||
You want to install these from puppetlabs' apt repo. There is a script,
|
||||
`:file:`install_puppet.sh` in the root of the system-config repository that
|
||||
will setup and install the puppet client. After that you must install the
|
||||
ansible playbooks and hiera config (used to maintain secrets).
|
||||
|
||||
Puppet 3 masters can run on Trusty, Precise, and Centos 6.
|
||||
Ansible and Puppet 3 is known to run on Precise, Trusty, Centos 6 and Centos 7.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo su -
|
||||
git clone https://git.openstack.org/openstack-infra/system-config /opt/system-config/production
|
||||
/opt/system-config/production/install_puppet.sh
|
||||
apt-get install puppetmaster-passenger hiera hiera-puppet
|
||||
|
||||
Finally, install the modules, fix your hostname and use ``puppet apply`` to
|
||||
finish configuration:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
bash /opt/system-config/production/install_puppet.sh
|
||||
bash /opt/system-config/production/install_modules.sh
|
||||
echo $REAL_HOSTNAME > /etc/hostname
|
||||
service hostname restart
|
||||
puppet apply --modulepath='/opt/system-config/production/modules:/etc/puppet/modules' -e 'include openstack_project::puppetmaster'
|
||||
|
||||
Note: Hiera uses a systemwide configuration file in ``/etc/puppet/hiera.yaml``
|
||||
Hiera uses a systemwide configuration file in ``/etc/puppet/hiera.yaml``
|
||||
and this setup supports multiple configurations. The two sets of environments
|
||||
that OpenStack Infrastructure uses are ``production`` and ``development``.
|
||||
``production`` is the default is and the environment used when nothing else is
|
||||
specified. Then the configuration needs to be placed into common.yaml in
|
||||
``production`` is the default and the environment used when nothing else is
|
||||
specified.
|
||||
|
||||
The hiera configuration is placed by puppet apply into common.yaml in
|
||||
``/etc/puppet/hieradata/production`` and ``/etc/puppet/hieradata/development``.
|
||||
The values are simple key-value pairs in yaml format. The keys needed are the
|
||||
keys referenced in your ``site.pp``, their values are typically obvious
|
||||
(strings, lists of strings). ``/etc/puppet/hieradata/`` and below should be
|
||||
owned by ``puppet:puppet`` and have mode ``0711``. The actual ``common.yaml``
|
||||
file should have mode 0600.
|
||||
owned by ``puppet:puppet`` and have mode ``0711``.
|
||||
|
||||
Below the ``hieradata`` directory, there should be a ``common.yaml`` file where
|
||||
settings that should be available to all servers in the infrastructure go,
|
||||
and then two directories full of files. The first is ``fqdn`` which should
|
||||
contain a yaml file for every server in the infrastructure named
|
||||
``${fqdn_of_server}.yaml``. That file has secrets that are only for that
|
||||
server. Additionally, some servers can have a ``$group`` defined in
|
||||
``manifests/site.pp``. There can be a correspondingly named yaml file in the
|
||||
``group`` directory that contains secrets to be made available to each
|
||||
server in the group.
|
||||
|
||||
All of the actual yaml files should have mode 0600 and be owned by root.
|
||||
|
||||
Adding a node
|
||||
-------------
|
||||
@ -86,70 +96,48 @@ For manual bootstrap, you need to run on the new server connecting
|
||||
wget https://git.openstack.org/cgit/openstack-infra/system-config/plain/install_puppet.sh
|
||||
bash -x install_puppet.sh
|
||||
|
||||
The node then needs to be configured to set a fixed hostname and the
|
||||
hostname of the puppet master with the following additions to
|
||||
``/etc/puppet/puppet.conf``:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[main]
|
||||
server=puppetmaster.openstack.org
|
||||
certname=review.openstack.org
|
||||
|
||||
The cert signing process needs to be started with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo puppet agent --test
|
||||
|
||||
This will make a request to the puppet master to have its SSL cert signed.
|
||||
On the puppet master:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo puppet cert list
|
||||
|
||||
You should get a list of entries similar to the one below::
|
||||
|
||||
review.openstack.org (44:18:BB:DF:08:50:62:70:17:07:82:1F:D5:70:0E:BF)
|
||||
|
||||
If you see the new node there you can sign its cert on the puppet master with:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo puppet cert sign review.openstack.org
|
||||
|
||||
Once the cert is signed, the puppet running orchestration will pick up
|
||||
the node and run puppet on it as needed.
|
||||
|
||||
Running Puppet on Nodes
|
||||
-----------------------
|
||||
|
||||
In OpenStack's Infrastructure, puppet runs are triggered from a cronjob
|
||||
running on the puppetmaster which in turn runs a single run of puppet on
|
||||
each host we know about. We do not use the daemon mode of puppet agent
|
||||
because it experiences random hangs, and also does not allow us to control
|
||||
sequencing in any meaningful way.
|
||||
running on the puppetmaster which in turn runs a single run of puppet apply on
|
||||
each host we know about.
|
||||
|
||||
The entry point for this process is ``/opt/system-config/production/run_all.sh``
|
||||
|
||||
There are a set of nodes, which are configured in puppet as "override" nodes,
|
||||
which are run in sequence before the rest of the nodes are run in parallel.
|
||||
There are a few sets of nodes which have their own playbooks so that they
|
||||
are run in sequence before the rest of the nodes are run in parallel.
|
||||
At the moment, this allows creation of git repos on the git slaves before
|
||||
creation of the master repos on the gerrit server.
|
||||
|
||||
If an admin needs to run puppet by hand, it's a simple matter of either
|
||||
logging in to the server in question and running
|
||||
`puppet apply /opt/system-config/production/manifests/site.pp` or, on the
|
||||
puppetmaster, running
|
||||
`ansible-playbook --limit='$HOST;localhost' /opt/system-config/production/playbooks/remote_puppet_all.yaml`
|
||||
as root, where `$HOST` is the host you want to run puppet on. If you are
|
||||
working with git, gerrit or afs servers, you'll want to replace
|
||||
`remote_puppet_all.yaml` with the appropriate specific playbook.
|
||||
The `;localhost` is important as some of the plays depend on performing a task
|
||||
on the localhost before continuing to the host in question, and without it in
|
||||
the limit section, the tasks for the host will have undefined values.
|
||||
|
||||
Testing new puppet code can be done via `puppet apply --noop` or by
|
||||
constructing a VM with a puppet install in it and just running `puppet apply`
|
||||
on the code in question. This should actually make it fairly easy to test
|
||||
how production works in a more self-contained manner.
|
||||
|
||||
|
||||
Disabling Puppet on Nodes
|
||||
-------------------------
|
||||
|
||||
In the case of needing to disable the running of puppet on a node, it's a
|
||||
simple matter of adding an entry to the ansible inventory "disabled" group.
|
||||
See the :ref:`disable-enable-puppet` section for more details.
|
||||
|
||||
Important Notes
|
||||
---------------
|
||||
|
||||
#. Make sure the site manifest **does not** include the puppet cron job, this
|
||||
conflicts with puppet master and can cause issues. The initial puppet run
|
||||
that create users should be done using the puppet agent configuration above.
|
||||
|
||||
#. If you do not see the cert in the master's cert list the agent's
|
||||
``/var/log/syslog`` should have an entry showing you why.
|
||||
that create users should be done using the puppet apply configuration above.
|
||||
|
||||
@ -279,6 +279,8 @@ repository ``https://git.openstack.org/openstack-infra/system-config``. This
|
||||
tool is run from a checkout on the puppetmaster - please see :file:`launch/README`
|
||||
for detailed instructions.
|
||||
|
||||
.. _disable-enable-puppet:
|
||||
|
||||
Disable/Enable Puppet
|
||||
=====================
|
||||
|
||||
|
||||
@ -1,4 +1,5 @@
|
||||
puppetmaster: puppetmaster.openstack.org
|
||||
copy_hieradata: true
|
||||
copy_puppet: true
|
||||
manifest: /opt/system-config/production/manifests/site.pp
|
||||
manifest_base: /opt/system-config
|
||||
puppetdb: https://puppetdb.openstack.org:8081
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user