e66eeb8c3c
This removes ansible configuration for the linaro cloud itself and the linaro cloud mirror. This cloud is in the process of going away and having these nodes in our inventory is creating base jobs failures due to unreachable nodes. This then dominoes into not running the LE refresh job and now some certs are not getting renewed. Clean this all up so that the rest of our systems are happy. Note that we don't fully clean up the idea of an unmanaged group as there may be other locations we want to do something similar (OpenMetal perhaps?). We also don't remove the openstack clouds.yaml entries for the linaro cloud yet. It isn't entirely clear when things will go offline, but it may be as late as August 10 so we keep those credentials around as they may be useful until then. Change-Id: Idd6b455de8da2aa9901bf989b1d131f1f4533420
154 lines
5.5 KiB
ReStructuredText
154 lines
5.5 KiB
ReStructuredText
:title: letsencrypt
|
|
|
|
.. _letsencrypt:
|
|
|
|
Let's Encrypt Certificates
|
|
##########################
|
|
|
|
We support provisioning certificates from https://letsencrypt.org for
|
|
hosts in the ``opendev.org`` namespace.
|
|
|
|
At a Glance
|
|
===========
|
|
|
|
:Ansible:
|
|
* :git_file:`playbooks/letsencrypt.yaml`
|
|
* :git_file:`playbooks/roles/letsencrypt-acme-sh-install`
|
|
* :git_file:`playbooks/roles/letsencrypt-request-certs`
|
|
* :git_file:`playbooks/roles/letsencrypt-install-txt-record`
|
|
* :git_file:`playbooks/roles/letsecnrypt-create-certs`
|
|
:Resources:
|
|
* https://letsencrypt.org
|
|
* https://github.com/Neilpang/acme.sh
|
|
:Chat:
|
|
* #opendev on OFTC
|
|
|
|
Overview
|
|
========
|
|
|
|
We support automatic provisioning of certificates from Let's Encrypt
|
|
to hosts in the ``opendev.org`` domain.
|
|
|
|
This is implemented in OpenDev via the roles driven from
|
|
:git_file:`playbooks/letsencrypt.yaml`. The overall
|
|
actions implemented by the above roles are roughly:
|
|
|
|
* Hosts that want a certificate use the ``amce.sh`` tool to request it
|
|
from the Let's Encrypt CA.
|
|
|
|
Creation or renewal requests receive a TXT record authentication
|
|
value that must be published to prove ownership of the domain. We
|
|
implement this by making the challenge-request hostname
|
|
``_acme-challenge.hostname.opendev.org`` a ``CNAME`` record to a
|
|
special "signing domain" ``acme.opendev.org``.
|
|
|
|
Note if valid certificates are present and they are not within the
|
|
renewal period (which is most of the time) no further action is
|
|
taken.
|
|
|
|
* The provided TXT record authentication values are installed and
|
|
published to the ``acme.opendev.org`` domain via the OpenDev
|
|
nameservers.
|
|
|
|
* The host can now finalise certificate creation. Let's Encrypt
|
|
checks ``_acme-chellenge.hostname.opendev.org``, which is a
|
|
``CNAME`` to ``acme.opendev.org``. Let's Encrypt then enumerates
|
|
the TXT records there, and once finding the required key will return
|
|
the signed keys to the host, which saves them to disk.
|
|
|
|
|
|
Configuring a host to get certificates
|
|
======================================
|
|
|
|
A basic configuration consists of the following steps:
|
|
|
|
1. Ensure the host is matched by the ``letsencrypt`` group in
|
|
:git_file:`inventory/groups.yaml`.
|
|
#. DNS entries for ``_acme-chellenge.hostname`` as a ``CNAME`` to
|
|
``opendev.org`` must be added and live in the ``opendev.org``
|
|
`zone.db
|
|
<https://opendev.org/opendev/zone-opendev.org/src/branch/master/zones/opendev.org/zone.db>`__
|
|
file. Follow the other examples to ensure other fields such as
|
|
``CAA`` records are set too.
|
|
|
|
Take care to list `all` hostnames that you wish covered by the
|
|
certificate (e.g. ``hostname01.opendev.org`` and
|
|
``hostname.opendev.org``)
|
|
#. Configure the certificates to be issued to the host.
|
|
|
|
The roles look for certificate configuration in a
|
|
``letsencrypt_certs`` variable defined for each host. This is
|
|
usually done via specific host variables in
|
|
``inventory/service/host_vars/<hostname>.opendev.org.yaml``. For a simple
|
|
host that wants a single certificate to cover its numeric hostname
|
|
and regular ``CNAME`` this would look like ::
|
|
|
|
letsencrypt_certs:
|
|
hostname01-opendev-org:
|
|
- hostname01.opendev.org
|
|
- hostname.opendev.org
|
|
|
|
This will result in certificate material in
|
|
``/etc/letsencrypt-certs/hostname01.opendev.org/`` on the host.
|
|
|
|
Note that the "certificate name" dictionary keys (just
|
|
``hostname01-opendev-org`` above) are essentially a free-form
|
|
string, but are used in the next step. Follow the naming
|
|
conventions for similar hosts.
|
|
|
|
For full details, including information on issuing multiple
|
|
certificates for a single host, see
|
|
:git_file:`playbooks/roles/letsencrypt-request-certs/README.rst`.
|
|
#. Define a handler for certificate creation and renewal actions.
|
|
|
|
When the certificate is created or renewed, the
|
|
``letsencrypt-create-certs`` role calls a predefined handler so
|
|
action can be taken. This handler name is constructed by
|
|
prepending ``letsencrypt updated`` to the certificate name above.
|
|
Thus in this example it would be ::
|
|
|
|
- name: letsencrypt updated hostname01-opendev-org
|
|
...
|
|
|
|
Usually these handlers are defined centrally in
|
|
:git_file:`playbooks/roles/letsencrypt-create-certs/handlers/main.yaml`
|
|
and common tasks such as restarting Apache have pre-defined tasks
|
|
available for easy import.
|
|
|
|
You may choose to define the handler in another way, but it *must*
|
|
exist (Ansible does not have a way to say "call this handler only
|
|
if it exists", thus a missing handler will cause an Ansible error
|
|
at runtime).
|
|
|
|
Debugging
|
|
=========
|
|
|
|
The Ansible run logs on ``bridge.opendev.org`` should be consulted if
|
|
the certificate material is not being created as expected.
|
|
|
|
Hosts will log their ``acme.sh`` output to
|
|
``/var/log/acme.sh/acme.sh.log``
|
|
|
|
The `G Suite Toolbox Dig <https://toolbox.googleapps.com/apps/dig/>`__
|
|
tool can be useful for checking DNS entries from a remote location.
|
|
|
|
Refreshing keys
|
|
===============
|
|
|
|
In normal operation there should be no need to manually refresh keys
|
|
on hosts. However there have been situations (such as LetsEncrypt
|
|
revoking certificates made during a certain period due to bugs) which
|
|
may necessitate a manual renewal.
|
|
|
|
The best way to do this is to move the ``.conf`` files from
|
|
``/etc/letsencrypt-certs/<certname>`` on the affected host and allow
|
|
the next Ansible pulse to renew.
|
|
|
|
.. code-block:: console
|
|
|
|
# cd /etc/letsencrypt-certs/<name>
|
|
# rename 's/.conf/.conf.old/' *.conf
|
|
# tail -f /var/log/acme.sh/acme.sh.log
|
|
... watch and should be renewed on next pulse
|
|
# rm *.conf.old
|