starlingx/docs
Code Issues Proposed changes

Updates on Certificate Management (pick)

Browse Source 
Removed rst substitution from tables and inline markups.

Updated table and reestructured sections in the overview.

Fixed issues, reworded paragraphs, changed titles.

Deleted unnecessary sections, added a new item to section and fixed editorial issues.

Fixed editorial and formatting issues.

Fixed more editorial and formatting issues.

Fixed formatting and editorial issues.

Added command line.

Fixed command line.

Signed-off-by: Elisamara Aoki Goncalves <elisamaraaoki.goncalves@windriver.com>
Change-Id: I69874db16c76d5aceac706f2b8033771780500ca
This commit is contained in:
Elisamara Aoki Goncalves 2021-10-21 16:20:15 -03:00
parent 0c65a21c2e
commit 4d8775ca61
22 changed files with 1230 additions and 295 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

73
doc/source/admintasks/kubernetes/installing-updating-the-docker-registry-certificate.rst
View File

@ -2,43 +2,56 @@
.. idr1582032622279
.. _installing-updating-the-docker-registry-certificate:
====================================================
Install/Update the Local Docker Registry Certificate
====================================================
==========================================
Install/Update Local Registry Certificates
==========================================
.. warning::
By default a self-signed certificate is generated at installation time for
the registry API. This applies to standalone system, central cloud and
subclouds of |DC| system. For more secure access, it is strongly recommended
to update the default self-signed certificate with an intermediate or Root
|CA|-signed certificate.
The local Docker registry provides secure HTTPS access using the registry API.
.. rubric:: |context|
By default a self-signed certificate is generated at installation time for the
registry API. For more secure access, an intermediate or Root CA-signed
certificate is strongly recommended.
The intermediate or Root CA-signed certificate for the registry must have at
least the following |SANs|: DNS:registry.local, DNS:registry.central, IP
Address:<oam-floating-ip-address>, IP Address:<mgmt-floating-ip-address>. Use
the :command:`system addrpool-list` command to get the |OAM| floating IP
The intermediate or Root |CA|-signed certificate for the registry must have at
least the following |SANs|: ``DNS:registry.local``, ``DNS:registry.central``,
IP Address:<oam-floating-ip-address>, IP Address:<mgmt-floating-ip-address>.
Use the :command:`system addrpool-list` command to get the |OAM| floating IP
Address and management floating IP Address for your system. You can add any
additional |DNS| entry\(s\) that you have set up for your |OAM| floating IP
Address.
Use the following procedure to install an intermediate or Root CA-signed
.. note::
The ``DNS:registry.central`` can be omitted from |SANs| for
standalone system and subcloud of |DC| system.
The update procedure for any type of system (standalone, central cloud and
subcloud of |DC| system) is the same.
Use the following procedure to install an intermediate or Root |CA|-signed
certificate to either replace the default self-signed certificate or to replace
an expired or soon to expire certificate.
.. rubric:: |prereq|
Obtain an intermediate or Root CA-signed certificate and key from a trusted
intermediate or Root Certificate Authority \(CA\). Refer to the documentation
for the external Root CA that you are using, on how to create public
certificate and private key pairs, signed by an intermediate or Root CA, for
Obtain an intermediate or Root |CA|-signed certificate and key from a trusted
intermediate or Root Certificate Authority \(|CA|\). Refer to the documentation
for the external Root |CA| that you are using, on how to create public
certificate and private key pairs, signed by an intermediate or Root |CA|, for
HTTPS.
.. xreflink
For lab purposes, see |sec-doc|: :ref:`Create Certificates Locally
using openssl <create-certificates-locally-using-openssl>` to create an
Intermediate or test Root CA certificate and key, and use it to sign test
Intermediate or test Root |CA| certificate and key, and use it to sign test
certificates.
Put the Privacy Enhanced Mail \(PEM\) encoded versions of the certificate and
@ -57,7 +70,7 @@ information, see, :ref:`Display Certificates Installed on a System <utility-scri
.. _installing-updating-the-docker-registry-certificate-d271e71:
#. In order to enable internal use of the Docker registry certificate, update
the trusted CA list for this system with the Root CA associated with the
the trusted |CA| list for this system with the Root |CA| associated with the
Docker registry certificate.
.. code-block:: none
@ -68,8 +81,8 @@ information, see, :ref:`Display Certificates Installed on a System <utility-scri
**<pathTocertificate>**
is the path to the intermediate or Root CA certificate associated with the
Docker registry's intermediate or Root CA-signed certificate.
is the path to the intermediate or Root |CA| certificate associated with the
Docker registry's intermediate or Root |CA|-signed certificate.
#. Update the Docker registry certificate using the
:command:`certificate-install` command.
@ -85,4 +98,22 @@ information, see, :ref:`Display Certificates Installed on a System <utility-scri
**<pathTocertificateAndKey>**
is the path to the file containing both the Docker registry's Intermediate
or Root CA-signed certificate and private key to install.
or Root |CA|-signed certificate and private key to install.
In |DC| system, the server certificate of central registry and the server
certificate of subcloud’s local registry can be arranged to be generated from
the same root |CA| certificate.
In this case, the generated server certificates need to be installed on the
central cloud and each of the subclouds.
The root |CA| certificate only needs to install on central cloud, the |DC|
orchestration will sync the root |CA| certificate to all the subclouds.
---------------------------------
Renew local registry certificates
---------------------------------
The local registry certificate is not automatically renewed, user MUST renew
the certificate prior to expiry, otherwise a variety of system operations will
fail.

58
doc/source/security/kubernetes/add-a-trusted-ca.rst
View File

@ -3,9 +3,14 @@
.. _add-a-trusted-ca:
==============================
Manage Trusted CA Certificates
System Trusted CA Certificates
==============================
|prod| also supports the ability to update the trusted |CA| certificate
bundle on all nodes in the system. This is required, for example, when
container images are being pulled from an external Docker registry with a
certificate signed by a non-well-known |CA|.
Generally a trusted |CA| certificate needs to be added if |prod| clients on
the hosts will be connecting to server\(s\) secured with SSL and whose
certificate is signed by an unknown |CA|.
@ -14,13 +19,20 @@ certificate is signed by an unknown |CA|.
:local:
:depth: 1
For example, a trusted |CA| certificate is required if your helm charts or
yaml manifest files refer to images stored in a docker registry whose
For example, a trusted |CA| certificate is required if your Helm charts or
yaml manifest files refer to images stored in a Docker registry whose
certificate has been signed by an unknown Certificate Authority.
Trusted |CA| certificates can be added as part of the Ansible Bootstrap
Playbook or by using the StarlingX/system REST API or CLI after installation.
-----------------------------------------
Install/Uninstall Trusted CA certificates
-----------------------------------------
Trusted |CA| certificates can be added as part of the Ansible Bootstrap
Playbook or by using the StarlingX/system REST API or CLI after installation.
.. _add-a-trusted-ca-section-N1002C-N1001C-N10001:
@ -29,7 +41,7 @@ Ansible Bootstrap Playbook
--------------------------
A trusted |CA| certificate may need to be specified as an override parameter
for the Ansible Bootstrap Playbook. Specifically, if the docker registries,
for the Ansible Bootstrap Playbook. Specifically, if the Docker registries,
specified by the bootstrap overrides file, use a certificate signed by an
unknown |CA|. If this is the case then the ssl\_ca\_cert parameter needs to
be specified in the ansible overrides file, /home/sysadmin/localhost.yml, as
@ -48,13 +60,13 @@ and the file may contain one or more |CA| certificates.
.. _add-a-trusted-ca-section-N10047-N1001C-N10001:
-----------------------------------------------------
StarlingX/System CLI – Trusted CA Certificate Install
-----------------------------------------------------
-------------------------------------------
System CLI – Trusted CA certificate install
-------------------------------------------
After installation, adding a trusted |CA| to the |prod| system may be required.
This is the case if images stored in a docker registry, whose certificate has
been signed by a not-well-known Certificate Authority, are referred to by helm
This is the case if images stored in a Docker registry, whose certificate has
been signed by a not-well-known Certificate Authority, are referred to by Helm
charts and/or yaml manifest files.
Multiple trusted |CA| certificates can be added with single install command by
@ -69,7 +81,7 @@ From the command line, run the :command:`certificate-install` command.
~(keystone_admin)]$ system certificate-install -m ssl_ca <trusted-ca-bundle-pem-file>
where ``<trusted-ca-bundle-pem-file>`` contains 1 or more public certificates
of CAs that should be trusted by |prod|.
of |CAs| that should be trusted by |prod|.
The system will print a list of the certificates that were successfully
@ -98,9 +110,9 @@ For example:
.. _add-a-trusted-ca-section-phr-jw4-3mb:
-------------------------------------------------------
StarlingX/System CLI – Trusted CA Certificate Uninstall
-------------------------------------------------------
---------------------------------------------
System CLI – Trusted CA certificate uninstall
---------------------------------------------
To remove a Trusted |CA| Certificate, first list the trusted |CAs| by
running the following command:
@ -120,3 +132,23 @@ running the following command:
where, <UUID> is the UUID of the ssl\_ca certtype to be removed.
-----------------------------------
Update/Renew trusted CA certficates
-----------------------------------
.. warning::
System trusted |CA| certificates can not be auto renewed, as they are not
owned by |prod|.
The administrator should update the trusted |CA| certificates following the
install/uninstall procedure as requested, or when trusted |CA| certificates
in use are approaching expiration.
For example, when the |CA| certificate signing an external Docker registry’s
server certificate needs to be renewed, either because an external Docker
registry has a new server certificate signed by a new |CA|, or the |CA|
certificate signing an external Docker registry’s current server certificate
approaching expiration, the administrator can update the |CA| certificate for
the external Docker registry access by uninstalling the old one, and installing
the new one.

101
doc/source/security/kubernetes/dc-admin-endpoint-certificates-8fe7adf3f932.rst Normal file
View File

@ -0,0 +1,101 @@
.. _dc-admin-endpoint-certificates-8fe7adf3f932:
=============================================
Distributed Cloud Admin Endpoint Certificates
=============================================
|DC| admin endpoint certificates are used to secure communication between
System Controller and subclouds by HTTPs.
These certificates are generated, stored and auto renewed internally by the
system.
---------------------------------------------------
Certificate management for admin REST API endpoints
---------------------------------------------------
All messaging between System Controllers and subclouds in the |prod-dc|
system uses the admin REST API service endpoints, which are all configured for
secure HTTPS.
|prod| supports automated HTTPS certificate renewal for |prod-dc| admin
endpoints.
.. contents:: |minitoc|
:local:
:depth: 1
.. certificate-management-for-admin-rest--api-endpoints-section-lkn-ypk-xnb:
-------------------------------------
Certificates on the System Controller
-------------------------------------
In a |prod-dc| system, the HTTPS certificates for admin endpoints are
managed by |prod| internally.
.. note::
All renewal operations are automatic, and no user operation is required.
For admin endpoints, the System Controllers in a |prod-dc| system manages the
following certificates:
.. certificate-management-for-admin-rest--api-endpoints-ul-zdc-pmk-xnb:
- **DC-AdminEp-Root-CA certificate**: This certificate expires in 1825 days
\(approximately 5 years\). Renewal of this certificate starts 30 days prior
to expiry.
The Root |CA| certificate is renewed on the System Controller. When the
certificate is renewed, |prod| renews the Intermediate |CA|
certificates for all subclouds.
- **DC-AdminEp-Intermediate-CA certificate for 'each' subcloud**: This
certificate expires in 365 days. Renewal of this certificate starts 30 days
prior to expiry. This certificate is used for all subclouds that are
unmanaged.
- **DC-AdminEp-endpoint**: This certificate expires in 180 days. Renewal of
this certificate starts 30 days prior to expiry.
.. certificate-management-for-admin-rest--api-endpoints-section-qdd-xpk-xnb:
----------------------------
Certificates on the subcloud
----------------------------
For admin endpoints, the subcloud controllers manage the following
certificates:
.. certificate-management-for-admin-rest--api-endpoints-ul-x51-3qk-xnb:
- **DC-AdminEp-Intermediate-CA certificate**: The Intermediate |CA|
certificate for a subcloud is renewed on the System Controller. It is sent
to the subcloud using a Rest API. Therefore, a subcloud needs to be online
to receive the renewed certificate.
If the subcloud is offline at the time when the subcloud Intermediate |CA|
certificate is renewed, the subcloud status **dc-cert** displays
"out-of-sync". Certificate renewal continues once the subcloud is online.
When renewal completes, the status changes to "in-sync". Subclouds start
admin endpoint certificate renewal once subcloud Intermediate |CA|
certificate renewal is complete.
- **DC-AdminEp certificate for the Subcloud**: This certificate expires in
180 days. Renewal of this certificate starts 30 days prior to expiry.
When the admin endpoint certificate is renewed, a new |TLS| certificate is
generated. The new |TLS| certificate is used to provide |TLS| termination.
The System Controller audits subcloud AdminEp certificates daily. It also
audits subcloud admin endpoints when a subcloud becomes online or managed. If
the subcloud admin endpoint is "out-of-sync", the System Controller initiates
Intermediate |CA| certificate renewal, to force subcloud renewal of the admin
endpoint certificate.

108
doc/source/security/kubernetes/etcd-certificates-c1fc943e4a9c.rst Normal file
View File

@ -0,0 +1,108 @@
.. _etcd-certificates-c1fc943e4a9c:
=================
Etcd Certificates
=================
Etcd is the consistent and highly-available key value store used as Kubernetes
backing store for all cluster data. Interaction with etcd service is by secure
HTTPs where x509 certificates are required.
Etcd certificates in |prod| includes:
- Etcd Root |CA| certificiate
- Etcd server certificate
- Etcd client certificate
- ``kube-apiserver``'s etcd client certificate
**etcd Root CA certificate**
This is the etcd Root |CA| certificate. It signs etcd server and client
certificates, and ``kube-apiserver`` etcd client certificate. This is also the
|CA| certificate used to verify various server and client certificates signed
by etcd Root |CA| certificate.
Etcd Root |CA| certificate and corresponding private keys are stored in file
systems:
- ``/etc/etcd/ca.crt``
- ``/etc/etcd/ca.key``
**etcd server certificate**
This is the etcd server’s serving certificate. Services such as
``kube-apiserver`` that access etcd verify this serving certificate with etcd
Root |CA| certificate.
Etcd server certificate and corresponding private keys are stored in file
systems:
- ``/etc/etcd/etcd-server.crt``
- ``/etc/etcd/etcd-server.key``
**etcd client certificate**
This is a client certificate generated from etcd Root |CA|. It can be used by
clients to identify itself when connecting to etcd by HTTPS.
Etcd client certificate and corresponding private key are stored in file system:
- ``/etc/etcd/etcd-client.crt``
- ``/etc/etcd/etcd-client.key``
**kube-apiserver's etcd client certificate**
This is the ``kube-apiserver`` client certificate generated from etcd Root |CA|. It
is used by ``kube-apiserver`` to identify itself when connecting to etcd by HTTPS.
Kube-apiserver’s etcd client certificate and corresponding private keys are
stored in file systems:
- ``/etc/kubernetes/pki/apiserver-etcd-client.crt``
- ``/etc/kubernetes/pki/apiserver-etcd-client.key``
---------------------------------------
Install custom etcd Root CA certificate
---------------------------------------
Etcd Root |CA| certificate and corresponding private key are generated during
system bootstrap. By default it has 10 years validation. Installing custom etcd
Root |CA| certificate is not currently supported.
The other etcd certificates are generated from the Root |CA| certificate during
system bootstrap once the Root |CA| certificate is generated and available.
These certificates have 10 years validation too.
------------------------------
Update/Renew etcd certificates
------------------------------
Updating etcd Root |CA| certificate is a complex process, because it is not
only the Root |CA| certificate need to be updated, but also all the other etcd
certificates signed by it need to be regenerated and updated. The update of the
etcd Root |CA| is currently not supported.
The other child certificates generated from the etcd Root |CA| are monitored by
a cronjob, which runs every day at midnight to check if any of these
certificates’ expiry date is approaching and renew them if the expiry date is
within 15 days.
If the renewal fails, a **250.003** alarm will be raised:
- `Kubernetes certificates have been renewed but not all services have been
updated.`
For this alarm, controller nodes need to lock/unlock for the services to
take the new certificates.
- `Kubernetes certificates renewal failed.`
For this alarm, the Kubernetes certificates need to be renewed manually,
during which services need to restart.
If this alarm is raised, the administrator should follow the recommended action
for the specific alarm.

172
doc/source/security/kubernetes/https-access-overview.rst
View File

@ -2,98 +2,102 @@
.. ddq1552672412979
.. _https-access-overview:
=====================
HTTPS Access Overview
=====================
==========================================
HTTPS and Certificates Management Overview
==========================================
You can enable secure HTTPS access and manage HTTPS certificates for all
external |prod| service endpoints.
Certificates are heavily used for secure HTTPS access and authentication on
|prod| platform. This table lists the major certificates being used in the
system, and indicates which certificates are automatically created/renewed by
the system versus which certificates must be manually created/renewed by the
system administrator. Details on manual management of certificates can be found
in the following sections.
These include:
.. table::
:widths: auto
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Certificate | Auto Created | Renewal Status |
+===========================================================+=============================================================================+====================================================================================================+
| Kubernetes Root CA Certificate | Yes | NOT AUTO-RENEWED; Default expiry is set at 10 years |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Cluster Admin client certificate used by kubectl | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| kube-controller-manager client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| kube-scheduler client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| kube-apiserver server certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| kube-apiserver's kubelet client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| kubelet client certificate | Yes | auto-renewed by kubelet feature enabled by default |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| etcd Root CA certificate | Yes | NOT AUTO-RENEWED; Default expiry is set at 10 years |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| etcd server certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| etcd client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| kube-apiserver's etcd client certificate | Yes | auto-renewed by cron job |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| StarlingX REST API & HORIZON Server Certificate | Yes (But the auto-created certificate is self-signed and should be changed) | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Local Registry Server Certificate | Yes (But the auto-created certificate is self-signed and should be changed) | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| OIDC Client and Dex Server Server Certificate | No | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| OIDC Client and Dex Server CA certificate | No | NOT AUTO-RENEWED. CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| OIDC Remote WAD CA Certificate | No | NOT AUTO-RENEWED. CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Vault Server Certificate | Yes | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Vault Root CA certificate | Yes | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Portieris Server Certificate | Yes | Auto renewed by cert-manager; BUT COSTOMER MUST restart Portieris after the certificate is renewed |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Portieris remote registry and notary server CA Certificate| No | NOT AUTO-RENEWED; CUSTOMER MUST renew via CLIs |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Root CA DC Admin Endpoint CA Certificate | Yes | auto-renewed |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| Intermediate CA DC Admin Endpoint CA Certificate | Yes | auto-renewed |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| DC Admin Endpoint Server Certificate | Yes | auto-renewed |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
| System trusted CA Certificates | No | NOT AUTO-RENEWED as these are certificates that are not necessarily owned by Cloud Platform |
+-----------------------------------------------------------+-----------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------+
.. _https-access-overview-ul-eyn-5ln-gjb:
Where:
.. contents::
:local:
:depth: 1
- Auto created: the certificate is generated during system deployment or
triggered by certain operations.
.. _https-access-overview-section-N10048-N10024-N10001:
- Renewal Status: whether the certificate is renewed automatically by the system
when expiry date approaches.
-------------------------------------------------------
REST API Applications and the web administration server
-------------------------------------------------------
The following sections provide details on managing these certificates.
By default, |prod| provides HTTP access to REST API application endpoints
\(Keystone, Barbican and |prod|\) and the web administration server. For
improved security, you can enable HTTPS access. When HTTPS access is
enabled, HTTP access is disabled.
- :ref:`StarlingX REST API Applications and the Web Administration Server Certificate <starlingx-rest-api-applications-and-the-web-administration-server>`
When HTTPS is enabled for the first time on a |prod| system, a self-signed
certificate and key are automatically generated and installed for
REST and Web Server endpoints. In order to connect, remote clients must be
configured to accept the self-signed certificate without verifying it. This
is called insecure mode.
- :ref:`Kubernetes Certificates <kubernetes-certificates-f4196d7cae9c>`
For secure-mode connections, an intermediate or Root |CA|-signed certificate
and key are required. The use of an intermediate or Root |CA|-signed
certificate is strongly recommended. Refer to the documentation for the
external intermediate or Root |CA| that you are using, on how to create public
certificate and private key pairs, signed by an intermediate or Root |CA|, for
HTTPS.
You can update the certificate and key used by |prod| for the
REST and Web Server endpoints at any time after installation.
For additional security, |prod| optionally supports storing the private key
of the StarlingX REST and Web Server certificate in a |prod| |TPM| hardware
device. |TPM| 2.0-compliant hardware must be available on the controller
hosts.
.. _https-access-overview-section-N1004F-N10024-N10001:
----------
Kubernetes
----------
For the Kubernetes API Server, HTTPS is always enabled. You must use a Root CA
certificate; intermediate CA certificates are not supported by upstream
Kubernetes. By default, a self-signed certificate and key is generated and
installed for the Kubernetes Root |CA| certificate and key. This Kubernetes
Root |CA| is used to create and sign various certificates used within
Kubernetes, including the certificate used by the kube-apiserver API endpoint.
It is recommended that you update the Kubernetes Root |CA| and with a custom
Root |CA| certificate and key, generated by yourself, and trusted by your
external servers connecting to |prod|'s Kubernetes API endpoint. The |prod|'s
Kubernetes Root |CA| is configured as part of the bootstrap during
installation.
.. _https-access-overview-section-N10094-N10024-N10001:
---------------------
Local Docker Registry
---------------------
For the Local Docker Registry, HTTPS is always enabled. Similarly, by default,
a self-signed certificate and key is generated and installed for this endpoint.
However, it is recommended that you update the certificate used after
installation with an intermediate or Root |CA|-signed certificate and key.
Refer to the documentation for the external intermediate or Root |CA| that you
are using, on how to create public certificate and private key pairs, signed by
a Root |CA|, for HTTPS.
.. _https-access-overview-section-N10086-N10024-N10001:
-----------
Trusted CAs
-----------
|prod| also supports the ability to update the trusted |CA| certificate
bundle on all nodes in the system. This is required, for example, when
container images are being pulled from an external docker registry with a
certificate signed by a non-well-known |CA|.
- :ref:`Local Registry Server Certificates <security-install-update-the-docker-registry-certificate>`
- :ref:`System Trusted CA Certificates <add-a-trusted-ca>`

20
doc/source/security/kubernetes/index.rst
View File

@ -49,6 +49,8 @@ Manage Non-Admin Type Users
assign-pod-security-policies
resource-management
.. _user-authentication-using-windows-active-directory-security-index:
**************************************************
User Authentication Using Windows Active Directory
**************************************************
@ -91,9 +93,9 @@ Firewall Options
security-default-firewall-rules
security-firewall-options
*************************
Secure HTTPS Connectivity
*************************
****************************
HTTPS Certificate Management
****************************
.. toctree::
:maxdepth: 2
@ -101,9 +103,15 @@ Secure HTTPS Connectivity
https-access-overview
utility-script-to-display-certificates
starlingx-rest-api-applications-and-the-web-administration-server
kubernetes-root-ca-certificate
kubernetes-certificates-f4196d7cae9c
etcd-certificates-c1fc943e4a9c
security-install-update-the-docker-registry-certificate
oidc-client-dex-server-certificates-dc174462d51a
portieris-server-certificate-a0c7054844bd
vault-server-certificate-8573125eeea6
dc-admin-endpoint-certificates-8fe7adf3f932
add-a-trusted-ca
one-single-root-ca-multiple-server-client-certificates-0692df6ce16d
************
Cert Manager
@ -116,6 +124,8 @@ Cert Manager
the-cert-manager-bootstrap-process
cert-manager-post-installation-setup
.. _portieris-admission-controller-security-index:
******************************
Portieris Admission Controller
******************************
@ -128,6 +138,8 @@ Portieris Admission Controller
portieris-clusterimagepolicy-and-imagepolicy-configuration
remove-portieris
.. _vault-secret-and-data-management-security-index:
********************************
Vault Secret and Data Management
********************************

35
doc/source/security/kubernetes/install-update-the-starlingx-rest-and-web-server-certificate.rst
View File

@ -6,30 +6,32 @@
Install/Update the StarlingX Rest and Web Server Certificate
============================================================
Use the following procedure to install or update the certificate for the REST
API application endpoints \(Keystone, Barbican and StarlingX\) and the web
administration server.
Use the following procedure to install or update the certificate for the |prod|
REST API application endpoints \(Keystone, Barbican and |prod|\) and the
|prod| web administration server.
.. rubric:: |prereq|
Obtain an intermediate or Root |CA|-signed certificate and key from a trusted
intermediate or Root |CA|. Refer to the documentation for the external
Obtain an intermediate or Root |CA|-signed server certificate and key from a
trusted Intermediate or Root |CA|. Refer to the documentation for the external
Intermediate or Root |CA| that you are using, on how to create public
certificate and private key pairs, signed by intermediate or a Root |CA|, for
HTTPS.
For lab purposes, see :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` for how to create a test
intermediate or Root |CA| certificate and key, and use it to sign test
certificates.
Intermediate or Root |CA| certificate and key, and use it to sign test
server certificates.
Put the |PEM| encoded versions of the certificate and key in a single file,
and copy the file to the controller host.
Put the |PEM| encoded versions of the server certificate and key in a single
file, and copy the file to the controller host.
.. note::
If you plan to use the container-based remote CLIs, due to a limitation
in the Python2 SSL certificate validation, the certificate used for the
'ssl' certificate must either have:
If you plan to use the container-based remote CLIs, due to a limitation in
the Python2 SSL certificate validation, the certificate used for the |prod|
REST API application endpoints and |prod| Web Administration Server ('ssl')
certificate must either have:
#. CN=IPADDRESS and SANs=IPADDRESS
@ -37,7 +39,7 @@ and copy the file to the controller host.
#. CN=FQDN and SANs=FQDN
where IPADDRESS and FQDN are for the OAM Floating IP Address.
where IPADDRESS and FQDN are for the OAM Floating IP Address.
.. rubric:: |proc|
@ -55,6 +57,11 @@ and copy the file to the controller host.
**<pathTocertificateAndKey>**
is the path to the file containing both the intermediate or Root
|CA|-signed certificate and private key to install.
|CA|-signed server certificate and private key to install.
.. warning::
The REST and Web Server certificate are not automatically renewed, user
MUST renew the certificate prior to expiry, otherwise a variety of system
operations will fail.

130
doc/source/security/kubernetes/kubernetes-certificates-f4196d7cae9c.rst Normal file
View File

@ -0,0 +1,130 @@
.. _kubernetes-certificates-f4196d7cae9c:
=======================
Kubernetes Certificates
=======================
For Kubernetes, HTTPS is always enabled for both internal and external
endpoints.
Kubernetes automatically creates all of its client and server certificates, and
signs them with a Kubernetes Root |CA|. This includes the server certificate
for the external ``kube-apiserver`` API endpoint. By default, the Kubernetes
Root |CA| is automatically generated at install time.
If desired, you can externally generate a Root |CA| certificate and key, and
configure it as the Kubernetes Root |CA| during installation. Upstream
Kubernetes (v1.18) only supports a Root |CA| for the Kubernetes Root |CA|; NOT
an Intermediate |CA|.
The public certificate of the Kubernetes Root |CA|, whether auto-generated or
specified, needs to be configured as a trusted |CA| by external servers
connecting to |prod|'s Kubernetes API endpoint (e.g. via a remotely installed
``kubectl`` client).
.. note::
Some platform services (sysinv, cert-mon and VIM for example) also use X509
certificates to access Kubernetes by HTTPS.
It is optional that you update the Kubernetes Root |CA| with a custom Root CA
certificate and key, generated by yourself, and trusted by your external
servers connecting to |prod|’s Kubernetes API endpoint. The |prod|’s Kubernetes
Root |CA| certificate and key are configured as part of the bootstrap during
installation.
.. note::
You must use a Root |CA| certificate; Intermediate |CA| certificates
are not supported by upstream Kubernetes.
Kubernetes certificates include:
- Kubernetes Root |CA| Certificate
- Cluster admin client certificate used by ``kubectl``
- ``kube-controller-manager`` client certificate
- ``kube-scheduler`` client certificate
- ``kube-apiserver`` server certificate
- ``kube-apiserver``'s kubelet client certificate
- ``kubelet`` client certificate
**Kubernetes Root CA Certificate**
The Kubernetes Root |CA| certificate signs all the other Kubernetes
certificates. This is also the |CA| certificate various components use to
verify server and client certificates signed by the Kubernetes Root |CA|
certificate. For example, applications running in pods use Kubernetes Root |CA|
certificate embedded in service account token to verify the ``kube-apiserver``'s
server certificate when it makes calls to the kube-apiserver.
Kubernetes Root |CA| certificate and corresponding private key are stored in
file system:
- ``/etc/kubernetes/pki/ca.crt``
- ``/etc/kubernetes/pki/ca.key``
.. note::
Kubernetes Root |CA| certificate is also embedded in various
configuration files and service account token.
**Cluster admin client certificate used by kubectl**
This is the client certificate signed by Kubernetes Root |CA| and embedded in
``/etc/kubernetes/admin.conf``. It is used by kubectl command to identify
itself to the ``kube-apiserver``.
**kube-controller-manager client certificate**
This is the client certificate signed by Kubernetes Root |CA| and embedded in
``/etc/kubernetes/controller-manager.conf``. It is used by
``kube-controller-manager`` pod to identify itself to ``kube-apiserver``.
**kube-scheduler client certificate**
This is the client certificate signed by Kubernetes Root |CA| and embedded in
``/etc/kubernetes/scheduler.conf``. It is used by ``kube-scheduler`` pod to
identify itself to the ``kube-apiserver``.
**kube-apiserver server certificate**
This is the kube-apiserver's serving certificate. Clients connecting to the
``kube-apiserver`` will verify this certificate using Kubernetes Root |CA|
certificate. The certificate and the corresponding private key are stored in
file system:
- ``/etc/kubernetes/pki/apiserver.crt``
- ``/etc/kubernetes/pki/apiserver.key``
**kube-apiserver's kubelet client certificate**
``kube-apiserver``'s client certificate for communications with ``kubelet``.
``kube-apiserver`` identifies itself with this certificate when it connects to
``kubelet``. The certificate and the corresponding private keys are stored in
file system:
- ``/etc/kubernetes/pki/apiserver-kubelet-client.crt``
- ``/etc/kubernetes/pki/apiserver-kubelet-client.key``
**kubelet client certificate**
This is the ``kubelet``’s client certificate (with private key in it).
``kubelet`` identifies itself with this certificate when it connects to
``kube-apiserver``. ``kubelet`` has Kubernetes Root |CA| certificate in
``/etc/kubernetes/kubelet.conf`` to verify peer certificates.
The certificate and its corresponding private key are store in file system as
one file:
- ``/var/lib/kubelet/pki/kubelet-client-current.pem``
This certificate is configured to auto renew.
.. toctree::
:maxdepth: 1
kubernetes-root-ca-certificate
update-renew-kubernetes-certificates-52b00bd0bdae

39
doc/source/security/kubernetes/kubernetes-root-ca-certificate.rst
View File

@ -2,26 +2,30 @@
.. imj1570020645091
.. _kubernetes-root-ca-certificate:
==============================
Kubernetes Root CA Certificate
==============================
=============================================
Install Custom Kubernetes Root CA Certificate
=============================================
By default, the K8S Root |CA| Certificate and Key are auto-generated and
result in the use of certificates signed by an unknown |CA| for Kubernetes;
for example, for the Kubernetes API server.
By default, the K8S Root |CA| certificate and key are auto-generated and result
in the other Kubernetes certificates being signed by an internal not well-known
|CA|; for example, for the Kubernetes API server certificate.
It is recommended that you update the Kubernetes Root |CA| and with a custom
Root |CA| certificate and key, generated by yourself, and trusted by external
servers connecting to the |prod|'s Kubernetes API endpoint.
It is optional that you update the Kubernetes Root |CA| with a custom Root
|CA| certificate and key, generated by yourself, and trusted by external servers
connecting to the |prod|’s Kubernetes API endpoint
The installation of the custom Kubernetes Root |CA| certificate can only be
done during system deployment by using bootstrap overrides.
See :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` for how to create a private Root
|CA| certificate and key.
.. caution::
The default duration for the generated Kubernetes Root CA certificate is 10
years. Replacing the Root |CA| certificate is a complex process, so the custom
certificate expiry should be set for a long period, if possible. Wind River
certificate expiry should be set for a long period, if possible. |org|
recommends setting the Root |CA| certificate with an expiry of at least 5-10
years.
@ -32,18 +36,18 @@ See :ref:`Create Certificates Locally using openssl
Use the bootstrap override values <k8s\_root\_ca\_cert> and
<k8s\_root\_ca\_key>, as part of the installation procedure to specify the
certificate and key for the Kubernetes root |CA|.
certificate and key for the Kubernetes Root |CA|.
**<k8s\_root\_ca\_cert>**
Specifies the certificate for the Kubernetes root |CA|. The
Specifies the certificate for the Kubernetes Root |CA|. The
<k8s\_root\_ca\_cert> value is the absolute path of the certificate
file. The certificate must be in |PEM| format and the value must be
provided as part of a pair with <k8s\_root\_ca\_key>.
**<k8s\_root\_ca\_key>**
Specifies the key for the Kubernetes root |CA|. The <k8s\_root\_ca\_key>
Specifies the key for the Kubernetes Root |CA|. The <k8s\_root\_ca\_key>
value is the absolute path of the certificate file. The certificate
must be in |PEM| format and the value must be provided as part of a pair
with <k8s\_root\_ca\_cert>.
@ -58,6 +62,7 @@ For example:
The playbook will not proceed if only one value is provided.
.. caution::
The default duration for the generated Kubernetes Root |CA|
certificate is 10 years. Replacing the Root |CA| certificate is an
involved process so the custom certificate expiry should be as long
@ -81,7 +86,7 @@ be an IP address or domain name. For example:
- 198.51.100.75
|prod| automatically updates this parameter to include IP records
for the |OAM| floating IP and both |OAM| unit IP addresses. Any DNS names
for the |OAM| floating IP and both |OAM| unit IP addresses. Any |DNS| names
associated with the |OAM| floating IP address should be added.
@ -90,9 +95,9 @@ associated with the |OAM| floating IP address should be added.
.. rubric:: |postreq|
Make the K8S Root |CA| certificate available to any remote server wanting to
connect remotely to the |prod|'s Kubernetes API, e.g. through kubectl or helm.
This Kubernetes Root CA certificate should be configured as a trusted |CA| on
the remote server.
connect remotely to the |prod|'s Kubernetes API, e.g. through ``kubectl`` or
Helm. This Kubernetes Root CA certificate should be configured as a trusted
|CA| on the remote server.
See the step :ref:`2.b
<security-install-kubectl-and-helm-clients-directly-on-a-host>` in

109
doc/source/security/kubernetes/oidc-client-dex-server-certificates-dc174462d51a.rst Normal file
View File

@ -0,0 +1,109 @@
.. _oidc-client-dex-server-certificates-dc174462d51a:
===================================
OIDC Client Dex Server Certificates
===================================
The oidc-auth-apps application installs a proxy |OIDC| identity provider (dex
server) that can be configured to proxy authentication requests to an |LDAP|
(s) identity provider, such as Windows Active Directory.
The oidc-auth-apps application also provides an |OIDC| client for accessing the
username and password |OIDC| login page for user authentication and retrieval
of tokens.
.. note::
For details on how installing, configuring, and using oidc-auth-apps,
refer to :ref:`User Authentication Using Windows Active Directory
<user-authentication-using-windows-active-directory-security-index>`.
This section is specifically about |OIDC| certificates management.
Oidc-auth-apps needs three certificates to work:
- |OIDC| client and identity provider server certificate (secret
``local-dex.tls``)
- |OIDC| trusted |CA| certificate (secret ``dex-client-secret``)
- Windows Active Directory |CA| certificate (secret wadcert)
**OIDC client and identity provider server certificate**
|OIDC| client and Identity provider server certificate is used to secure the
connection between |OIDC| client and identity provider by HTTPS.
This certificate is stored in Kubernetes TLS secret ``local-dex.tls``.
**OIDC client and identity provider trusted CA certificate**
The |OIDC| trusted |CA| certificate is the |CA| certificate that signs the
|OIDC| client and identity server certificate.
It has to be installed for |OIDC| client to verify identity server’s
certificate for HTTPS connection.
|OIDC| trusted |CA| certificate is stored in Kubernetes secret
``dex-client-secret``.
**Windows Active Directory CA certificate (WAD CA certificate)**
|WAD| certificate is the |CA| certificate that signed the Windows Active
Directory that |OIDC| is configured to proxy authentication requests to.
In order for |OIDC| identity provider (as the authentication proxy) to securely
connect and authenticate users to the Windows Active Directory by HTTPS, the
|WAD|’s |CA| certificate needs to installed and configured for |OIDC| to trust
the Windows Active Directory.
-------------------------
Install OIDC certificates
-------------------------
|OIDC| certificates are not auto generated.
They need to be installed as Kubernetes secrets as part of the |OIDC| app
configuration.
Refer to :ref:`Configure OIDC Auth Applications
<configure-oidc-auth-applications>`, on how to install |OIDC| certificates into
Kubernetes secrets.
------------------------------
Update/Renew OIDC certificates
------------------------------
.. warning::
|OIDC| certificates are not auto renewed. They have to be updated manually
by updating the secrets from the new certificate files and restart the
``oidc-auth`` application.
.. rubric:: |proc|
#. Update/renew |OIDC| client and identity provider server certificate:
.. code-block:: none
~(keystone_admin)]$ kubectl create secret tls local-dex.tls --cert=/home/sysadmin/new_ssl/dex-cert.pem --key=/home/sysadmin/new_ssl/dex-key.pem --save-config --dry-run=client -n kube-system -o yaml | kubectl apply -f -
#. Update/renew |OIDC| trusted |CA| certificate:
.. code-block:: none
~(keystone_admin)]$ kubectl create secret generic dex-client-secret --from-file=/home/sysadmin/new_ssl/dex-ca.pem --save-config --dry-run=client -n kube-system -o yaml | kubectl apply -f -
#. Update/renew |WAD| |CA| certificate:
.. code-block:: none
~(keystone_admin)]$ kubectl create secret generic wadcert --from-file=/home/sysadmin/new_ssl/AD_CA.cer –save-config –dry-run=client -n kube-system -o yaml | kubectl apply -f -
#. Restart |OIDC| client and identity provider proxy (dex-server):
.. code-block:: none
~(keystone_admin)]$ kubectl rollout restart deployment oidc-dex -n kube-system
~(keystone_admin)]$ kubectl rollout restart deployment stx-oidc-client -n kube-system

111
doc/source/security/kubernetes/one-single-root-ca-multiple-server-client-certificates-0692df6ce16d.rst Normal file
View File

@ -0,0 +1,111 @@
.. _one-single-root-ca-multiple-server-client-certificates-0692df6ce16d:
=================================
Certificate Management Guidelines
=================================
A recommended guideline is to use one single Root |CA| certificate to generate
multiple server/client certificates for different uses in the system.
This simplifies the overall configuration of your certificate chains, as well
as it means you need only provide a single Root |CA| certificate for clients to
trust when interfacing to the system.
.. rubric:: |proc|
The following is a use case for |DC| system where one single Root |CA| is used
to generate REST API/Horizon server certificates, central/subcloud registry
server certificates, and how to install these certificates and update system’s
trusted |CA| list.
#. Generate a Root |CA| certificate on System Controller or a Linux server
with openssl installed.
Refer to :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` on how to generate a Root |CA|
certificate, and save the Root |CA| certificate and corresponding private
key in a directory, for example:
.. code-block:: none
../root_CA/root-ca-cert.pem
../root_CA/root-ca-key.pem
#. Generate REST API/Horizon server certificates for System Controller and
subclouds.
Refer to :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` on how to generate server
certificates from the Root |CA| certificate.
Pay attention to the notes about the certificate’s |SAN| on section
:ref:`Install/Update the StarlingX Rest and Web Server Certificate
<install-update-the-starlingx-rest-and-web-server-certificate>`.
Optionally, set the subject fields uniquely for systemController and each of
the subclouds.
Generate REST API/Horizon server certificate for the central cloud and each
of the subclouds, and save them in a directory, for example:
.. code-block:: none
.. /REST_certificates/central-rest-server-cert.pem
.. /REST_certificates/subcloud1-rest-server-cert.pem
.. /REST_certificates/subcloud2-rest-server-cert.pem
...
#. Generate registry server certificates for central cloud and subclouds.
Refer to :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` on how to generate server
certificates from the self-signed Root |CA| certificate.
Refer to :ref:`Install/Update the Local Docker Registry Certificate
<installing-updating-the-docker-registry-certificate>` for the requirements
on certificate’s |SANs|.
Optionally set the subject fields uniquely for System Controller and each
of the subclouds.
Generate registry server certificate for central cloud and each of the
subclouds, and save them is a directory, for example:
.. code-block:: none
.. /registry_certificates/central-registry-server-cert.pem
.. /registry_certificates/subcloud1-registry-server-cert.pem
.. /registry_certificates/subcloud2-registry-server-cert.pem
...
#. Install the Root |CA| certificate as trusted |CA| on System Controller.
The single Root |CA| certificate only need to be installed on System
Controller.
It will sync to all the subclouds.
Wait until subclouds are insync.
#. Install the REST API/Horizon server certificates to the central and subclouds.
Once all subclouds are insync, install the central cloud’s REST
API/Horizon server certificate to the central cloud, and the subcloud’s
REST API/Horizon server certificate to each of the subclouds.
This can be done manually or by some auto tools such as ansible.
#. Install the registry server certificates to central and subclouds.
Similarly, once all subclouds are in-sync, install the central cloud’s
registry certificate to the central cloud, and the subcloud’s registry
server certificate to each of the subclouds.
This can be done manually or by some auto tools such as ansible.
#. Provide the single Root |CA| public certificate, from step 1
(`../root_CA/root-ca-cert.pem`), to any remote user using remote clients to
interface with the |prod| system.
These remote users/clients will need to be configured to trust this Root
|CA|.

73
doc/source/security/kubernetes/portieris-server-certificate-a0c7054844bd.rst Normal file
View File

@ -0,0 +1,73 @@
.. _portieris-server-certificate-a0c7054844bd:
============================
Portieris Server Certificate
============================
Portieris allows you to configure trust policies for an individual namespace or
cluster-wide, and checks the image against a signed image list on a specified
notary server to enforce the configured image policies.
Refer to :ref:`Portieris Admission Controller
<portieris-admission-controller-security-index>` for details about Portieris
installation and configuration.
The |prod| implementation of Portieris is integrated with cert-manager.
Once Portieris application is applied, the server certificate is created in
cert-manager and stored in a secret in the Portieris namespace.
The server certificate has default 3 month validity.
- Certificate in cert-manager: portieris-certs
- Certificate secret: portieris-certs
This server certificate is used by Portieris webhook for secure communication
with ``kube-apiserver``.
In order for Portieris on the |prod| to securely access registries or notary
servers with certificates signed by a custom |CA| certificate, the caCert:
CERTIFICATE override must be added to the portieris-certs Helm chart so that
Portieris trusts the custom |CA| certificate.
This must be passed as the base-64 encoded (b64enc) format of the |CA|
certificate and may contain one or more |CA| certificates.
------------------------------
Install Portieris certificates
------------------------------
The Portieris server certificate is automatically created and managed by
cert-manager once Portieris application is applied.
One or more |CA| certificates can be installed for Portieris to trust
registries and notary servers.
Refer to :ref:`Install Portieris <install-portieris>` for |CA|
certificates installation.
-----------------------------------
Update/Renew Portieris certificates
-----------------------------------
Portieris server certificate is managed by cert-manager.
It will be automatically renewed when the certificate is within one month of
expiration.
.. note::
Currently notification of the renewal is not supported.
It is recommended to re-configure the automatically configured Portieris
Certificate to have a long duration since certificate renewal is not fully
supported for Portieris.
|CA| certificates can be updated the same way as installation.
Once |CA| certificates are updated, you must restart Portieris using the command:
.. code-block::
~(keystone_admin)]$ kubectl rollout restart deployment portieris-portieris -n portieris

2
doc/source/security/kubernetes/remote-windows-active-directory-accounts.rst
View File

@ -10,6 +10,6 @@ Remote Windows Active Directory Accounts
Accounts and native Kubernetes |RBAC| policies for authentication and
authorization of users of the Kubernetes API, |CLI|, and Dashboard.
See :ref:`User Authentication Using Windows Active Directory
See :ref:`Overview of Windows Active Directory
<overview-of-windows-active-directory>` for more details.

138
doc/source/security/kubernetes/secure-starlingx-rest-and-web-certificates-private-key-storage-with-tpm.rst
View File

@ -6,15 +6,15 @@
Secure StarlingX REST and Web Certificate's Private Key Storage with TPM
========================================================================
For increased security, the StarlingX REST and Web Server's certificate can
For increased security, the |prod| REST and Web Server's certificate can
be installed such that the private key is stored in a |TPM| 2.0 device on
the controller.
.. rubric:: |context|
|TPM| is an industry standard cryptographic processor that enables secure
storage of secrets. |prod| can use a TPM device, if present, to securely
store the private key of the StarlingX REST and Web Server's certificate.
storage of secrets. |prod| can use a |TPM| device, if present, to securely
store the private key of the |prod| REST and Web Server's certificate.
The |TPM| is used to wrap the private key within the |TPM| device. Each
wrapping is unique to that |TPM| device and cannot be synchronized between
@ -23,7 +23,7 @@ is always secured to both the active and standby controllers' |TPM| devices
at the same time. Given this operational constraint, |prod| has measures in
place to detect when the standby controller is reinstalled or replaced, and
raise appropriate alarms to prevent an Unlock or Swact of a new standby
controller until the StarlingX REST and Web Server's certificate is
controller until the |prod| REST and Web Server's certificate is
re-installed, in order to update the new standby controller's |TPM| device.
.. rubric:: |prereq|
@ -31,15 +31,15 @@ re-installed, in order to update the new standby controller's |TPM| device.
.. _secure-starlingx-rest-and-web-certificates-private-key-storage-with-tpm-ul-xj3-mqc-d1b:
- Obtain an intermediate or Root |CA|-signed certificate and key from a
trusted intermediate or Root |CA|. Refer to the documentation for the
external intermediate or Root |CA| that you are using, on how to create
public certificate and private key pairs, signed by an intermediate or
- Obtain an Intermediate or Root |CA|-signed certificate and key from a
trusted Intermediate or Root |CA|. Refer to the documentation for the
external Intermediate or Root |CA| that you are using, on how to create
public certificate and private key pairs, signed by an Intermediate or
Root-signed |CA|, for HTTPS.
For lab purposes, see :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` for details on how to create
a test intermediate or Root |CA| certificate and key, and use it to sign
a test Intermediate or Root |CA| certificate and key, and use it to sign
test certificates.
Put the |PEM| encoded versions of the certificate and key in a
@ -54,8 +54,23 @@ re-installed, in order to update the new standby controller's |TPM| device.
- HTTPS must be enabled on the system.
.. note::
If you plan to use the container-based remote CLIs, due to a limitation in
the Python2 SSL certificate validation, the certificate used for the |prod|
REST API application endpoints and |prod| Web Administration Server ('ssl')
certificate must either have:
#. CN=IPADDRESS and SANs=IPADDRESS
or
#. CN=FQDN and SANs=FQDN
where IPADDRESS and |FQDN| are for the |OAM| Floating IP Address.
.. caution::
Do not install the certificate using |TPM| on controller-0 before the
standby controller-1 has been provisioned and unlocked. If this happens,
you cannot unlock controller-1. To recover, do the following:
@ -76,12 +91,12 @@ re-installed, in order to update the new standby controller's |TPM| device.
.. _secure-starlingx-rest-and-web-certificates-private-key-storage-with-tpm-steps-hnx-qf5-x1b:
#. Install the StarlingX REST and Web Server's certificate using |TPM| to
#. Install the |prod| REST and Web Server's certificate using |TPM| to
securely store the private key:
.. code-block:: none
$ system certificate-install –m tpm_mode <pathTocertificateAndKey>
~(keystone_admin)]$ system certificate-install –m tpm_mode <pathTocertificateAndKey>
where:
@ -91,11 +106,13 @@ re-installed, in order to update the new standby controller's |TPM| device.
|CA|-signed certificate and private key to install.
.. warning::
For security purposes, the utility deletes the provided SSL private
key from the file system and asks for confirmation during the
installation. You should store a copy of the SSL private key off-site.
.. note::
Only X.509 based RSA key certificates are supported \(PKCS12 format
and ECDSA keys are not supported\). Additionally, 4096 bit RSA key
lengths are not supported.
@ -161,7 +178,7 @@ scenario, you must reinstall the certificate:
~(keystone_admin)]$ system certificate-install -m tpm_mode
<pathTocertificateAndKey>
To disable the use of |TPM| to store the private key of the StarlingX REST
To disable the use of |TPM| to store the private key of the |prod| REST
and Web Server's certificate, install the certificate without the |TPM|
option:
@ -169,3 +186,100 @@ option:
~(keystone_admin)]$ system certificate-install <pathTocertificateAndKey>
.. warning::
The REST and Web Server certificate are not automatically renewed, user
MUST renew the certificate prior to expiry, otherwise a variety of system
operations will fail.
.. _tpm-configuration-considerations:
--------------------------------
TPM configuration considerations
--------------------------------
There are some considerations to account for when configuring or
reconfiguring |TPM|.
This includes certain behavior and warnings that you may encounter when
configuring |TPM|. The same behavior and warnings are seen when performing
these actions in the Horizon Web interface, also.
.. _tpm-configuration-considerations-ul-fbm-1fy-f1b:
- The :command:`certificate-show tpm` command will indicate the status of
the |TPM| configuration on the hosts, either **tpm-config-failed** or
**tpm-config-applied**.
.. code-block:: none
~(keystone_admin)]$ system certificate-show tpm
+-------------+-----------------------------------------------------+
| Property | Value |
+-------------+-----------------------------------------------------+
| uuid | ed3d6a22-996d-421b-b4a5-64ab42ebe8be |
| certtype | tpm_mode |
| signature | tpm_mode_13214262027721489760 |
| start_date | 2018-03-21T14:53:03+00:00 |
| expiry_date | 2019-03-21T14:53:03+00:00 |
| details | {u'state': {u'controller-1': u'tpm-config-applied', |
| | u'controller-0': u'tpm-config-applied'}} |
+-------------+-----------------------------------------------------+
- If either controller has state **tpm-config-failed**, then a **500.100**
alarm will be raised for the host.
.. code-block:: none
~(keystone_admin)]$ fm alarm-list
+----------+------------------+------------------+----------+------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+------------------+------------------+----------+------------+
| 500.100 | TPM configuration| host=controller-1| major | 2017-06-1..|
| | failed or device.| | |.586010 |
+----------+------------------+------------------+----------+------------+
- An UNLOCKED controller node that is not in TPM applied configuration
state \(**tpm-config-applied**\) will be prevented from being Swacted To or
upgraded.
The following warning is generated when you attempt to swact:
.. code-block:: none
~(keystone_admin)]$ system host-swact controller-0
TPM configuration not fully applied on host controller-1; Please
run https-certificate-install before re-attempting.
- A LOCKED controller node that is not in |TPM| applied configuration state
\(**tpm-config-applied**\) will be prevented from being UNLOCKED.
The :command:`host-list` command below shows controller-1 as locked and
disabled.
.. code-block:: none
~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1 | controller-0 | controller | unlocked | enabled | available |
| 2 | controller-1 | controller | locked | disabled | online |
+----+--------------+-------------+----------------+-------------+--------------+
The following warning is generated when you attempt to UNLOCK a
controller not in a **tpm-config-applied** state:
.. code-block:: none
~[keystone_admin)]$ system host-unlock controller-1
TPM configuration not fully applied on host controller-1; Please
run https-certificate-install before re-attempting

45
doc/source/security/kubernetes/security-install-update-the-docker-registry-certificate.rst
View File

@ -2,20 +2,35 @@
.. vri1561486014514
.. _security-install-update-the-docker-registry-certificate:
=================================
Local Docker Registry Certificate
=================================
==================================
Local Registry Server Certificates
==================================
The local Docker registry provides secure HTTPS access using the registry API.
For the Local Docker Registry, HTTPS is always enabled. By default, a
self-signed server certificate and key is generated and installed for this
endpoint. However, it is strongly recommended that you update the server
certificate used after installation with an Intermediate or Root |CA|-signed
server certificate and key. Refer to the documentation for the external
Intermediate or Root |CA| that you are using, on how to create public
certificate and private key pairs, signed by a Root |CA|, for HTTPS.
The local Docker registry provides Docker image service that can be accessed
using the registry API by secure HTTPS. Standalone system, central cloud and
every subcloud of |DC| system has their own Docker registry called
`registry.local`.
The Docker registry on the central cloud of |DC| system has an
alias of `registry.central`, which is used by subcloud to remotely login or
pull images from this central Docker registry.
.. rubric:: |context|
By default a self-signed certificate is generated at installation time for the
registry API. For more secure access, an intermediate or Root |CA|-signed
registry API. For more secure access, an Intermediate or Root |CA|-signed
certificate is strongly recommended.
The intermediate or Root |CA|-signed certificate for the registry must have at
least the following |SANs|: DNS:registry.local, DNS:registry.central, IP
The Intermediate or Root |CA|-signed certificate for the registry must have at
least the following |SANs|: ``DNS:registry.local``, ``DNS:registry.central``, IP
Address:<oam-floating-ip-address>, IP Address:<mgmt-floating-ip-address>. Use
the :command:`system addrpool-list` command to get the |OAM| floating IP
Address and management floating IP Address for your system. You can add any
@ -29,19 +44,19 @@ an expired or soon to expire certificate.
.. rubric:: |prereq|
Obtain an intermediate or Root |CA|-signed certificate and key from a trusted
intermediate or Root |CA|. Refer to the documentation for the external Root
Intermediate or Root |CA|. Refer to the documentation for the external Root
|CA| that you are using, on how to create public certificate and private key
pairs, signed by an intermediate or Root |CA|, for HTTPS.
pairs, signed by an Intermediate or Root |CA|, for HTTPS.
For lab purposes, see :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` for how to create a test
intermediate or Root |CA| certificate and key, and use it to sign test
Intermediate or Root |CA| certificate and key, and use it to sign test
certificates.
Put the |PEM| encoded versions of the certificate and key in a single file,
and copy the file to the controller host.
Also, obtain the certificate of the intermediate or Root |CA| that signed the
Also, obtain the certificate of the Intermediate or Root |CA| that signed the
above certificate.
.. rubric:: |proc|
@ -62,7 +77,7 @@ above certificate.
``<pathTocertificate>``
is the path to the intermediate or Root |CA| certificate associated
with the Docker registry's intermediate or Root |CA|-signed
with the Docker registry's Intermediate or Root |CA|-signed
certificate.
#. Update the Docker registry certificate using the
@ -79,6 +94,8 @@ above certificate.
``<pathTocertificateAndKey>``
is the path to the file containing both the Docker registry's
intermediate or Root CA-signed certificate and private key to install.
Intermediate or Root CA-signed certificate and private key to install.
Refer to :ref:`Install/Update Local Registry Certificates
<installing-updating-the-docker-registry-certificate>` on how to install/update
and renew local registry certificates.

2
doc/source/security/kubernetes/starlingx-openstack-kubernetes-from-stsadmin-account-login.rst
View File

@ -6,7 +6,7 @@
For StarlingX, Platform OpenStack and Kubernetes CLIs from the 'sysadmin' Linux Account Login
=============================================================================================
You can establish credentials for StarlingX, Platform OpenStack and Kubernetes
You can establish credentials for |prod|, Platform OpenStack and Kubernetes
|CLIs| from the 'sysadmin' Linux account login.
.. rubric:: |context|

44
doc/source/security/kubernetes/starlingx-rest-api-applications-and-the-web-administration-server.rst
View File

@ -6,35 +6,43 @@
StarlingX REST API Applications and the Web Administration Server Certificate
=============================================================================
|prod| provides support for secure HTTPS external connections used for
StarlingX REST API application endpoints \(Keystone, Barbican and
StarlingX\) and the |prod| web administration server.
By default, |prod| provides HTTP access to REST API application endpoints
\(Keystone, Barbican and |prod|\) and the web administration server. For
improved security, you can enable HTTPS access. When HTTPS access is
enabled, HTTP access is disabled.
By default HTTPS access to StarlingX REST and Web Server's endpoints are
disabled; i.e. accessible via HTTP only.
When HTTPS is enabled for the first time on a |prod| system, a self-signed
server certificate and key are automatically generated and installed for REST
and Web Server endpoints. In order to connect, remote clients must be
configured to accept the self-signed server certificate without verifying it.
This is called insecure mode.
For secure HTTPS access, an x509 certificate and key is required. When HTTPS is
enabled for the first time on a |prod| system, a self-signed certificate and
key are automatically generated and installed for the StarlingX REST and Web
Server endpoints.
For secure-mode connections, an Intermediate or Root |CA|-signed server
certificate and key are required. The use of an Intermediate or Root
|CA|-signed server certificate is strongly recommended. Refer to the
documentation for the external Intermediate or Root |CA| that you are using, on
how to create public certificate and private key pairs, signed by an
Intermediate or Root |CA|, for HTTPS.
.. note::
Refer to the documentation for the external Intermediate or Root |CA| that
you are using, on how to create public certificate and private key pairs,
signed by an Intermediate or Root |CA|, for HTTPS.
For secure remote access, an intermediate or Root |CA|-signed certificate and
key are required. The use of a Root |CA|-signed certificate is strongly
recommended.
You can update the certificate and key used by |prod| for the
REST and Web Server endpoints at any time after installation.
You can update the certificate used for HTTPS access at any time.
For additional security, |prod| optionally supports storing the private key
of the |prod| REST and Web Server certificate in a |TPM| hardware
device. |TPM| 2.0-compliant hardware must be available on the controller
hosts.
For more details, refer to:
.. toctree::
:maxdepth: 1
:maxdepth: 1
enable-https-access-for-starlingx-rest-and-web-server-endpoints
install-update-the-starlingx-rest-and-web-server-certificate
secure-starlingx-rest-and-web-certificates-private-key-storage-with-tpm
tpm-configuration-considerations
enable-https-access-for-starlingx-rest-and-web-server-endpoints
install-update-the-starlingx-rest-and-web-server-certificate
secure-starlingx-rest-and-web-certificates-private-key-storage-with-tpm

95
doc/source/security/kubernetes/tpm-configuration-considerations.rst
View File

@ -1,95 +0,0 @@
.. qjd1552681409626
.. _tpm-configuration-considerations:
================================
TPM Configuration Considerations
================================
There are some considerations to account for when configuring or
reconfiguring |TPM|.
This includes certain behavior and warnings that you may encounter when
configuring TPM. The same behavior and warnings are seen when performing
these actions in the Horizon Web interface, also.
.. _tpm-configuration-considerations-ul-fbm-1fy-f1b:
- The command :command:`certificate-show tpm` will indicate the status of
the TPM configuration on the hosts, either **tpm-config-failed** or
**tpm-config-applied**.
.. code-block:: none
~(keystone_admin)]$ system certificate-show tpm
+-------------+-----------------------------------------------------+
| Property | Value |
+-------------+-----------------------------------------------------+
| uuid | ed3d6a22-996d-421b-b4a5-64ab42ebe8be |
| certtype | tpm_mode |
| signature | tpm_mode_13214262027721489760 |
| start_date | 2018-03-21T14:53:03+00:00 |
| expiry_date | 2019-03-21T14:53:03+00:00 |
| details | {u'state': {u'controller-1': u'tpm-config-applied', |
| | u'controller-0': u'tpm-config-applied'}} |
+-------------+-----------------------------------------------------+
- If either controller has state **tpm-config-failed**, then a 500.100
alarm will be raised for the host.
.. code-block:: none
~(keystone_admin)]$ fm alarm-list
+----------+------------------+------------------+----------+------------+
| Alarm ID | Reason Text | Entity ID | Severity | Time Stamp |
+----------+------------------+------------------+----------+------------+
| 500.100 | TPM configuration| host=controller-1| major | 2017-06-1..|
| | failed or device.| | |.586010 |
+----------+------------------+------------------+----------+------------+
- An UNLOCKED controller node that is not in TPM applied configuration
state \(**tpm-config-applied**\) will be prevented from being Swacted To or
upgraded.
The following warning is generated when you attempt to swact:
.. code-block:: none
~(keystone_admin)]$ system host-swact controller-0
TPM configuration not fully applied on host controller-1; Please
run https-certificate-install before re-attempting.
- A LOCKED controller node that is not in TPM applied configuration state
\(**tpm-config-applied**\) will be prevented from being UNLOCKED.
The :command:`host-list` command below shows controller-1 as locked and
disabled.
.. code-block:: none
~(keystone_admin)]$ system host-list
+----+--------------+-------------+----------------+-------------+--------------+
| id | hostname | personality | administrative | operational | availability |
+----+--------------+-------------+----------------+-------------+--------------+
| 1 | controller-0 | controller | unlocked | enabled | available |
| 2 | controller-1 | controller | locked | disabled | online |
+----+--------------+-------------+----------------+-------------+--------------+
The following warning is generated when you attempt to UNLOCK a
controller not in a **tpm-config-applied** state:
.. code-block:: none
~[keystone_admin)]$ system host-unlock controller-1
TPM configuration not fully applied on host controller-1; Please
run https-certificate-install before re-attempting

32
doc/source/security/kubernetes/update-renew-kubernetes-certificates-52b00bd0bdae.rst Normal file
View File

@ -0,0 +1,32 @@
.. _update-renew-kubernetes-certificates-52b00bd0bdae:
====================================
Update/Renew Kubernetes Certificates
====================================
Updating Kubernetes Root |CA| certificate is a complex process, because it is
not only the Root |CA| certificate that needs to be updated, but also all the
other Kubernetes certificates signed by it need to be regenerated and updated.
The update of the Kubernetes Root |CA| certificate is currently not supported.
The other leaf certificates generated from the Kubernetes Root |CA| are
monitored by a cronjob, which runs every day at midnight to check if any of
these certificates’ expiry date is approaching, and renew them if the expiry
date is within 15 days.
If the renewal fails, a **250.003** alarm will be raised:
- `Kubernetes certificates have been renewed but not all services have been
updated.`
For this alarm, controller nodes need to lock/unlock for the services to
take the new certificates.
- `Kubernetes certificates renewal failed.`
For this alarm, the Kubernetes certificates need to be renewed manually,
during which services need to restart.
If this alarm is raised, the administrator should follow the recommended action
for the specific alarm.

2
doc/source/security/kubernetes/utility-script-to-display-certificates.rst
View File

@ -48,7 +48,7 @@ For example:
For scalability in a Distributed cloud system, the Subcloud ICA certificates
are redirected to a file. The script displays the path to the file with a note
at the end of the output file.
at the end of the displayed output.
.. code-block:: none

134
doc/source/security/kubernetes/vault-server-certificate-8573125eeea6.rst Normal file
View File

@ -0,0 +1,134 @@
.. _vault-server-certificate-8573125eeea6:
========================
Vault Server Certificate
========================
|prod| integrates open source Vault containerized security application
\(Optional\) into the |prod| solution.
Vault is a containerized secrets management application that provides encrypted
storage with policy-based access control and supports multiple secrets storage
engines and auth methods.
Refer to :ref:`Vault Secret and Data Management
<vault-secret-and-data-management-security-index>` for details about Vault
installation and configuration.
Accessing Vault is secured by HTTPS. Vault server certificate and the Root |CA|
certificate from which the server certificate is generated are stored in
Kubernetes secrets in Vault namespace.
- Vault-ca: the Vault Root |CA| certificate
- Vault-server-tls: the Vault server certificate
The client that accesses Vault server verifies Vault server certificate with
vault-ca Root |CA| certificate. So the client needs to be configured to trust
vault-ca Root |CA| certificate.
In section :ref:`Configure Vault Using the Vault REST API <configure-vault>`,
there are examples using curl to access Vault services.
--------------------------------
Install Vault server certificate
--------------------------------
By default, the Root |CA| certificate and key are automatically created and the
Vault server certificate is generated from the Root |CA| certificate during the
Vault app application.
The Root |CA| certificate has 10 years validity while the server certificate
has 3 month validity.
-------------------------------
Update/Renew Vault certificates
-------------------------------
.. warning::
Vault certificates are not auto renewed. They have to be updated manually
by updating the secrets from the new certificate files.
Refer to :ref:`Create Certificates Locally using openssl
<create-certificates-locally-using-openssl>` on how to generate certificate
using openssl in general.
.. rubric:: |proc|
The following procedure is an example of the steps to generate new Vault server
certificate from the existing Root |CA| certificate using openssl and update
corresponding secret for Vault to use the new certificate.
The existing Root |CA| has 10 years validity so the example below is to renew
the Vault server certificate from it.
#. Retrieve Vault Root |CA| certificate and private key from secret to files:
.. code-block:: none
~(keystone_admin)]$ mkdir /home/sysadmin/vault_ca_cert
~(keystone_admin)]$ echo $(kubectl get secrets -n vault vault-ca -o jsonpath='{.data.tls\.crt}') | base64 --decode > /home/sysadmin/vault_ca_cert/vault_ca_cert.pem
~(keystone_admin)]$ echo $(kubectl get secrets -n vault vault-ca -o jsonpath='{.data.tls\.key}') | base64 --decode > /home/sysadmin/vault_ca_cert/vault_ca_key.pem
#. Create and sign a server certificate and key:
- Create the Server private key.
.. code-block::
~(keystone_admin)]$ mkdir /home/sysadmin/vault_new_certs
~(keystone_admin)]$ openssl genrsa -out /home/sysadmin/vault_new_certs/vault-server-tls-key.pem 2048
- Create the server certificate signing request (csr).
Create a csr configuration file
``/home/sysadmin/vault_new_certs/extfile.cnf`` with the following content:
.. code-block::
[req]
prompt = no
x509_extensions = v3_req
distinguished_name = dn
[dn]
O = stx
[v3_req]
basicConstraints = critical, CA:FALSE
extendedKeyUsage = TLS Web Server Authentication, TLS Web Client Authentication
subjectAltName = @alt_names
[alt_names]
DNS.1 = sva-vault
DNS.2 = *.sva-vault-internal
DNS.3 = *.vault.pod.cluster.local
DNS.4 = sva-vault.vault
DNS.5 = sva-vault.vault.svc
DNS.6 = sva-vault.vault.svc.cluster.local
DNS.7 = sva-vault-active.vault.svc.cluster.local
IP.1 = 127.0.0.1
~(keystone_admin)]$ openssl req -new -key /home/sysadmin/vault_new_certs/vault-server-tls-key.pem -out /home/sysadmin/vault_new_certs/vault-server-tls.csr -config /home/sysadmin/vault_new_certs/extfile.cnf
- Use the Root |CA| to sign the server certificate:
.. code-block::
~(keystone_admin)]$ openssl x509 -req -in /home/sysadmin/vault_new_certs/vault-server-tls.csr -CA /home/sysadmin/vault_ca_cert/vault_ca_cert.pem -CAkey /home/sysadmin/vault_ca_cert/vault_ca_key.pem -CAcreateserial -out /home/sysadmin/vault_new_certs/vault-server-tls-cert.pem -days 365 -extensions v3_req -extfile /home/sysadmin/vault_new_certs/extfile.cnf
#. Update vault-server-tls secret with the new vault server certificate:
.. code-block::
~(keystone_admin)]$ kubectl create secret tls vault-server-tls --cert=/home/sysadmin/vault_new_certs/vault-server-tls-cert.pem --key=/home/sysadmin/vault_new_certs/vault-server-tls-key.pem --save-config --dry-run=client -n vault -o yaml | kubectl apply -f -
#. Restart vault-manager, agent-injector and vault servers to use the new server certificate:
.. code-block::
~(keystone_admin)]$ kubectl rollout restart statefulset sva-vault-manager -n vault
~(keystone_admin)]$ kubectl rollout restart deployment sva-vault-agent-injector -n vault
~(keystone_admin)]$ kubectl delete pod sva-vault-0 -n vault
~(keystone_admin)]$ kubectl delete pod sva-vault-1 -n vault
~(keystone_admin)]$ kubectl delete pod sva-vault-2 -n vault

2
doc/source/shared/abbrevs.txt
View File

@ -33,6 +33,7 @@
.. |CSKs| replace:: :abbr:`CSKs (Code Signing Keys)`
.. |CVE| replace:: :abbr:`CVE (Common Vulnerabilities and Exposures)`
.. |DAD| replace:: :abbr:`DAD (Duplicate Address Detection)`
.. |DC| replace:: :abbr:`DC (Distributed Cloud)`
.. |DHCP| replace:: :abbr:`DHCP (Dynamic Host Configuration Protocol)`
.. |DMA| replace:: :abbr:`DMA (Direct Memory Access)`
.. |DNS| replace:: :abbr:`DNS (Domain Name System)`
@ -151,5 +152,6 @@
.. |VTEP| replace:: abbr:`VTEP (Virtual Tunnel End Point)`
.. |VXLAN| replace:: :abbr:`VXLAN (Virtual eXtensible Local Area Network)`
.. |VXLANs| replace:: :abbr:`VXLANs (Virtual eXtensible Local Area Networks)`
.. |WAD| replace:: :abbr:`WAD (Windows Active Directory)`
.. |XML| replace:: :abbr:`XML (eXtensible Markup Language)`
.. |YAML| replace:: :abbr:`YAML (YAML Ain't Markup Language)`