306 lines
10 KiB
ReStructuredText
306 lines
10 KiB
ReStructuredText
.. _external-ceph-guide:
|
|
|
|
=============
|
|
External Ceph
|
|
=============
|
|
|
|
Kolla Ansible does not provide support for provisioning and configuring a
|
|
Ceph cluster directly. Instead, administrators should use a tool dedicated
|
|
to this purpose, such as:
|
|
|
|
* `ceph-ansible <https://docs.ceph.com/projects/ceph-ansible/en/latest/>`_
|
|
* `cephadm <https://docs.ceph.com/en/latest/cephadm/install/>`_
|
|
|
|
The desired pool(s) and keyrings should then be created via the Ceph CLI
|
|
or similar.
|
|
|
|
Requirements
|
|
~~~~~~~~~~~~
|
|
|
|
* An existing installation of Ceph
|
|
* Existing Ceph storage pools
|
|
* Existing credentials in Ceph for OpenStack services to connect to Ceph
|
|
(Glance, Cinder, Nova, Gnocchi, Manila)
|
|
|
|
Refer to https://docs.ceph.com/en/latest/rbd/rbd-openstack/ for details on
|
|
creating the pool and keyrings with appropriate permissions for each service.
|
|
|
|
Configuring External Ceph
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Ceph integration is configured for different OpenStack services independently.
|
|
|
|
Glance
|
|
------
|
|
|
|
Ceph RBD can be used as a storage backend for Glance images. Configuring Glance
|
|
for Ceph includes the following steps:
|
|
|
|
#. Enable Glance Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
glance_backend_ceph: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_glance_keyring`` (default: ``ceph.client.glance.keyring``)
|
|
* ``ceph_glance_user`` (default: ``glance``)
|
|
* ``ceph_glance_pool_name`` (default: ``images``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/glance/ceph.conf``
|
|
|
|
.. path /etc/kolla/config/glance/ceph.conf
|
|
.. code-block:: ini
|
|
|
|
[global]
|
|
fsid = 1d89fec3-325a-4963-a950-c4afedd37fe3
|
|
mon_initial_members = ceph-0
|
|
mon_host = 192.168.0.56
|
|
auth_cluster_required = cephx
|
|
auth_service_required = cephx
|
|
auth_client_required = cephx
|
|
|
|
#. Copy Ceph keyring to ``/etc/kolla/config/glance/<ceph_glance_keyring>``
|
|
|
|
#. For copy-on-write set following in ``/etc/kolla/config/glance.conf``:
|
|
|
|
.. path /etc/kolla/config/glance.conf
|
|
.. code-block:: ini
|
|
|
|
[GLOBAL]
|
|
show_image_direct_url = True
|
|
|
|
.. warning::
|
|
|
|
``show_image_direct_url`` can present a security risk if using more
|
|
than just Ceph as Glance backend(s). Please see
|
|
:glance-doc:`Glance show_image_direct_url <configuration/glance_api.html#DEFAULT.show_image_direct_url>`
|
|
|
|
Cinder
|
|
------
|
|
|
|
Ceph RBD can be used as a storage backend for Cinder volumes. Configuring
|
|
Cinder for Ceph includes following steps:
|
|
|
|
#. When using external Ceph, there may be no nodes defined in the storage
|
|
group. This will cause Cinder and related services relying on this group to
|
|
fail. In this case, operator should add some nodes to the storage group,
|
|
all the nodes where ``cinder-volume`` and ``cinder-backup`` will run:
|
|
|
|
.. code-block:: ini
|
|
|
|
[storage]
|
|
control01
|
|
|
|
#. Enable Cinder Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
cinder_backend_ceph: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_cinder_keyring`` (default: ``ceph.client.cinder.keyring``)
|
|
* ``ceph_cinder_user`` (default: ``cinder``)
|
|
* ``ceph_cinder_pool_name`` (default: ``volumes``)
|
|
* ``ceph_cinder_backup_keyring``
|
|
(default: ``ceph.client.cinder-backup.keyring``)
|
|
* ``ceph_cinder_backup_user`` (default: ``cinder-backup``)
|
|
* ``ceph_cinder_backup_pool_name`` (default: ``backups``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/cinder/ceph.conf``
|
|
|
|
Separate configuration options can be configured for
|
|
cinder-volume and cinder-backup by adding ceph.conf files to
|
|
``/etc/kolla/config/cinder/cinder-volume`` and
|
|
``/etc/kolla/config/cinder/cinder-backup`` respectively. They
|
|
will be merged with ``/etc/kolla/config/cinder/ceph.conf``.
|
|
|
|
#. Copy Ceph keyring files to:
|
|
|
|
* ``/etc/kolla/config/cinder/cinder-volume/<ceph_cinder_keyring>``
|
|
* ``/etc/kolla/config/cinder/cinder-backup/<ceph_cinder_keyring>``
|
|
* ``/etc/kolla/config/cinder/cinder-backup/<ceph_cinder_backup_keyring>``
|
|
|
|
.. note::
|
|
|
|
``cinder-backup`` requires two keyrings for accessing volumes
|
|
and backup pool.
|
|
|
|
Nova must also be configured to allow access to Cinder volumes:
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_cinder_keyring`` (default: ``ceph.client.cinder.keyring``)
|
|
|
|
#. Copy Ceph keyring file(s) to:
|
|
|
|
* ``/etc/kolla/config/nova/<ceph_cinder_keyring>``
|
|
|
|
Nova
|
|
----
|
|
|
|
Ceph RBD can be used as a storage backend for Nova instance ephemeral disks.
|
|
This avoids the requirement for local storage for instances on compute nodes.
|
|
It improves the performance of migration, since instances' ephemeral disks do
|
|
not need to be copied between hypervisors.
|
|
|
|
Configuring Nova for Ceph includes following steps:
|
|
|
|
#. Enable Nova Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
nova_backend_ceph: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_nova_keyring`` (by default it's the same as
|
|
``ceph_cinder_keyring``)
|
|
* ``ceph_nova_user`` (by default it's the same as ``ceph_cinder_user``)
|
|
* ``ceph_nova_pool_name`` (default: ``vms``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/nova/ceph.conf``
|
|
#. Copy Ceph keyring file(s) to:
|
|
|
|
* ``/etc/kolla/config/nova/<ceph_nova_keyring>``
|
|
|
|
.. note::
|
|
|
|
If you are using a Ceph deployment tool that generates separate Ceph
|
|
keys for Cinder and Nova, you will need to override
|
|
``ceph_nova_keyring`` and ``ceph_nova_user`` to match.
|
|
|
|
Gnocchi
|
|
-------
|
|
|
|
Ceph object storage can be used as a storage backend for Gnocchi metrics.
|
|
Configuring Gnocchi for Ceph includes following steps:
|
|
|
|
#. Enable Gnocchi Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
gnocchi_backend_storage: "ceph"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_gnocchi_keyring``
|
|
(default: ``ceph.client.gnocchi.keyring``)
|
|
* ``ceph_gnocchi_user`` (default: ``gnocchi``)
|
|
* ``ceph_gnocchi_pool_name`` (default: ``gnocchi``)
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/gnocchi/ceph.conf``
|
|
#. Copy Ceph keyring to ``/etc/kolla/config/gnocchi/<ceph_gnocchi_keyring>``
|
|
|
|
Manila
|
|
------
|
|
|
|
CephFS can be used as a storage backend for Manila shares. Configuring Manila
|
|
for Ceph includes following steps:
|
|
|
|
#. Enable Manila Ceph backend in ``globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
enable_manila_backend_cephfs_native: "yes"
|
|
|
|
#. Configure Ceph authentication details in ``/etc/kolla/globals.yml``:
|
|
|
|
* ``ceph_manila_keyring`` (default: ``ceph.client.manila.keyring``)
|
|
* ``ceph_manila_user`` (default: ``manila``)
|
|
|
|
.. note::
|
|
|
|
Required Ceph identity caps for manila user are documented in
|
|
:manila-doc:`CephFS Native driver <admin/cephfs_driver.html#authorizing-the-driver-to-communicate-with-ceph>`.
|
|
|
|
#. Copy Ceph configuration file to ``/etc/kolla/config/manila/ceph.conf``
|
|
#. Copy Ceph keyring to ``/etc/kolla/config/manila/<ceph_manila_keyring>``
|
|
|
|
#. If using multiple filesystems (Ceph Pacific+), set
|
|
``manila_cephfs_filesystem_name`` in ``/etc/kolla/globals.yml`` to the
|
|
name of the Ceph filesystem Manila should use.
|
|
By default, Manila will use the first filesystem returned by
|
|
the ``ceph fs volume ls`` command.
|
|
|
|
#. Setup Manila in the usual way
|
|
|
|
For more details on the rest of the Manila setup, such as creating the share
|
|
type ``default_share_type``, please see :doc:`Manila in Kolla <manila-guide>`.
|
|
|
|
For more details on the CephFS Native driver, please see
|
|
:manila-doc:`CephFS Native driver <admin/cephfs_driver.html>`.
|
|
|
|
RadosGW
|
|
-------
|
|
|
|
As of the Xena 13.0.0 release, Kolla Ansible supports integration with Ceph
|
|
RadosGW. This includes:
|
|
|
|
* Registration of Swift-compatible endpoints in Keystone
|
|
* Load balancing across RadosGW API servers using HAProxy
|
|
|
|
See the `Ceph documentation
|
|
<https://docs.ceph.com/en/latest/radosgw/keystone/>`__ for further information,
|
|
including changes that must be applied to the Ceph cluster configuration.
|
|
|
|
Enable Ceph RadosGW integration:
|
|
|
|
.. code-block:: yaml
|
|
|
|
enable_ceph_rgw: true
|
|
|
|
Keystone integration
|
|
====================
|
|
|
|
A Keystone user and endpoints are registered by default, however this may be
|
|
avoided by setting ``enable_ceph_rgw_keystone`` to ``false``. If registration
|
|
is enabled, the username is defined via ``ceph_rgw_keystone_user``, and this
|
|
defaults to ``ceph_rgw``. The hostnames used by the endpoints default to
|
|
``ceph_rgw_external_fqdn`` and ``ceph_rgw_internal_fqdn`` for the public and
|
|
internal endpoints respectively. These default to ``kolla_external_fqdn`` and
|
|
``kolla_internal_fqdn`` respectively. The port used by the endpoints is defined
|
|
via ``ceph_rgw_port``, and defaults to 6780.
|
|
|
|
By default RadosGW supports both Swift and S3 API, and it is not completely
|
|
compatible with Swift API. The option ``ceph_rgw_swift_compatibility`` can
|
|
enable/disable complete RadosGW compatibility with Swift API. This should
|
|
match the configuration used by Ceph RadosGW. After changing the value, run
|
|
the ``kolla-ansible deploy`` command to enable.
|
|
|
|
By default, the RadosGW endpoint URL does not include the project (account) ID.
|
|
This prevents cross-project and public object access. This can be resolved by
|
|
setting ``ceph_rgw_swift_account_in_url`` to ``true``. This should match the
|
|
``rgw_swift_account_in_url`` configuration option in Ceph RadosGW.
|
|
|
|
Load balancing
|
|
==============
|
|
|
|
.. warning::
|
|
|
|
Users of Ceph RadosGW can generate very high volumes of traffic. It is
|
|
advisable to use a separate load balancer for RadosGW for anything other
|
|
than small or lightly utilised RadosGW deployments, however this is
|
|
currently out of scope for Kolla Ansible.
|
|
|
|
Load balancing is enabled by default, however this may be avoided by setting
|
|
``enable_ceph_rgw_loadbalancer`` to ``false``. If using load balancing, the
|
|
RadosGW hosts and ports must be configured. Each item should contain
|
|
``host`` and ``port`` keys. The ``ip`` and ``port`` keys are optional. If
|
|
``ip`` is not specified, the ``host`` values should be resolvable from the host
|
|
running HAProxy. If the ``port`` is not specified, the default HTTP (80) or
|
|
HTTPS (443) port will be used. For example:
|
|
|
|
.. code-block:: yaml
|
|
|
|
ceph_rgw_hosts:
|
|
- host: rgw-host-1
|
|
- host: rgw-host-2
|
|
ip: 10.0.0.42
|
|
port: 8080
|
|
|
|
The HAProxy frontend port is defined via ``ceph_rgw_port``, and defaults to
|
|
6780.
|