8cc569306a
Per [1] and exchange on IRC. [1] http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026437.html Change-Id: I322500e7204eb129d7bf085006627e8c4aaaa934
316 lines
9.6 KiB
ReStructuredText
316 lines
9.6 KiB
ReStructuredText
.. _swift-guide:
|
|
|
|
==============================
|
|
Swift - Object storage service
|
|
==============================
|
|
|
|
Overview
|
|
~~~~~~~~
|
|
|
|
Kolla can deploy a full working Swift setup in either a **all-in-one** or
|
|
**multinode** setup.
|
|
|
|
Networking
|
|
~~~~~~~~~~
|
|
|
|
The following interfaces are used by Swift:
|
|
|
|
External API interface (``kolla_external_vip_interface``)
|
|
This interface is used by users to access the Swift public API.
|
|
Internal API interface (``api_interface``)
|
|
This interface is used by users to access the Swift internal API. It is also
|
|
used by HAProxy to access the Swift proxy servers.
|
|
Swift Storage interface (``swift_storage_interface``)
|
|
This interface is used by the Swift proxy server to access the account,
|
|
container and object servers.
|
|
Swift replication interface (``swift_replication_interface``)
|
|
This interface is used for Swift storage replication traffic.
|
|
This is optional as the default configuration uses
|
|
the ``swift_storage_interface`` for replication traffic.
|
|
|
|
Disks with a partition table (recommended)
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Swift requires block devices to be available for storage. To prepare a disk
|
|
for use as a Swift storage device, a special partition name and filesystem
|
|
label need to be added.
|
|
|
|
The following should be done on each storage node, the example is shown
|
|
for three disks:
|
|
|
|
.. warning::
|
|
|
|
ALL DATA ON DISK will be LOST!
|
|
|
|
.. code-block:: console
|
|
|
|
index=0
|
|
for d in sdc sdd sde; do
|
|
parted /dev/${d} -s -- mklabel gpt mkpart KOLLA_SWIFT_DATA 1 -1
|
|
sudo mkfs.xfs -f -L d${index} /dev/${d}1
|
|
(( index++ ))
|
|
done
|
|
|
|
For evaluation, loopback devices can be used in lieu of real disks:
|
|
|
|
.. code-block:: console
|
|
|
|
index=0
|
|
for d in sdc sdd sde; do
|
|
free_device=$(losetup -f)
|
|
fallocate -l 1G /tmp/$d
|
|
losetup $free_device /tmp/$d
|
|
parted $free_device -s -- mklabel gpt mkpart KOLLA_SWIFT_DATA 1 -1
|
|
sudo mkfs.xfs -f -L d${index} ${free_device}p1
|
|
(( index++ ))
|
|
done
|
|
|
|
Disks without a partition table
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Kolla also supports unpartitioned disk (filesystem on ``/dev/sdc`` instead of
|
|
``/dev/sdc1``) detection purely based on filesystem label. This is generally
|
|
not a recommended practice but can be helpful for Kolla to take over Swift
|
|
deployment already using disk like this.
|
|
|
|
Given hard disks with labels swd1, swd2, swd3, use the following settings in
|
|
``ansible/roles/swift/defaults/main.yml``.
|
|
|
|
.. code-block:: yaml
|
|
|
|
swift_devices_match_mode: "prefix"
|
|
swift_devices_name: "swd"
|
|
|
|
Rings
|
|
~~~~~
|
|
|
|
Before running Swift we need to generate **rings**, which are binary compressed
|
|
files that at a high level let the various Swift services know where data is in
|
|
the cluster. We hope to automate this process in a future release.
|
|
|
|
The following example commands should be run from the ``operator`` node to
|
|
generate rings for a demo setup. The commands work with **disks with partition
|
|
table** example listed above. Please modify accordingly if your setup is
|
|
different.
|
|
|
|
If using a separate replication network it is necessary to add the replication
|
|
network IP addresses to the rings. See the :swift-doc:`Swift documentation
|
|
<replication_network.html#dedicated-replication-network>` for details on
|
|
how to do that.
|
|
|
|
Prepare for Rings generating
|
|
----------------------------
|
|
|
|
To prepare for Swift Rings generating, run the following commands to initialize
|
|
the environment variable and create ``/etc/kolla/config/swift`` directory:
|
|
|
|
.. code-block:: console
|
|
|
|
STORAGE_NODES=(192.168.0.2 192.168.0.3 192.168.0.4)
|
|
KOLLA_SWIFT_BASE_IMAGE="kolla/centos-source-swift-base:4.0.0"
|
|
mkdir -p /etc/kolla/config/swift
|
|
|
|
Generate Object Ring
|
|
--------------------
|
|
|
|
To generate Swift object ring, run the following commands:
|
|
|
|
.. code-block:: console
|
|
|
|
docker run \
|
|
--rm \
|
|
-v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
|
|
$KOLLA_SWIFT_BASE_IMAGE \
|
|
swift-ring-builder \
|
|
/etc/kolla/config/swift/object.builder create 10 3 1
|
|
|
|
for node in ${STORAGE_NODES[@]}; do
|
|
for i in {0..2}; do
|
|
docker run \
|
|
--rm \
|
|
-v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
|
|
$KOLLA_SWIFT_BASE_IMAGE \
|
|
swift-ring-builder \
|
|
/etc/kolla/config/swift/object.builder add r1z1-${node}:6000/d${i} 1;
|
|
done
|
|
done
|
|
|
|
Generate Account Ring
|
|
---------------------
|
|
|
|
To generate Swift account ring, run the following commands:
|
|
|
|
.. code-block:: console
|
|
|
|
docker run \
|
|
--rm \
|
|
-v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
|
|
$KOLLA_SWIFT_BASE_IMAGE \
|
|
swift-ring-builder \
|
|
/etc/kolla/config/swift/account.builder create 10 3 1
|
|
|
|
for node in ${STORAGE_NODES[@]}; do
|
|
for i in {0..2}; do
|
|
docker run \
|
|
--rm \
|
|
-v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
|
|
$KOLLA_SWIFT_BASE_IMAGE \
|
|
swift-ring-builder \
|
|
/etc/kolla/config/swift/account.builder add r1z1-${node}:6001/d${i} 1;
|
|
done
|
|
done
|
|
|
|
Generate Container Ring
|
|
-----------------------
|
|
|
|
To generate Swift container ring, run the following commands:
|
|
|
|
.. code-block:: console
|
|
|
|
docker run \
|
|
--rm \
|
|
-v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
|
|
$KOLLA_SWIFT_BASE_IMAGE \
|
|
swift-ring-builder \
|
|
/etc/kolla/config/swift/container.builder create 10 3 1
|
|
|
|
for node in ${STORAGE_NODES[@]}; do
|
|
for i in {0..2}; do
|
|
docker run \
|
|
--rm \
|
|
-v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
|
|
$KOLLA_SWIFT_BASE_IMAGE \
|
|
swift-ring-builder \
|
|
/etc/kolla/config/swift/container.builder add r1z1-${node}:6002/d${i} 1;
|
|
done
|
|
done
|
|
|
|
.. end
|
|
|
|
Rebalance
|
|
---------
|
|
|
|
To rebalance the ring files:
|
|
|
|
.. code-block:: console
|
|
|
|
for ring in object account container; do
|
|
docker run \
|
|
--rm \
|
|
-v /etc/kolla/config/swift/:/etc/kolla/config/swift/ \
|
|
$KOLLA_SWIFT_BASE_IMAGE \
|
|
swift-ring-builder \
|
|
/etc/kolla/config/swift/${ring}.builder rebalance;
|
|
done
|
|
|
|
For more information, see :swift-doc:`the Swift documentation
|
|
<install/initial-rings.html>`.
|
|
|
|
Deploying
|
|
~~~~~~~~~
|
|
|
|
Enable Swift in ``/etc/kolla/globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
enable_swift : "yes"
|
|
|
|
If you are to deploy multiple policies, override the variable
|
|
``swift_extra_ring_files`` with the list of your custom ring files, .builder
|
|
and .ring.gz all together. This will append them to the list of default rings.
|
|
|
|
.. code-block:: yaml
|
|
|
|
swift_extra_ring_files:
|
|
- object-1.builder
|
|
- object-1.ring.gz
|
|
|
|
Once the rings are in place, deploying Swift is the same as any other Kolla
|
|
Ansible service:
|
|
|
|
.. code-block:: console
|
|
|
|
# kolla-ansible deploy -i <path/to/inventory-file>
|
|
|
|
Verification
|
|
~~~~~~~~~~~~
|
|
|
|
A very basic smoke test:
|
|
|
|
.. code-block:: console
|
|
|
|
$ openstack container create mycontainer
|
|
|
|
+---------------------------------------+--------------+------------------------------------+
|
|
| account | container | x-trans-id |
|
|
+---------------------------------------+--------------+------------------------------------+
|
|
| AUTH_7b938156dba44de7891f311c751f91d8 | mycontainer | txb7f05fa81f244117ac1b7-005a0e7803 |
|
|
+---------------------------------------+--------------+------------------------------------+
|
|
|
|
$ openstack object create mycontainer README.rst
|
|
|
|
+---------------+--------------+----------------------------------+
|
|
| object | container | etag |
|
|
+---------------+--------------+----------------------------------+
|
|
| README.rst | mycontainer | 2634ecee0b9a52ba403a503cc7d8e988 |
|
|
+---------------+--------------+----------------------------------+
|
|
|
|
$ openstack container show mycontainer
|
|
|
|
+--------------+---------------------------------------+
|
|
| Field | Value |
|
|
+--------------+---------------------------------------+
|
|
| account | AUTH_7b938156dba44de7891f311c751f91d8 |
|
|
| bytes_used | 6684 |
|
|
| container | mycontainer |
|
|
| object_count | 1 |
|
|
+--------------+---------------------------------------+
|
|
|
|
$ openstack object store account show
|
|
|
|
+------------+---------------------------------------+
|
|
| Field | Value |
|
|
+------------+---------------------------------------+
|
|
| Account | AUTH_7b938156dba44de7891f311c751f91d8 |
|
|
| Bytes | 6684 |
|
|
| Containers | 1 |
|
|
| Objects | 1 |
|
|
+------------+---------------------------------------+
|
|
|
|
S3 API
|
|
~~~~~~
|
|
|
|
The Swift S3 API can be enabled by setting ``enable_swift_s3api`` to ``true``
|
|
in ``globals.yml``. It is disabled by default. In order to use this API it is
|
|
necessary to obtain EC2 credentials from Keystone. See the :swift-doc:`the
|
|
Swift documentation
|
|
<admin/middleware.html#module-swift.common.middleware.s3api.s3api>` for
|
|
details.
|
|
|
|
Swift Recon
|
|
~~~~~~~~~~~
|
|
|
|
Enable Swift Recon in ``/etc/kolla/globals.yml``:
|
|
|
|
.. code-block:: yaml
|
|
|
|
enable_swift_recon : "yes"
|
|
|
|
|
|
The Swift role in Kolla Ansible is still using the old role format. Unlike many
|
|
other Kolla Ansible roles, it won't automatically add the new volume to the
|
|
containers in existing deployments when running `kolla-ansible reconfigure`.
|
|
Instead we must use the `kolla-ansible upgrade` command, which will remove the
|
|
existing containers and then put them back again.
|
|
|
|
Example usage:
|
|
|
|
.. code-block:: console
|
|
|
|
$ sudo docker exec swift_object_server swift-recon --all`
|
|
|
|
|
|
|
|
For more information, see :swift-doc:`the Swift documentation
|
|
<admin/objectstorage-monitoring.html>`.
|