Authors/ChangeLog for 2.26.0

Change-Id: Ia8e31ed0d5aefe67f2f926dc92d9acd6c0c98007

AUTHORS

@@ -193,6 +193,7 @@ Hanxi Liu (hanxi.liu@easystack.cn)
 Harshada Mangesh Kakad (harshadak@metsi.co.uk)
 Harshit Chitalia (harshit@acelio.com)
 HCLTech-SSW (hcl_ss_oss@hcl.com)
+Hervé Beraud (hberaud@redhat.com)
 hgangwx (hgangwx@cn.ibm.com)
 Hisashi Osanai (osanai.hisashi@gmail.com)
 Hodong Hwang (hodong.hwang@kt.com)
@@ -268,6 +269,7 @@ Madhuri Kumari (madhuri.rai07@gmail.com)
 Mahati Chamarthy (mahati.chamarthy@gmail.com)
 malei (malei@maleideMacBook-Pro.local)
 Mandell Degerness (mdegerness@swiftstack.com)
+manuvakery1 (manu.km@idrive.com)
 maoshuai (fwsakura@163.com)
 Marcelo Martins (btorch@gmail.com)
 Maria Malyarova (savoreux69@gmail.com)
@@ -438,9 +440,11 @@ zengjia (zengjia@awcloud.com)
 Zhang Guoqing (zhang.guoqing@99cloud.net)
 Zhang Jinnan (ben.os@99cloud.net)
 zhang.lei (zhang.lei@99cloud.net)
+zhangboye (zhangboye@inspur.com)
 zhangdebo1987 (zhangdebo@inspur.com)
 zhangyanxian (zhangyanxianmail@163.com)
 Zhao Lei (zhaolei@cn.fujitsu.com)
+zhaoleilc (15247232416@163.com)
 Zheng Yao (zheng.yao1@zte.com.cn)
 zheng yin (yin.zheng@easystack.cn)
 Zhenguo Niu (zhenguo@unitedstack.com)

CHANGELOG

@@ -1,3 +1,127 @@
+swift (2.26.0, OpenStack Victoria)
+
+    * Extend concurrent reads to erasure coded policies. Previously, the
+      options `concurrent_gets` and `concurrency_timeout` only applied to
+      replicated policies.
+
+    * Add a new `concurrent_ec_extra_requests` option to allow the proxy to
+      make some extra backend requests immediately. The proxy will respond as
+      soon as there are enough responses available to reconstruct.
+
+    * The concurrent read options (`concurrent_gets`, `concurrency_timeout`,
+      and `concurrent_ec_extra_requests`) may now be configured per
+      storage-policy.
+
+    * Replication servers can now handle all request methods. This allows
+      ssync to work with a separate replication network.
+
+    * All background daemons now use the replication network. This allows
+      better isolation between external, client-facing traffic and internal,
+      background traffic. Note that during a rolling upgrade, replication
+      servers may respond with `405 Method Not Allowed`. To avoid this,
+      operators should remove the config option `replication_server = true`
+      from their replication servers; this will allow them to handle all
+      request methods before upgrading.
+
+    * S3 API improvements:
+
+      * Fixed some SignatureDoesNotMatch errors when using the AWS .NET SDK.
+
+      * Add basic read support for object tagging. This improves
+        compatibility with AWS CLI version 2. Write support is not
+        yet implemented, so the tag set will always be empty.
+
+      * CompleteMultipartUpload requests may now be safely retried.
+
+      * Improved quota-exceeded error messages.
+
+      * Improved logging and statsd metrics. Be aware that this will cause
+        an increase in the proxy-logging statsd metrics emited for S3
+        responses. However, this should more accurately reflect the state
+        of the system.
+
+      * S3 requests are now less demanding on the container layer.
+
+    * Python 3 bug fixes:
+
+      * Fixed an error when reading encrypted data that was written while
+        running Python 2 for a path that includes non-ASCII characters. This
+        was caused by a difference in string types that resulted in
+        ambiguity when decrypting. To prevent the ambiguity for new data, set
+        `meta_version_to_write = 3` in your keymaster configuration after
+        upgrading all proxy servers.
+
+        If upgrading from Swift 2.20.0 or Swift 2.19.1 or earlier, set
+        `meta_version_to_write = 1` in your keymaster configuration prior
+        to upgrading.
+
+      * Object expiration respects the `expiring_objects_container_divisor`
+        config option.
+
+      * `fallocate_reserve` may be specified as a percentage in more places.
+
+      * The ETag-quoting middleware no longer raises TypeErrors.
+
+    * Sharding improvements:
+
+      * Prevent object updates from auto-creating shard containers. This
+        ensures more consistent listings for sharded containers during
+        rebalances.
+
+      * Deleted shard containers are no longer considered root containers.
+        This prevents unnecessary sharding audit failures and allows the
+        deleted shard database to actually be unlinked.
+
+      * `swift-container-info` now summarizes shard range information.
+        Pass `-v`/`--verbose` if you want to see all of them.
+
+      * Improved container-sharder stat reporting to reduce load on root
+        container databases.
+
+      * Don't inject shard ranges when user quits.
+
+    * Servers now open one listen socket per worker, ensuring each worker
+      serves roughly the same number of concurrent connections.
+
+    * Server workers may now be gracefully terminated via `SIGHUP` or
+      `SIGUSR1`. The parent process will then spawn a fresh worker.
+
+    * During rebalances, clients should no longer get 404s for data that
+      exists but whose replicas are overloaded.
+
+    * Improved cache management for account and container responses.
+
+    * Allow proxy-logging middlewares to be configured more independently.
+
+    * Allow operators to pass either raw or URL-quoted paths to
+      swift-get-nodes. Notably, this allows swift-get-nodes to work with
+      the reserved namespace used for object versioning.
+
+    * Container read ACLs now work with object versioning. This only
+      allows access to the most-recent version via an unversioned URL.
+
+    * Improved how containers reclaim deleted rows to reduce locking and object
+      update throughput.
+
+    * Large object reads log fewer client disconnects.
+
+    * Allow ratelimit to be placed multiple times in a proxy pipeline,
+      such as both before s3api and auth (to handle swift requests without
+      needing to make an auth decision) and after (to limit S3 requests).
+
+    * Shuffle object-updater work. This somewhat reduces the impact a
+      single overloaded database has on other containers' listings.
+
+    * Fix a proxy-server error when retrieving erasure coded data when
+      there are durable fragments but not enough to reconstruct.
+
+    * Fix an error in the proxy server when finalizing data.
+
+    * Improve performance when increasing partition power.
+
+    * Various other minor bug fixes and improvements.
+
+
 swift (2.25.0, OpenStack Ussuri)
 
     * WSGI server processes can now notify systemd when they are ready.

releasenotes/notes/2_26_0_release-6548eadcba544f72.yaml

@@ -0,0 +1,216 @@
+---
+features:
+  - |
+    Extend concurrent reads to erasure coded policies. Previously, the
+    options ``concurrent_gets`` and ``concurrency_timeout`` only applied to
+    replicated policies.
+
+  - |
+    Add a new ``concurrent_ec_extra_requests`` option to allow the proxy to
+    make some extra backend requests immediately. The proxy will respond as
+    soon as there are enough responses available to reconstruct.
+
+  - |
+    The concurrent read options (``concurrent_gets``, ``concurrency_timeout``,
+    and ``concurrent_ec_extra_requests``) may now be configured per
+    storage-policy.
+
+  - |
+    Replication servers can now handle all request methods. This allows
+    ssync to work with a separate replication network.
+
+  - |
+    All background daemons now use the replication network. This allows
+    better isolation between external, client-facing traffic and internal,
+    background traffic. Note that during a rolling upgrade, replication
+    servers may respond with ``405 Method Not Allowed``. To avoid this,
+    operators should remove the config option ``replication_server = true``
+    from their replication servers; this will allow them to handle all
+    request methods before upgrading.
+
+  - |
+    S3 API improvements:
+
+    * Fixed some SignatureDoesNotMatch errors when using the AWS .NET SDK.
+
+    * Add basic read support for object tagging. This improves
+      compatibility with AWS CLI version 2. Write support is not
+      yet implemented, so the tag set will always be empty.
+
+    * CompleteMultipartUpload requests may now be safely retried.
+
+    * Improved quota-exceeded error messages.
+
+    * Improved logging and statsd metrics. Be aware that this will cause
+      an increase in the proxy-logging statsd metrics emited for S3
+      responses. However, this should more accurately reflect the state
+      of the system.
+
+    * S3 requests are now less demanding on the container layer.
+
+  - |
+    Servers now open one listen socket per worker, ensuring each worker
+    serves roughly the same number of concurrent connections.
+
+  - |
+    Server workers may now be gracefully terminated via ``SIGHUP`` or
+    ``SIGUSR1``. The parent process will then spawn a fresh worker.
+
+  - |
+    Allow proxy-logging middlewares to be configured more independently.
+
+  - |
+    Improve performance when increasing partition power.
+
+issues:
+  - |
+    In a rolling upgrade from liberasurecode 1.5.0 or earlier to 1.6.0 or
+    later, object-servers may quarantine newly-written data, leading to
+    availability issues or even data loss. See `bug 1886088
+    <https://bugs.launchpad.net/liberasurecode/+bug/1886088>`__ for more
+    information, including how to determine whether you are affected.
+    Several mitigations are available to operators:
+
+    * If proxy and object layers can be upgraded independently and proxies
+      can be upgraded quickly:
+
+      1. Stop and disable the object-reconstructor before upgrading. This
+         ensures no upgraded object server starts writing new fragments
+         that old object servers would quarantine.
+
+      2. Upgrade liberasurecode on all object servers. Object servers can
+         now read both old and new fragments.
+
+      3. Upgrade liberasurecode on all proxy servers. Newly-written data
+         will now use new fragments. Note that not-yet-upgraded proxies
+         will not be able to read these newly-written fragments but will
+         instead respond ``500 Internal Server Error``.
+
+      4. After upgrading, re-enable and restart the object-reconstructor.
+
+    * If your users can tolerate it, consider a read-only rolling upgrade.
+      Before upgrading, enable the `read-only middleware
+      <https://docs.openstack.org/swift/latest/middleware.html#read-only>`__
+      cluster-wide to prevent new writes during the upgrade. Additionally,
+      stop and disable the object-reconstructor as above. Upgrade normally,
+      then disable the read-only middleware and re-enable and restart the
+      object-reconstructor.
+
+    * Avoid upgrading liberasurecode until swift and liberasurecode
+      better-support a rolling upgrade. Swift remains compatible with
+      liberasurecode 1.5.0 and earlier.
+
+    .. note::
+       Ubuntu 18.04 and RDO's CentOS 7 repos package liberasurecode 1.5.0,
+       while Ubuntu 20.04 and RDO's CentOS 8 repos currently package
+       liberasurecode 1.6.0 or 1.6.1. Take care when upgrading major distro
+       versions!
+
+upgrade:
+  - |
+    **If your cluster has encryption enabled and is still running Swift
+    under Python 2**, we recommend upgrading Swift *before* transitioning to
+    Python 3. Otherwise, new writes to objects with non-ASCII characters
+    in their paths may result in corrupted downloads when read from a
+    proxy-server still running old swift on Python 2. See `bug 1888037
+    <https://bugs.launchpad.net/swift/+bug/1888037>`__ for more information.
+    Note that new tags including a fix for the bug are planned for all
+    maintained stable branches; upgrading to any one of those should be
+    sufficient to ensure a smooth upgrade to the latest Swift.
+
+  - |
+    The above bug was caused by a difference in string types that resulted
+    in ambiguity when decrypting. To prevent the ambiguity for new data, set
+    ``meta_version_to_write = 3`` in your keymaster configuration *after*
+    upgrading all proxy servers.
+
+    If upgrading from Swift 2.20.0 or Swift 2.19.1 or earlier, set
+    ``meta_version_to_write = 1`` in your keymaster configuration *prior*
+    to upgrading.
+
+    See the provided ``keymaster.conf-sample`` for more information about
+    this setting.
+
+  - |
+    **If your cluster is configured with a separate replication network**,
+    note that background daemons will switch to using this network for all
+    traffic. If your account, container, or object replication servers are
+    configured with ``replication_server = true``, these daemons may log a
+    flood of ``405 Method Not Allowed`` messages during a rolling upgrade.
+    To avoid this, comment out the option and restart replication servers
+    before upgrading.
+
+fixes:
+  - |
+    Python 3 bug fixes:
+
+    * Fixed an error when reading encrypted data that was written while
+      running Python 2 for a path that includes non-ASCII characters.
+
+    * Object expiration respects the ``expiring_objects_container_divisor``
+      config option.
+
+    * ``fallocate_reserve`` may be specified as a percentage in more places.
+
+    * The ETag-quoting middleware no longer raises TypeErrors.
+
+  - |
+    Sharding improvements:
+
+    * Prevent object updates from auto-creating shard containers. This
+      ensures more consistent listings for sharded containers during
+      rebalances.
+
+    * Deleted shard containers are no longer considered root containers.
+      This prevents unnecessary sharding audit failures and allows the
+      deleted shard database to actually be unlinked.
+
+    * ``swift-container-info`` now summarizes shard range information.
+      Pass ``-v``/``--verbose`` if you want to see all of them.
+
+    * Improved container-sharder stat reporting to reduce load on root
+      container databases.
+
+    * Don't inject shard ranges when user quits.
+
+  - |
+    During rebalances, clients should no longer get 404s for data that
+    exists but whose replicas are overloaded.
+
+  - |
+    Improved cache management for account and container responses.
+
+  - |
+    Allow operators to pass either raw or URL-quoted paths to
+    ``swift-get-nodes``. Notably, this allows ``swift-get-nodes`` to
+    work with the reserved namespace used for object versioning.
+
+  - |
+    Container read ACLs now work with object versioning. This only
+    allows access to the most-recent version via an unversioned URL.
+
+  - |
+    Improved how containers reclaim deleted rows to reduce locking and object
+    update throughput.
+
+  - |
+    Large object reads log fewer client disconnects.
+
+  - |
+    Allow ratelimit to be placed multiple times in a proxy pipeline,
+    such as both before s3api and auth (to handle swift requests without
+    needing to make an auth decision) and after (to limit S3 requests).
+
+  - |
+    Shuffle object-updater work. This somewhat reduces the impact a
+    single overloaded database has on other containers' listings.
+
+  - |
+    Fix a proxy-server error when retrieving erasure coded data when
+    there are durable fragments but not enough to reconstruct.
+
+  - |
+    Fix an error in the proxy server when finalizing data.
+
+  - |
+    Various other minor bug fixes and improvements.