swift/etc
Alistair Coles bbaed18e9b diskfile: don't remove recently written non-durables
DiskFileManager will remove any stale files during
cleanup_ondisk_files(): these include tombstones and nondurable EC
data fragments whose timestamps are older than reclaim_age. It can
usually be safely assumed that a non-durable data fragment older than
reclaim_age is not going to become durable. However, if an agent PUTs
objects with specified older X-Timestamps (for example the reconciler
or container-sync) then there is a window of time during which the
object server has written an old non-durable data file but has not yet
committed it to make it durable.

Previously, if another process (for example the reconstructor) called
cleanup_ondisk_files during this window then the non-durable data file
would be removed. The subsequent attempt to commit the data file would
then result in a traceback due to there no longer being a data file to
rename, and of course the data file is lost.

This patch modifies cleanup_ondisk_files to not remove old, otherwise
stale, non-durable data files that were only written to disk in the
preceding 'commit_window' seconds. 'commit_window' is configurable for
the object server and defaults to 60.0 seconds.

Closes-Bug: #1936508
Related-Change: I0d519ebaaade35249fb7b17bd5f419ffdaa616c0
Change-Id: I5f3318a44af64b77a63713e6ff8d0fd3b6144f13
2021-07-19 21:18:02 +01:00
..
account-server.conf-sample Allow floats for all intervals 2021-05-05 15:30:21 -07:00
container-reconciler.conf-sample Add concurrency to reconciler 2021-07-14 12:27:26 -07:00
container-server.conf-sample sharder: avoid small tail shards 2021-07-07 13:59:36 +01:00
container-sync-realms.conf-sample Allow floats for all intervals 2021-05-05 15:30:21 -07:00
dispersion.conf-sample Fix swift-dispersion in multi-region setups 2016-06-01 15:35:47 +02:00
drive-audit.conf-sample py3: Fix swift-drive-audit 2019-10-13 21:55:58 -07:00
internal-client.conf-sample add symlink to container sync default and sample config 2017-12-14 12:13:20 -08:00
keymaster.conf-sample py3: Work with proper native string paths in crypto meta 2020-07-29 17:33:54 -07:00
memcache.conf-sample Memcached client TLS support 2021-01-06 09:47:38 -08:00
mime.types-sample PEP 8 compliance and small modification to mime.types file 2010-11-23 19:26:02 -06:00
object-expirer.conf-sample Allow floats for all intervals 2021-05-05 15:30:21 -07:00
object-server.conf-sample diskfile: don't remove recently written non-durables 2021-07-19 21:18:02 +01:00
proxy-server.conf-sample Merge "Allow floats for all intervals" 2021-05-25 18:15:56 +00:00
rsyncd.conf-sample Update SAIO & docker image to use 62xx ports 2020-07-20 15:17:12 -07:00
swift-rsyslog.conf-sample Add sample rsyslog.conf. 2013-06-25 10:24:26 +08:00
swift.conf-sample Update docs to discourage policy names being numbers 2021-03-26 09:17:34 +00:00