openstack/swift
Code Issues Proposed changes

pickle_async_update should create tmp_dir

Browse Source 
While creating a probe test for the expirer daemon, I found
the following error scenario:

1. Introduce a new object server. Initially it doesn't have a tmp_dir.
2. Have the object-replicator replicate some objects, one of them
    with an expiration (X-Delete-At).
3. Send a DELETE request for the expired object.

While beginning to process the DELETE request, the fresh
object server still doesn't have a tmp_dir created.
Since the object has an old expiration value, the object server
will first call "delete_at_update", before creating a tombstone.
delete_at_update then must create an async_pending,
which will lead to an IO error, since tmp_dir doesn't exist.

As said, I have witnessed this in practice in the probe test I wrote
at https://review.openstack.org/#/c/326903/.

This patch changes pickle_async_update behavior to create
tmp_dir, in case it doesn't exist.

Change-Id: I88b0e5f75a2a28d6880694ff327ac2763c816d24
This commit is contained in:
Or Ozeri 2016-06-16 11:14:14 +03:00
parent 3944d82038
commit da4a59f8e2
2 changed files with 3 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
swift/obj/diskfile.py
View File

@ -1115,12 +1115,14 @@ class BaseDiskFileManager(object):
""" """
device_path = self.construct_dev_path(device) device_path = self.construct_dev_path(device)
async_dir = os.path.join(device_path, get_async_dir(policy)) async_dir = os.path.join(device_path, get_async_dir(policy))
tmp_dir = os.path.join(device_path, get_tmp_dir(policy))
mkdirs(tmp_dir)
ohash = hash_path(account, container, obj) ohash = hash_path(account, container, obj)
write_pickle( write_pickle(
data, data,
os.path.join(async_dir, ohash[-3:], ohash + '-' + os.path.join(async_dir, ohash[-3:], ohash + '-' +
Timestamp(timestamp).internal), Timestamp(timestamp).internal),
os.path.join(device_path, get_tmp_dir(policy))) tmp_dir)
self.logger.increment('async_pendings') self.logger.increment('async_pendings')
def get_diskfile(self, device, partition, account, container, obj, def get_diskfile(self, device, partition, account, container, obj,

4
test/unit/obj/test_diskfile.py
View File

@ -246,10 +246,6 @@ class TestDiskFileModuleMethods(unittest.TestCase):
self.assertFalse(os.path.isdir(tmp_path)) self.assertFalse(os.path.isdir(tmp_path))
pickle_args = (self.existing_device, 'a', 'c', 'o', pickle_args = (self.existing_device, 'a', 'c', 'o',
'data', 0.0, policy) 'data', 0.0, policy)
# async updates don't create their tmpdir on their own
self.assertRaises(OSError, self.df_mgr.pickle_async_update,
*pickle_args)
os.makedirs(tmp_path)
# now create a async update # now create a async update
self.df_mgr.pickle_async_update(*pickle_args) self.df_mgr.pickle_async_update(*pickle_args)
# check tempdir # check tempdir