eb66cfe767
This patch implements the register/deregister/lookup operations for the sharding manager. As a result of this patch, the sharding driver now takes the control driver as a paramater. This gives it the access needed to make decisions about queue placement (and lookups) based on the entries in the control plane. In order to get the sharding manager working correctly, several methods had to be overridden to properly handle the QueueNotMapped error. When a QueueNotMapped error is encountered under normal conditions, it is equivalent to the target resource not being found. The overrides mimic the default behavior of those controllers' methods in the case where the queue isn't found. An inconsistency was found between the base storage driver and the concrete implementations. message_controller.list did not provide an include_claimed parameter. This has been corrected. To avoid DuplicateOptErrors in the mongodb driver in a sharded context, a filtering algorithm is applied. To make it clear to data drivers that they need to apply this filtering algorithm, conf.dynamic is set by the sharding driver. Unit tests have been added for the weighted select algorithm. Up next: sharded queue listing support, caching Change-Id: Ic36cd657ba13fa4c0b624dc5c71bfe99f9a70139 Partitally-implements: blueprint storage-sharding
66 lines
2.0 KiB
Python
66 lines
2.0 KiB
Python
# Copyright (c) 2013 Rackspace Hosting, Inc.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
# you may not use this file except in compliance with the License.
|
|
# You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
# implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
|
|
"""utils: general-purpose utilities."""
|
|
|
|
from oslo.config import cfg
|
|
import six
|
|
|
|
|
|
def fields(d, names, pred=lambda x: True,
|
|
key_transform=lambda x: x, value_transform=lambda x: x):
|
|
"""Returns the entries in this dictionary with keys appearing in names.
|
|
:type d: dict
|
|
:type names: [a]
|
|
:param pred: a filter that is applied to the values of the dictionary.
|
|
:type pred: (a -> bool)
|
|
:param key_transform: a transform to apply to the key before returning it
|
|
:type key_transform: a -> a
|
|
:param value_transform: a transform to apply to the value before
|
|
returning it
|
|
:type value_transform: a -> a
|
|
:rtype: dict
|
|
|
|
"""
|
|
return dict((key_transform(k), value_transform(v))
|
|
for k, v in six.iteritems(d)
|
|
if k in names and pred(v))
|
|
|
|
|
|
_pytype_to_cfgtype = {
|
|
six.text_type: cfg.StrOpt,
|
|
int: cfg.IntOpt,
|
|
bool: cfg.BoolOpt,
|
|
float: cfg.FloatOpt,
|
|
list: cfg.ListOpt,
|
|
dict: cfg.DictOpt
|
|
}
|
|
|
|
|
|
def dict_to_conf(options):
|
|
"""Converts a python dictionary to a list of oslo.config.cfg.Opt
|
|
|
|
:param options: The python dictionary to convert
|
|
:type options: dict
|
|
:returns: a list of options compatible with oslo.config
|
|
:rtype: [oslo.config.cfg.Opt]
|
|
"""
|
|
opts = []
|
|
for k, v in six.iteritems(options):
|
|
opt_type = _pytype_to_cfgtype[type(v)]
|
|
opts.append(opt_type(name=k, default=v))
|
|
|
|
return opts
|