x/valet
Code Issues Proposed changes

Syncrhonized plugins codebase

Browse Source
This commit is contained in:
Omar Rivera 2017-05-05 12:07:42 -05:00
parent 76ab80040a
commit d2e79829a3
49 changed files with 655 additions and 562 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
valet_plugins/.coveragerc → plugins/.coveragerc
View File

106
plugins/.gitignore vendored Normal file
View File

@ -0,0 +1,106 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
#ignore thumbnails created by windows
Thumbs.db
#Ignore files build by Visual Studio
*.obj
*.exe
*.pdb
*.user
*.aps
*.pch
*.vspscc
*_i.c
*_p.c
*.ncb
*.suo
*.tlb
*.tlh
*.bak
*.cache
*.ilk
*.log
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.eggs/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.cache
nosetests.xml
coverage.xml
.project
.pydevproject
.settings/
# Translations
*.mo
*.pot
# Django stuff:
*.log
# Sphinx documentation
docs/_build/
# PyBuilder
target/
#ignore thumbnails created by windows
Thumbs.db
#Ignore files build by Visual Studio
*.obj
*.exe
*.pdb
*.user
*.aps
*.pch
*.vspscc
*_i.c
*_p.c
*.ncb
*.suo
*.tlb
*.tlh
*.bak
*.cache
*.ilk
[Bb]in
[Dd]ebug*/
*.lib
*.sbr
obj/
[Rr]elease*/
_ReSharper*/
[Tt]est[Rr]esult*
.idea/*

4
plugins/.gitreview Normal file
View File

@ -0,0 +1,4 @@
[gerrit]
host=gerrit.mtn5.cci.att.com
port=29418
project=aic-valet-openstack-plugins.git

0
valet_plugins/.testr.conf → plugins/.testr.conf
View File

0
valet_plugins/LICENSE → plugins/LICENSE
View File

8
valet_plugins/README → plugins/README
View File

@ -1,8 +1,12 @@
Ostro version 2.0.2 Installation and Usage Guide
Author: Gueyoung Jung
Contact: gjung@research.att.com
INSTALLATION
You can download the latest Ostro python code from repository (GitHub).
You can download the latest Ostro python code from repository (CodeCloud).
USAGE
@ -49,3 +53,5 @@ Ostro will run as a daemon process. Go to “ostro_server” directory, then sta
To stop this daemon process:
python ostro_daemon.py stop

34
plugins/README.md Normal file
View File

@ -0,0 +1,34 @@
# Valet
Valet gives OpenStack the ability to optimize cloud resources while simultaneously meeting a cloud application's QoS requirements. Valet provides an api service, a placement optimizer (Ostro), a high availability data storage and persistence layer (Music), and a set of OpenStack plugins.
**IMPORTANT**: [Overall AT&T AIC Installation of Valet is covered in a separate document](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/doc/aic/README.md).
Learn more about Valet:
* [OpenStack Newton Summit Presentation](https://www.openstack.org/videos/video/valet-holistic-data-center-optimization-for-openstack) (Austin, TX, 27 April 2016)
* [Presentation Slides](http://www.research.att.com/export/sites/att_labs/techdocs/TD_101806.pdf) (PDF)
Valet consists of the following components:
* [valet-openstack](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/valet_os/README.md): a set of OpenStack plugins used to interact with Valet
* [valet-api](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/valet_api/README.md): an API engine used to interact with Valet
* [Ostro](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/ostro/atRef/refs/heads/master/renderFile/README): a placement optimization engine
* [Music](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/music/atRef/refs/heads/master/renderFile/README.md): a data storage and persistence service
* [ostro-listener](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/ostro_listener/README.md): a message bus listener used in conjunction with Ostro and Music
* [havalet](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/havalet/README.md): a service that assists in providing high availability for Valet
Additional documents:
* [OpenStack Heat Resource Plugins](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/valet_os/etc/valet_os/heat/README.md): Heat resources
* [Placement API](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/valet_api/doc/README.md): API requests/responses
* [Using Postman with valet-api](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/valet_api/valet_api/tests/README.md): Postman support
## Thank You
Alicia Abella, Saar Alaluf, Bharath Balasubramanian, Roy Ben Hai, Shimon Benattar, Yael Ben Shalom, Benny Bustan, Rachel Cohen, Joe D'Andrea, Harel Dorfman, Boaz Elitzur, P.K. Esrawan, Inbal Harduf, Matti Hiltunen, Doron Honigsberg, Kaustubh Joshi, Gueyoung Jung, Gerald Karam, David Khanin, Israel Kliger, Erez Korn, Max Osipov, Chris Rice, Amnon Sagiv, Gideon Shafran, Galit Shemesh, Anna Yefimov; AT&T Advanced Technology and Architecture, AT&T Technology Development - AIC, Additional partners in AT&T Domain 2.0. Apologies if we missed anyone (please advise via email!).
## Contact
Joe D'Andrea <jdandrea@research.att.com>

7
valet_plugins/RELEASE → plugins/RELEASE
View File

@ -12,8 +12,15 @@ Valet1.0/Ostro features
- Resource standby
When allocating resources (CPU, memory, disk, and later network bandwidth), Ostro intentionally leaves a certain percentage of resources as unused. This is because of the concern of load spikes of tenant applications. Later, we will deploy more dynamic mechanism in the future version of Ostro.
- High availability
Ostro replicas run as active-passive way. When active Ostro fails, automatically the passive one is activated via HAValet. All data is updated in MUSIC database at runtime whenever it is changed. When the passive Ostro is activated, it gets data from MUSIC to initialize its status rather than from OpenStack. Ostro also takes ping messages to show if it is alive or not.
- Runtime update via the Oslo message bus or RO
Working on this.
- Migration tip
Working on this.

2
valet_plugins/requirements.txt → plugins/requirements.txt
View File

@ -3,4 +3,4 @@
# process, which may cause wedges in the gate later.
pip
simplejson
simplejson

3
valet_plugins/setup.cfg → plugins/setup.cfg
View File

@ -5,7 +5,7 @@ summary = Valet Orchestration Plugins for OpenStack
description-file = README.md
author = AT&T
author-email = jdandrea@research.att.com
homepage = https://github.com/att-comdev/valet
homepage = https://codecloud.web.att.com/projects/ST_CLOUDQOS/
classifier =
Environment :: OpenStack
Intended Audience :: Information Technology
@ -31,3 +31,4 @@ data_files =
# ValetFilter = valet_os.cinder.valet_filter:ValetFilter
heat.stack_lifecycle_plugins =
valet.lifecycle_plugin = valet_plugins.plugins.heat.plugins:ValetLifecyclePlugin

32
plugins/setup.py Normal file
View File

@ -0,0 +1,32 @@
# -*- encoding: utf-8 -*-
#
# Copyright (c) 2014-2016 AT&T
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing permissions and
# limitations under the License.
'''Setup'''
import setuptools
# In python < 2.7.4, a lazy loading of package `pbr` will break
# setuptools if some other modules registered functions in `atexit`.
# solution from: http://bugs.python.org/issue15881#msg170215
try:
import multiprocessing # noqa # pylint: disable=W0611,C0411
except ImportError:
pass
setuptools.setup(
setup_requires=['pbr'],
pbr=True)

0
valet_plugins/test-requirements.txt → plugins/test-requirements.txt
View File

4
valet_plugins/tox.ini → plugins/tox.ini
View File

@ -1,7 +1,6 @@
[tox]
#minversion = 2.0
envlist = pep8
#py27
envlist = py27
#py27-constraints, pep8-constraints
#py34-constraints,py27-constraints,pypy-constraints,pep8-constraints
#skipsdist = True
@ -53,3 +52,4 @@ show-source = True
ignore = E123,E125,E501,H401,H501,H301
builtins = _
exclude=.venv,.git,.tox,dist,doc,*lib/python*,*egg,build,*egg-info

0
valet_plugins/valet_plugins/__init__.py → plugins/valet_plugins/__init__.py
View File

0
valet_plugins/valet_plugins/common/__init__.py → plugins/valet_plugins/common/__init__.py
View File

129
valet_plugins/valet_plugins/common/valet_api.py → plugins/valet_plugins/common/valet_api.py
View File

@ -1,19 +1,20 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2014-2017 AT&T Intellectual Property
# Copyright (c) 2014-2016 AT&T
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing permissions and
# limitations under the License.
"""Valet API Wrapper."""
'''Valet API Wrapper'''
from heat.common.i18n import _
import json
@ -29,45 +30,44 @@ LOG = logging.getLogger(__name__)
def _exception(exc, exc_info, req):
"""Handle an exception."""
'''Handle an exception'''
response = None
try:
response = json.loads(req.text)
if req is not None:
response = json.loads(req.text)
except Exception as e:
# FIXME(GJ): if Valet returns error
LOG.error("Exception is: %s, body is: %s" % (e, req.text))
return
return None
if 'error' in response:
if response and 'error' in response:
# FIXME(GJ): if Valet returns error
error = response.get('error')
msg = "%(explanation)s (valet-api: %(message)s)" % {
'explanation': response.get('explanation',
_('No remediation available')),
'explanation': response.get('explanation', _('No remediation available')),
'message': error.get('message', _('Unknown error'))
}
raise ValetAPIError(msg)
LOG.error("Response with error: " + msg)
return "error"
else:
# TODO(JD): Re-evaluate if this clause is necessary.
exc_class, exc, traceback = exc_info # pylint: disable=W0612
msg = _("%(exc)s for %(method)s %(url)s with body %(body)s") %\
{'exc': exc, 'method': exc.request.method, 'url': exc.request.url,
'body': exc.request.body}
my_exc = ValetAPIError(msg)
# traceback can be added to the end of the raise
raise my_exc.__class__, my_exc
msg = _("%(exc)s for %(method)s %(url)s with body %(body)s") % {'exc': exc, 'method': exc.request.method, 'url': exc.request.url, 'body': exc.request.body}
LOG.error("Response is none: " + msg)
return "error"
# TODO(UNKNOWN): Improve exception reporting back up to heat
# TODO(JD): Improve exception reporting back up to heat
class ValetAPIError(Exception):
"""Valet API Error."""
'''Valet API Error'''
pass
class ValetAPIWrapper(object):
"""Valet API Wrapper."""
'''Valet API Wrapper'''
def __init__(self):
"""Initializer."""
'''Initializer'''
self.headers = {'Content-Type': 'application/json'}
self.opt_group_str = 'valet'
self.opt_name_str = 'url'
@ -76,7 +76,7 @@ class ValetAPIWrapper(object):
self._register_opts()
def _api_endpoint(self):
"""Return API endpoint."""
'''Returns API endpoint'''
try:
opt = getattr(cfg.CONF, self.opt_group_str)
endpoint = opt[self.opt_name_str]
@ -89,80 +89,75 @@ class ValetAPIWrapper(object):
raise # exception.Error(_('API Endpoint not defined.'))
def _get_timeout(self):
"""Get timeout.
Return Valet plugin API request timeout
tuple (conn_timeout, read_timeout).
"""
conn_timeout = 3
read_timeout = 5
'''Returns Valet plugin API request timeout tuple (conn_timeout, read_timeout)'''
read_timeout = 600
try:
opt = getattr(cfg.CONF, self.opt_group_str)
conn_timeout = opt[self.opt_conn_timeout]
# conn_timeout = opt[self.opt_conn_timeout]
read_timeout = opt[self.opt_read_timeout]
except Exception:
pass
return conn_timeout, read_timeout
# Timeout accepts tupple on 'requests' version 2.4.0 and above - adding *connect* timeouts
# return conn_timeout, read_timeout
return read_timeout
def _register_opts(self):
"""Register options."""
'''Register options'''
opts = []
option = cfg.StrOpt(self.opt_name_str, default=None,
help=_('Valet API endpoint'))
option = cfg.StrOpt(self.opt_name_str, default=None, help=_('Valet API endpoint'))
opts.append(option)
option = cfg.IntOpt(self.opt_conn_timeout, default=3,
help=_('Valet Plugin Connect Timeout'))
option = cfg.IntOpt(self.opt_conn_timeout, default=3, help=_('Valet Plugin Connect Timeout'))
opts.append(option)
option = cfg.IntOpt(self.opt_read_timeout, default=5,
help=_('Valet Plugin Read Timeout'))
option = cfg.IntOpt(self.opt_read_timeout, default=5, help=_('Valet Plugin Read Timeout'))
opts.append(option)
opt_group = cfg.OptGroup(self.opt_group_str)
cfg.CONF.register_group(opt_group)
cfg.CONF.register_opts(opts, group=opt_group)
# TODO(UNKOWN): Keep stack param for now. We may need it again.
# TODO(JD): Keep stack param for now. We may need it again.
def plans_create(self, stack, plan, auth_token=None): # pylint: disable=W0613
"""Create a plan."""
'''Create a plan'''
response = None
try:
req = None
timeout = self._get_timeout()
url = self._api_endpoint() + '/plans/'
payload = json.dumps(plan)
self.headers['X-Auth-Token'] = auth_token
req = requests.post(url, data=payload, headers=self.headers,
timeout=timeout)
req = requests.post(url, data=payload, headers=self.headers, timeout=timeout)
req.raise_for_status()
response = json.loads(req.text)
except (requests.exceptions.HTTPError,
requests.exceptions.ConnectTimeout,
requests.exceptions.ConnectionError) \
except (requests.exceptions.HTTPError, requests.exceptions.Timeout, requests.exceptions.ConnectionError)\
as exc:
_exception(exc, sys.exc_info(), req)
return _exception(exc, sys.exc_info(), req)
except Exception as e:
LOG.error("Exception (at plans_create) is: %s" % e)
return None
return response
# TODO(UNKNOWN): Keep stack param for now. We may need it again.
# TODO(JD): Keep stack param for now. We may need it again.
def plans_delete(self, stack, auth_token=None): # pylint: disable=W0613
"""Delete a plan."""
'''Delete a plan'''
try:
req = None
timeout = self._get_timeout()
url = self._api_endpoint() + '/plans/' + stack.id
self.headers['X-Auth-Token'] = auth_token
req = requests.delete(url, headers=self.headers, timeout=timeout)
except (requests.exceptions.HTTPError,
requests.exceptions.ConnectTimeout,
requests.exceptions.ConnectionError)\
except (requests.exceptions.HTTPError, requests.exceptions.Timeout, requests.exceptions.ConnectionError)\
as exc:
_exception(exc, sys.exc_info(), req)
return _exception(exc, sys.exc_info(), req)
except Exception as e:
LOG.error("Exception (plans_delete) is: %s" % e)
return None
# Delete does not return a response body.
def placement(self, orch_id, res_id, hosts=None, auth_token=None):
"""Reserve previously made placement."""
'''Reserve previously made placement.'''
try:
req = None
payload = None
timeout = self._get_timeout()
url = self._api_endpoint() + '/placements/' + orch_id
self.headers['X-Auth-Token'] = auth_token
@ -172,15 +167,19 @@ class ValetAPIWrapper(object):
"resource_id": res_id
}
payload = json.dumps(kwargs)
req = requests.post(url, data=payload, headers=self.headers,
timeout=timeout)
req = requests.post(url, data=payload, headers=self.headers, timeout=timeout)
else:
req = requests.get(url, headers=self.headers, timeout=timeout)
# TODO(UNKNOWN): Raise an exception IFF the scheduler can handle it
# TODO(JD): Raise an exception IFF the scheduler can handle it
# req.raise_for_status()
response = json.loads(req.text)
except Exception: # pylint: disable=W0702
except (requests.exceptions.HTTPError, requests.exceptions.Timeout, requests.exceptions.ConnectionError)\
as exc:
return _exception(exc, sys.exc_info(), req)
except Exception as e: # pylint: disable=W0702
LOG.error("Exception (placement) is: %s" % e)
# FIXME: Find which exceptions we should really handle here.
response = None

55
valet_plugins/valet_plugins/heat/GroupAssignment.py → plugins/valet_plugins/heat/GroupAssignment.py
View File

@ -1,19 +1,20 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2014-2017 AT&T Intellectual Property
# Copyright (c) 2014-2016 AT&T
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing permissions and
# limitations under the License.
"""GroupAssignment Heat Resource Plugin."""
'''GroupAssignment Heat Resource Plugin'''
from heat.common.i18n import _
from heat.engine import constraints
@ -26,20 +27,15 @@ LOG = logging.getLogger(__name__)
class GroupAssignment(resource.Resource):
"""Group Assignment.
''' A Group Assignment describes one or more resources assigned to a particular type of group.
Group Assignment describes one or more resources assigned to a
particular type of group.
Assignments can reference other assignments, so long as there are no circular references.
There are three types of groups: affinity, diversity, and exclusivity.
Exclusivity groups have a unique name, assigned through Valet.
Assignments can reference other assignments, so long as there are no
circular references. There are three types of groups: affinity, diversity,
and exclusivity. Exclusivity groups have a unique name, assigned through
Valet.
This resource is purely informational in nature and makes no changes to
heat, nova, or cinder. The Valet Heat Lifecycle Plugin passes this
information to the optimizer.
"""
This resource is purely informational in nature and makes no changes to heat, nova, or cinder.
The Valet Heat Lifecycle Plugin passes this information to the optimizer.
'''
_RELATIONSHIP_TYPES = (
AFFINITY, DIVERSITY, EXCLUSIVITY,
@ -57,7 +53,7 @@ class GroupAssignment(resource.Resource):
GROUP_NAME: properties.Schema(
properties.Schema.STRING,
_('Group name. Required for exclusivity groups.'),
# TODO(UNKNOWN): Add a custom constraint
# TODO(JD): Add a custom constraint
# Constraint must ensure a valid and allowed name
# when an exclusivity group is in use.
# This is presently enforced by valet-api and can also
@ -92,19 +88,18 @@ class GroupAssignment(resource.Resource):
}
def handle_create(self):
"""Create resource."""
'''Create resource'''
self.resource_id_set(self.physical_resource_name())
def handle_update(self, json_snippet, templ_diff, # pylint: disable=W0613
prop_diff):
"""Update resource."""
def handle_update(self, json_snippet, templ_diff, prop_diff): # pylint: disable=W0613
'''Update resource'''
self.resource_id_set(self.physical_resource_name())
def handle_delete(self):
"""Delete resource."""
'''Delete resource'''
self.resource_id_set(None)
def resource_mapping():
"""Map names to resources."""
'''Map names to resources.'''
return {'ATT::Valet::GroupAssignment': GroupAssignment, }

2
valet_plugins/valet_plugins/heat/README.md → plugins/valet_plugins/heat/README.md
View File

@ -1,6 +1,6 @@
# OpenStack Heat Resource Plugins
[Valet](https://github.com/att-comdev/valet/blob/master/README.md) works with OpenStack Heat through the use of Resource Plugins. This document explains what they are and how they work. As new plugins become formally introduced, they will be added here.
[Valet](https://codecloud.web.att.com/plugins/servlet/readmeparser/display/ST_CLOUDQOS/allegro/atRef/refs/heads/master/renderFile/README.md) works with OpenStack Heat through the use of Resource Plugins. This document explains what they are and how they work. As new plugins become formally introduced, they will be added here.
The following is current as of Valet Release 1.0.

0
valet_plugins/valet_plugins/heat/__init__.py → plugins/valet_plugins/heat/__init__.py
View File

0
valet_plugins/valet_plugins/plugins/__init__.py → plugins/valet_plugins/plugins/__init__.py
View File

0
valet_plugins/valet_plugins/plugins/heat/__init__.py → plugins/valet_plugins/plugins/heat/__init__.py
View File

58
valet_plugins/valet_plugins/plugins/heat/plugins.py → plugins/valet_plugins/plugins/heat/plugins.py
View File

@ -1,19 +1,20 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2014-2017 AT&T Intellectual Property
# Copyright (c) 2014-2016 AT&T
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing permissions and
# limitations under the License.
"""Valet Plugins for Heat."""
'''Valet Plugins for Heat'''
from heat.engine import lifecycle_plugin
@ -30,13 +31,13 @@ LOG = logging.getLogger(__name__)
def validate_uuid4(uuid_string):
"""Validate that a UUID string is in fact a valid uuid4.
''' Validate that a UUID string is in fact a valid uuid4.
Happily, the uuid module does the actual checking for us.
It is vital that the 'version' kwarg be passed to the
UUID() call, otherwise any 32-character hex string
is considered valid.
"""
'''
try:
val = uuid.UUID(uuid_string, version=4)
except ValueError:
@ -54,13 +55,11 @@ def validate_uuid4(uuid_string):
class ValetLifecyclePlugin(lifecycle_plugin.LifecyclePlugin):
"""Base class for pre-op and post-op work on a stack.
''' Base class for pre-op and post-op work on a stack.
Implementations should extend this class and override the methods.
"""
'''
def __init__(self):
"""Initialize."""
self.api = valet_api.ValetAPIWrapper()
self.hints_enabled = False
@ -69,11 +68,10 @@ class ValetLifecyclePlugin(lifecycle_plugin.LifecyclePlugin):
self.hints_enabled = cfg.CONF.stack_scheduler_hints
def _parse_stack_preview(self, dest, preview):
"""Walk the preview list (possibly nested).
''' Walk the preview list (possibly nested)
extracting parsed template dicts and storing modified versions in a flat
dict.
"""
extracting parsed template dicts and storing modified versions in a flat dict.
'''
# The preview is either a list or not.
if not isinstance(preview, list):
# Heat does not assign orchestration UUIDs to
@ -88,21 +86,21 @@ class ValetLifecyclePlugin(lifecycle_plugin.LifecyclePlugin):
preview.uuid and validate_uuid4(preview.uuid):
key = preview.uuid
else:
# TODO(UNK): Heat should be authoritative for UUID assignments.
# TODO(JD): Heat should be authoritative for UUID assignments.
# This will require a change to heat-engine.
# Looks like it may be: heat/db/sqlalchemy/models.py#L279
# It could be that nested stacks aren't added to the DB yet.
key = str(uuid.uuid4())
parsed = preview.parsed_template()
parsed['name'] = preview.name
# TODO(UNKWN): Replace resource referenced names with their UUIDs.
# TODO(JD): Replace resource referenced names with their UUIDs.
dest[key] = parsed
else:
for item in preview:
self._parse_stack_preview(dest, item)
def do_pre_op(self, cnxt, stack, current_stack=None, action=None):
"""Method to be run by heat before stack operations."""
''' Method to be run by heat before stack operations. '''
if not self.hints_enabled or stack.status != 'IN_PROGRESS':
return
@ -133,19 +131,19 @@ class ValetLifecyclePlugin(lifecycle_plugin.LifecyclePlugin):
self.api.plans_create(stack, plan, auth_token=cnxt.auth_token)
def do_post_op(self, cnxt, stack, # pylint: disable=R0913
current_stack=None, action=None, is_stack_failure=False):
"""Method to be run by heat after stack operations, including failures.
def do_post_op(self, cnxt, stack, current_stack=None, action=None, # pylint: disable=R0913
is_stack_failure=False):
''' Method to be run by heat after stack operations, including failures.
On failure to execute all the registered pre_ops, this method will be
called if and only if the corresponding pre_op was successfully called.
On failures of the actual stack operation, this method will
be called if all the pre operations were successfully called.
"""
'''
pass
def get_ordinal(self):
"""Ordinal to order class instances for pre /post operation execution.
''' An ordinal used to order class instances for pre and post operation execution.
The values returned by get_ordinal are used to create a partial order
for pre and post operation method invocations. The default ordinal
@ -156,5 +154,5 @@ class ValetLifecyclePlugin(lifecycle_plugin.LifecyclePlugin):
class1inst will be executed after the method on class2inst.
If class1inst.ordinal() == class2inst.ordinal(), then the order of
method invocation is indeterminate.
"""
'''
return 100

0
valet_plugins/valet_plugins/plugins/nova/__init__.py → plugins/valet_plugins/plugins/nova/__init__.py
View File

161
valet_plugins/valet_plugins/plugins/nova/valet_filter.py → plugins/valet_plugins/plugins/nova/valet_filter.py
View File

@ -1,19 +1,20 @@
# -*- encoding: utf-8 -*-
#
# Copyright 2014-2017 AT&T Intellectual Property
# Copyright (c) 2014-2016 AT&T
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing permissions and
# limitations under the License.
"""Valet Nova Scheduler Filter."""
'''Valet Nova Scheduler Filter'''
from keystoneclient.v2_0 import client
@ -26,12 +27,14 @@ from valet_plugins.common import valet_api
from oslo_config import cfg
from oslo_log import log as logging
import time
CONF = cfg.CONF
LOG = logging.getLogger(__name__)
class ValetFilter(filters.BaseHostFilter):
"""Filter on Valet assignment."""
'''Filter on Valet assignment.'''
# Host state does not change within a request
run_filter_once_per_request = True
@ -40,7 +43,7 @@ class ValetFilter(filters.BaseHostFilter):
_auth_token = None
def __init__(self):
"""Initializer."""
'''Initializer'''
self.api = valet_api.ValetAPIWrapper()
self.opt_group_str = 'valet'
self.opt_failure_mode_str = 'failure_mode'
@ -50,8 +53,11 @@ class ValetFilter(filters.BaseHostFilter):
self.opt_auth_uri_str = 'admin_auth_url'
self._register_opts()
self.retries = 60
self.interval = 1
def _authorize(self):
"""Keystone AuthN."""
'''Keystone AuthN'''
opt = getattr(cfg.CONF, self.opt_group_str)
project_name = opt[self.opt_project_name_str]
username = opt[self.opt_username_str]
@ -68,61 +74,68 @@ class ValetFilter(filters.BaseHostFilter):
self._auth_token = keystone_client.auth_token
def _is_same_host(self, host, location): # pylint: disable=R0201
"""Return true if host matches location."""
'''Returns true if host matches location'''
return host == location
def _register_opts(self):
"""Register Options."""
'''Register Options'''
opts = []
option = cfg.StrOpt(
self.opt_failure_mode_str,
choices=['reject', 'yield'],
default='reject',
help=_('Mode to operate in if Valet planning fails for any reason.'))
option = cfg.StrOpt(self.opt_failure_mode_str, choices=['reject', 'yield'], default='reject',
help=_('Mode to operate in if Valet planning fails for any reason.'))
opts.append(option)
option = cfg.StrOpt(self.opt_project_name_str, default=None,
help=_('Valet Project Name'))
option = cfg.StrOpt(self.opt_project_name_str, default=None, help=_('Valet Project Name'))
opts.append(option)
option = cfg.StrOpt(self.opt_username_str, default=None,
help=_('Valet Username'))
option = cfg.StrOpt(self.opt_username_str, default=None, help=_('Valet Username'))
opts.append(option)
option = cfg.StrOpt(self.opt_password_str, default=None,
help=_('Valet Password'))
option = cfg.StrOpt(self.opt_password_str, default=None, help=_('Valet Password'))
opts.append(option)
option = cfg.StrOpt(self.opt_auth_uri_str, default=None,
help=_('Keystone Authorization API Endpoint'))
option = cfg.StrOpt(self.opt_auth_uri_str, default=None, help=_('Keystone Authorization API Endpoint'))
opts.append(option)
opt_group = cfg.OptGroup(self.opt_group_str)
cfg.CONF.register_group(opt_group)
cfg.CONF.register_opts(opts, group=opt_group)
# TODO(UNKNOWN): Factor out common code between this and the cinder filter
# TODO(JD): Factor out common code between this and the cinder filter
def filter_all(self, filter_obj_list, filter_properties):
"""Filter all hosts in one swell foop."""
'''Filter all hosts in one swell foop'''
hints_key = 'scheduler_hints'
orch_id_key = 'heat_resource_uuid'
ad_hoc = False
yield_all = False
location = None
res_id = None
opt = getattr(cfg.CONF, self.opt_group_str)
failure_mode = opt[self.opt_failure_mode_str]
# Get the resource_id (physical id)
# Get the resource_id (physical id) and host candidates
request_spec = filter_properties.get('request_spec')
instance_properties = request_spec.get('instance_properties')
res_id = instance_properties.get('uuid')
hosts = [obj.host for obj in filter_obj_list]
# TODO(JD): If we can't reach Valet at all, we may opt to fail
# TODO(JD): all hosts depending on a TBD config flag.
if orch_id_key not in filter_properties.get(hints_key, {}):
# Max
# fix for invaid authorization in yield mode
failed = None
try:
self._authorize()
LOG.warn(_LW("Valet: Heat Stack Lifecycle Scheduler Hints not "
"found. Performing ad-hoc placement."))
except Exception as ex:
failed = ex
if failed:
LOG.warn("Failed to filter the hosts, failure mode is %s" % failure_mode)
if failure_mode == 'yield':
LOG.warn(failed)
yield_all = True
else:
LOG.error(failed)
# if not filter_properties.get(hints_key, {}).has_key(orch_id_key):
elif orch_id_key not in filter_properties.get(hints_key, {}):
LOG.info(_LW("Valet: Heat Stack Lifecycle Scheduler Hints not found. Performing ad-hoc placement."))
ad_hoc = True
# We'll need the flavor.
@ -153,17 +166,26 @@ class ValetFilter(filters.BaseHostFilter):
plan = {
'plan_name': res_id,
'stack_id': res_id,
'locations': hosts,
'timeout': '%d sec' % timeout,
'resources': resources
}
try:
response = self.api.plans_create(None, plan,
auth_token=self._auth_token)
except Exception:
# TODO(UNKNOWN): Get context from exception
LOG.error(_LE("Valet did not respond to ad hoc placement "
"request."))
response = None
count = 0
response = None
while count < self.retries:
try:
response = self.api.plans_create(None, plan, auth_token=self._auth_token)
except Exception:
# TODO(JD): Get context from exception
LOG.error(_LE("Raise exception for ad hoc placement request."))
response = None
if response is None:
count += 1
LOG.warn("Valet conn error for plan_create, Retry " + str(count) + " for stack = " + res_id)
time.sleep(self.interval)
else:
break
if response and response.get('plan'):
plan = response['plan']
@ -174,40 +196,41 @@ class ValetFilter(filters.BaseHostFilter):
location = placement['location']
if not location:
LOG.error(_LE("Valet ad-hoc placement unknown for resource id "
"%s.") % res_id)
LOG.error(_LE("Valet ad-hoc placement unknown for resource id %s.") % res_id)
if failure_mode == 'yield':
LOG.warn(_LW("Valet will yield to Nova for placement "
"decisions."))
LOG.warn(_LW("Valet will yield to Nova for placement decisions."))
yield_all = True
else:
yield_all = False
else:
orch_id = filter_properties[hints_key][orch_id_key]
self._authorize()
hosts = [obj.host for obj in filter_obj_list]
try:
response = self.api.placement(orch_id, res_id, hosts=hosts,
auth_token=self._auth_token)
except Exception:
print("Exception in creating placement")
LOG.error(_LW("Valet did not respond to placement request."))
response = None
count = 0
response = None
while count < self.retries:
try:
response = self.api.placement(orch_id, res_id, hosts=hosts, auth_token=self._auth_token)
except Exception:
LOG.error(_LW("Raise exception for placement request."))
response = None
if response is None:
count += 1
LOG.warn("Valet conn error for placement Retry " + str(count) + " for stack = " + orch_id)
time.sleep(self.interval)
else:
break
if response and response.get('placement'):
placement = response['placement']
if placement.get('location'):
location = placement['location']
if not location:
# TODO(UNKNOWN): Get context from exception
LOG.error(_LE("Valet placement unknown for resource id {0},"
"orchestration id {1}.").format(res_id, orch_id))
# TODO(JD): Get context from exception
LOG.error(_LE("Valet placement unknown for resource id {0}, orchestration id {1}.").format(res_id, orch_id))
if failure_mode == 'yield':
LOG.warn(_LW("Valet will yield to Nova for placement"
"decisions."))
LOG.warn(_LW("Valet will yield to Nova for placement decisions."))
yield_all = True
else:
yield_all = False
@ -220,19 +243,15 @@ class ValetFilter(filters.BaseHostFilter):
match = self._is_same_host(obj.host, location)
if match:
if ad_hoc:
LOG.info(_LI("Valet ad-hoc placement for resource "
"id {0}: {1}.").format(res_id, obj.host))
LOG.info(_LI("Valet ad-hoc placement for resource id {0}: {1}.").format(res_id, obj.host))
else:
LOG.info(_LI("Valet placement for resource id %s, "
"orchestration id {0}: {1}.").format(
res_id, orch_id, obj.host))
LOG.info(_LI("Valet placement for resource id %s, orchestration id {0}: {1}.").format(res_id, orch_id, obj.host))
else:
match = None
if yield_all or match:
yield obj
def host_passes(self, host_state, # pylint: disable=W0613,R0201
filter_properties):
"""Individual host pass check."""
def host_passes(self, host_state, filter_properties): # pylint: disable=W0613,R0201
'''Individual host pass check'''
# Intentionally let filter_all() handle in one swell foop.
return False

0
valet_plugins/valet_plugins/tests/__init__.py → plugins/valet_plugins/tests/__init__.py
View File

49
plugins/valet_plugins/tests/base.py Normal file
View File

@ -0,0 +1,49 @@
# -*- encoding: utf-8 -*-
#
# Copyright (c) 2014-2016 AT&T
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from oslo_config import fixture as fixture_config
from oslo_log import log as logging
from oslotest.base import BaseTestCase
LOG = logging.getLogger(__name__)
class Base(BaseTestCase):
"""Test case base class for all unit tests."""
def __init__(self, *args, **kwds):
''' '''
super(Base, self).__init__(*args, **kwds)
self.CONF = self.useFixture(fixture_config.Config()).conf
def setUp(self):
super(Base, self).setUp()
def run_test(self, stack_name, template_path):
''' main function '''
pass
def validate(self, result):
self.assertEqual(True, result.ok, result.message)
def validate_test(self, result):
self.assertTrue(result)
def get_name(self):
pass

0
valet_plugins/valet_plugins/tests/unit/__init__.py → plugins/valet_plugins/tests/unit/__init__.py
View File

0
valet_plugins/valet_plugins/tests/unit/mocks/heat/__init__.py → plugins/valet_plugins/tests/unit/mocks/heat/__init__.py
View File

0
valet_plugins/valet_plugins/tests/unit/mocks/heat/common/__init__.py → plugins/valet_plugins/tests/unit/mocks/heat/common/__init__.py
View File

7
plugins/valet_plugins/tests/unit/mocks/heat/common/i18n.py Normal file
View File

@ -0,0 +1,7 @@
'''
Created on Sep 14, 2016
@author: stack
'''
_ = None

0
valet_plugins/valet_plugins/tests/unit/mocks/heat/engine/__init__.py → plugins/valet_plugins/tests/unit/mocks/heat/engine/__init__.py
View File

25
plugins/valet_plugins/tests/unit/mocks/heat/engine/lifecycle_plugin.py Normal file
View File

@ -0,0 +1,25 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
'''
Created on Sep 14, 2016
@author: stack
'''
class LifecyclePlugin(object):
''' classdocs '''
def __init__(self, params):
''' Constructor '''

0
valet_plugins/valet_plugins/tests/unit/mocks/nova/__init__.py → plugins/valet_plugins/tests/unit/mocks/nova/__init__.py
View File

33
plugins/valet_plugins/tests/unit/mocks/nova/i18n.py Normal file
View File

@ -0,0 +1,33 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
'''
Created on Sep 14, 2016
@author: stack
'''
def _(string):
pass
def _LI(string):
pass
def _LW(string):
pass
def _LE(string):
return string

0
valet_plugins/valet_plugins/tests/unit/mocks/nova/scheduler/__init__.py → plugins/valet_plugins/tests/unit/mocks/nova/scheduler/__init__.py
View File

24
plugins/valet_plugins/tests/unit/mocks/nova/scheduler/filters.py Normal file
View File

@ -0,0 +1,24 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
'''
Created on Sep 15, 2016
@author: stack
'''
class BaseHostFilter(object):
''' classdocs '''
def __init__(self, params):
''' Constructor '''

32
valet_plugins/valet_plugins/tests/unit/test_plugins.py → plugins/valet_plugins/tests/unit/test_plugins.py
View File

@ -1,19 +1,15 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Test Plugins."""
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import mock
from valet_plugins.plugins.heat.plugins import ValetLifecyclePlugin
@ -21,22 +17,18 @@ from valet_plugins.tests.base import Base
class TestPlugins(Base):
"""Test valet_plugins.plugins.heat.plugins ValetLifecyclePlugin."""
def setUp(self):
"""Setup Test Plugins and call ValetLifecyclePlugin Init."""
super(TestPlugins, self).setUp()
self.valet_life_cycle_plugin = self.init_ValetLifecyclePlugin()
@mock.patch('valet_plugins.common.valet_api.ValetAPIWrapper')
def init_ValetLifecyclePlugin(self, mock_class):
"""Called by setup to init, return ValetLifecyclePlugin()."""
with mock.patch('oslo_config.cfg.CONF'):
return ValetLifecyclePlugin()
def test_do_pre_op(self):
"""Validate life cycle pre_ops by checking api method calls."""
stack = mock.MagicMock()
stack.status = "IN_PROGRESS"
@ -58,10 +50,8 @@ class TestPlugins(Base):
self.valet_life_cycle_plugin.hints_enabled = True
stack.status = "IN_PROGRESS"
self.valet_life_cycle_plugin.do_pre_op(cnxt, stack, action="DELETE")
self.validate_test("plans_delete" in
self.valet_life_cycle_plugin.api.method_calls[0])
self.validate_test("plans_delete" in self.valet_life_cycle_plugin.api.method_calls[0])
# action create
self.valet_life_cycle_plugin.do_pre_op(cnxt, stack, action="CREATE")
self.validate_test("plans_create" in
self.valet_life_cycle_plugin.api.method_calls[1])
self.validate_test("plans_create" in self.valet_life_cycle_plugin.api.method_calls[1])

32
plugins/valet_plugins/tests/unit/test_valet_api.py Normal file
View File

@ -0,0 +1,32 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import mock
from valet_plugins.tests.base import Base
from valet_plugins.common.valet_api import ValetAPIWrapper, requests
class TestValetApi(Base):
def setUp(self):
super(TestValetApi, self).setUp()
self.valet_api_wrapper = self.init_ValetAPIWrapper()
@mock.patch.object(ValetAPIWrapper, "_register_opts")
def init_ValetAPIWrapper(self, mock_api):
mock_api.return_value = None
return ValetAPIWrapper()
@mock.patch.object(requests, 'request')
def test_plans_create(self, mock_request):
mock_request.post.return_value = None

71
plugins/valet_plugins/tests/unit/test_valet_filter.py Normal file
View File

@ -0,0 +1,71 @@
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from keystoneclient.v2_0 import client
import mock
from valet_plugins.common import valet_api
from valet_plugins.plugins.nova.valet_filter import ValetFilter
from valet_plugins.tests.base import Base
class TestResources(object):
def __init__(self, host_name):
self.host = host_name
class TestValetFilter(Base):
def setUp(self):
super(TestValetFilter, self).setUp()
client.Client = mock.MagicMock()
self.valet_filter = self.init_ValetFilter()
@mock.patch.object(valet_api.ValetAPIWrapper, '_register_opts')
@mock.patch.object(ValetFilter, '_register_opts')
def init_ValetFilter(self, mock_opt, mock_init):
mock_init.return_value = None
mock_opt.return_value = None
return ValetFilter()
@mock.patch.object(valet_api.ValetAPIWrapper, 'plans_create')
@mock.patch.object(valet_api.ValetAPIWrapper, 'placement')
def test_filter_all(self, mock_placement, mock_create):
mock_placement.return_value = None
mock_create.return_value = None
with mock.patch('oslo_config.cfg.CONF') as config:
setattr(config, "valet", {self.valet_filter.opt_failure_mode_str: "yield",
self.valet_filter.opt_project_name_str: "test_admin_tenant_name",
self.valet_filter.opt_username_str: "test_admin_username",
self.valet_filter.opt_password_str: "test_admin_password",
self.valet_filter.opt_auth_uri_str: "test_admin_auth_url"})
filter_properties = {'request_spec': {'instance_properties': {'uuid': ""}},
'scheduler_hints': {'heat_resource_uuid': "123456"},
'instance_type': {'name': "instance_name"}}
resources = self.valet_filter.filter_all([TestResources("first_host"), TestResources("second_host")], filter_properties)
for resource in resources:
self.validate_test(resource.host in "first_host, second_host")
self.validate_test(mock_placement.called)
filter_properties = {'request_spec': {'instance_properties': {'uuid': ""}},
'scheduler_hints': "scheduler_hints",
'instance_type': {'name': "instance_name"}}
resources = self.valet_filter.filter_all([TestResources("first_host"), TestResources("second_host")], filter_properties)
for _ in resources:
self.validate_test(mock_create.called)

24
valet_plugins/README.md
View File

@ -1,24 +0,0 @@
# Valet
Valet gives OpenStack the ability to optimize cloud resources while simultaneously meeting a cloud application's QoS requirements. Valet provides an api service, a placement optimizer (Ostro), a high availability data storage and persistence layer (Music), and a set of OpenStack plugins.
**IMPORTANT**: [Overall Installation of Valet is covered in a separate document](https://github.com/att-comdev/valet/blob/master/doc/valet_os.md).
Learn more about Valet:
* [OpenStack Newton Summit Presentation](https://www.openstack.org/videos/video/valet-holistic-data-center-optimization-for-openstack) (Austin, TX, 27 April 2016)
* [Presentation Slides](http://www.research.att.com/export/sites/att_labs/techdocs/TD_101806.pdf) (PDF)
Valet consists of the following components:
* [valet-openstack](https://github.com/att-comdev/valet/blob/master/doc/valet_os.md): a set of OpenStack plugins used to interact with Valet
* [valet-api](https://github.com/att-comdev/valet/blob/master/doc/valet_api.md): an API engine used to interact with Valet
* [Ostro](https://github.com/att-comdev/valet/blob/master/doc/ostro.md): a placement optimization engine
* [Music](https://github.com/att/music): a data storage and persistence service
* [ostro-listener](https://github.com/att-comdev/valet/blob/master/doc/ostro_listener.md): a message bus listener used in conjunction with Ostro and Music
Additional documents:
* [OpenStack Heat Resource Plugins](https://github.com/att-comdev/valet/blob/master/valet_plugins/valet_plugins/heat/README.md): Heat resources
* [Placement API](https://github.com/att-comdev/valet/blob/master/doc/valet_api.md): API requests/responses
* [Using Postman with valet-api](https://github.com/att-comdev/valet/blob/master/valet/tests/api/README.md): Postman support

30
valet_plugins/setup.py
View File

@ -1,30 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Setup."""
import setuptools
# In python < 2.7.4, a lazy loading of package `pbr` will break
# setuptools if some other modules registered functions in `atexit`.
# solution from: http://bugs.python.org/issue15881#msg170215
try:
import multiprocessing # noqa # pylint: disable=W0611,C0411
except ImportError:
pass
setuptools.setup(
setup_requires=['pbr'],
pbr=True)

4
valet_plugins/valet_plugins/PKG-INFO
View File

@ -1,4 +0,0 @@
Metadata-Version: 1.2
Name: valet_plugins
Version: 0.1.0
Author-email: jdandrea@research.att.com

53
valet_plugins/valet_plugins/tests/base.py
View File

@ -1,53 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Tests Base."""
from oslo_config import fixture as fixture_config
from oslo_log import log as logging
from oslotest.base import BaseTestCase
LOG = logging.getLogger(__name__)
class Base(BaseTestCase):
"""Test case base class for all unit tests."""
def __init__(self, *args, **kwds):
"""Initialize."""
super(Base, self).__init__(*args, **kwds)
self.CONF = self.useFixture(fixture_config.Config()).conf
def setUp(self):
"""Setup."""
super(Base, self).setUp()
def run_test(self, stack_name, template_path):
"""Main function."""
pass
def validate(self, result):
"""Validate."""
self.assertEqual(True, result.ok, result.message)
def validate_test(self, result):
"""Validate Test."""
self.assertTrue(result)
def get_name(self):
"""Get Name."""
pass

19
valet_plugins/valet_plugins/tests/unit/mocks/heat/common/i18n.py
View File

@ -1,19 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""i18n."""
_ = None

23
valet_plugins/valet_plugins/tests/unit/mocks/heat/engine/lifecycle_plugin.py
View File

@ -1,23 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Lifecycle Plugin."""
class LifecyclePlugin(object):
"""Classdoc."""
def __init__(self, params):
"""Constructor."""

32
valet_plugins/valet_plugins/tests/unit/mocks/nova/i18n.py
View File

@ -1,32 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""i18n."""
def _(string):
pass
def _LI(string):
pass
def _LW(string):
pass
def _LE(string):
return string

23
valet_plugins/valet_plugins/tests/unit/mocks/nova/scheduler/filters.py
View File

@ -1,23 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Filters."""
class BaseHostFilter(object):
"""Classdocs."""
def __init__(self, params):
"""Constructor."""

40
valet_plugins/valet_plugins/tests/unit/test_valet_api.py
View File

@ -1,40 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Test Valet API."""
import mock
from valet_plugins.tests.base import Base
from valet_plugins.common.valet_api import ValetAPIWrapper, requests
class TestValetApi(Base):
"""Test Valet Plugins API."""
def setUp(self):
"""Setup Test Valet Api and call ValetAPIWrapper init."""
super(TestValetApi, self).setUp()
self.valet_api_wrapper = self.init_ValetAPIWrapper()
@mock.patch.object(ValetAPIWrapper, "_register_opts")
def init_ValetAPIWrapper(self, mock_api):
"""Called by setup, mock api return value to none."""
mock_api.return_value = None
return ValetAPIWrapper()
@mock.patch.object(requests, 'request')
def test_plans_create(self, mock_request):
"""Test Plans create, mock request return value to none."""
mock_request.post.return_value = None

91
valet_plugins/valet_plugins/tests/unit/test_valet_filter.py
View File

@ -1,91 +0,0 @@
#
# Copyright 2014-2017 AT&T Intellectual Property
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Test Valet Filter."""
from keystoneclient.v2_0 import client
import mock
from valet_plugins.common import valet_api
from valet_plugins.plugins.nova.valet_filter import ValetFilter
from valet_plugins.tests.base import Base
class TestResources(object):
"""Test Resources."""
def __init__(self, host_name):
"""Initialize."""
self.host = host_name
class TestValetFilter(Base):
"""Test Valet Filter Base."""
def setUp(self):
"""Setup by mocking client and init valet filter."""
super(TestValetFilter, self).setUp()
client.Client = mock.MagicMock()
self.valet_filter = self.init_ValetFilter()
@mock.patch.object(valet_api.ValetAPIWrapper, '_register_opts')
@mock.patch.object(ValetFilter, '_register_opts')
def init_ValetFilter(self, mock_opt, mock_init):
"""Called by setup, mock init and opt and return ValetFilter()."""
mock_init.return_value = None
mock_opt.return_value = None
return ValetFilter()
@mock.patch.object(valet_api.ValetAPIWrapper, 'plans_create')
@mock.patch.object(valet_api.ValetAPIWrapper, 'placement')
def test_filter_all(self, mock_placement, mock_create):
"""Test Filter All by validating resource host values."""
mock_placement.return_value = None
mock_create.return_value = None
with mock.patch('oslo_config.cfg.CONF') as config:
setattr(config, "valet",
{self.valet_filter.opt_failure_mode_str: "yield",
self.valet_filter.opt_project_name_str:
"test_admin_tenant_name",
self.valet_filter.opt_username_str: "test_admin_username",
self.valet_filter.opt_password_str: "test_admin_password",
self.valet_filter.opt_auth_uri_str: "test_admin_auth_url"})
filter_properties = {'request_spec': {'instance_properties':
{'uuid': ""}},
'scheduler_hints':
{'heat_resource_uuid': "123456"},
'instance_type': {'name': "instance_name"}}
resources = self.valet_filter.filter_all(
[TestResources("first_host"), TestResources("second_host")],
filter_properties)
for resource in resources:
self.validate_test(resource.host in "first_host, second_host")
self.validate_test(mock_placement.called)
filter_properties = {'request_spec': {'instance_properties':
{'uuid': ""}},
'scheduler_hints': "scheduler_hints",
'instance_type': {'name': "instance_name"}}
resources = self.valet_filter.filter_all(
[TestResources("first_host"), TestResources("second_host")],
filter_properties)
for _ in resources:
self.validate_test(mock_create.called)