x/deb-os-cloud-config
Code Issues Proposed changes

Add support for creating baremetal flavors

Browse Source 
To improve the support for heterogenous nodes, add support for
deleting existing flavors in Nova, and creating baremetal flavors
that describe all nodes, allowing the caller to also override which
flavors are created. A setup-flavors CLI tool is provided for ease
of use with -incubator.

Change-Id: Ie139304016ef7553a46d9708be4281cc6afccb07
This commit is contained in:
Steve Kowalik 2014-10-07 16:18:09 +11:00
parent d1a2269186
commit e90348f862
6 changed files with 298 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.rst
View File

@ -34,6 +34,11 @@ Features
* Register services, such as Glance and Cinder with a configured Keystone.
* setup-flavors:
* Creates flavors in Nova, either describing the distinct set of nodes the
cloud has registered, or a custom set of flavors that has been specified.
* setup-neutron:
* Configure Neutron at the cloud (not the host) level, setting up either a

42
doc/source/usage.rst
View File

@ -203,3 +203,45 @@ default nameserver of 8.8.8.8, and the ext-net subnet has a CIDR of
192.0.2.0/24, a gateway of 192.0.2.1 and allocates DHCP from 192.0.2.45 until
192.0.2.64. setup-neutron will also create a router for the float network,
setting the external network as the gateway.
----------------
Creating flavors
----------------
The setup-flavors command line utility creates flavors in Nova -- either using
the nodes that have been registered to provide a distinct set of hardware that
is provisioned, or by specifing the set of flavors that should be created.
.. note::
setup-flavors will delete the existing default flavors, such as m1.small
and m1.xlarge. For this use case, the cloud that is having flavors created
is a cloud only using baremetal hardware, so only needs to describe the
hardware available.
Utilising the /tmp/one-node file specified in the register-nodes example
above, create a flavor::
setup-flavors -n /tmp/one-node
Which results in a flavor called "baremetal_2048_30_None_1".
If the ROOT_DISK environment variable is set in the environment, that will be
used as the disk size, leaving the remainder set as ephemeral storage, giving
a flavor name of "baremetal_2048_10_20_1".
Conversely, you can specify a JSON file describing the flavors to create::
setup-flavors -f /tmp/one-flavor
Where /tmp/one-flavor contains::
[
{
"name": "controller",
"memory": "2048",
"disk": "30",
"arch": "i386",
"cpu": "1"
}
]

80
os_cloud_config/cmd/setup_flavors.py Normal file
View File

@ -0,0 +1,80 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import argparse
import json
import logging
import os
import textwrap
from os_cloud_config.cmd.utils import _clients as clients
from os_cloud_config.cmd.utils import environment
from os_cloud_config import flavors
def parse_args():
description = textwrap.dedent("""
Create flavors describing the compute resources the cloud has
available.
If the list of flavors is only meant to encompass hardware that the cloud
has available, a JSON file describing the nodes can be specified, see
register-nodes for the format.
If a custom list of flavors is to be created, they can be specified as a
list of JSON objects. Each list item is a JSON object describing one
flavor, which has a "name" for the flavor, "memory" in MB, "cpu" in
threads, "disk" in GB and "arch" as one of i386/amd64/etc.
""")
parser = argparse.ArgumentParser(
description=description,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-n', '--nodes', dest='nodes',
help='A JSON file containing a list of nodes that '
'distinct flavors will be generated and created from')
group.add_argument('-f', '--flavors', dest='flavors',
help='A JSON file containing a list of flavors to '
'create directly')
parser.add_argument('-k', '--kernel', dest='kernel',
help='ID of the kernel in Glance', required=True)
parser.add_argument('-r', '--ramdisk', dest='ramdisk',
help='ID of the ramdisk in Glance', required=True)
environment._add_logging_arguments(parser)
return parser.parse_args()
def main():
args = parse_args()
environment._configure_logging(args)
try:
environment._ensure()
client = clients.get_nova_bm_client()
flavors.cleanup_flavors(client)
root_disk = os.environ.get('ROOT_DISK', None)
if args.nodes:
with open(args.nodes, 'r') as nodes_file:
nodes_list = json.load(nodes_file)
flavors.create_flavors_from_nodes(
client, nodes_list, args.kernel, args.ramdisk, root_disk)
elif args.flavors:
with open(args.flavors, 'r') as flavors_file:
flavors_list = json.load(flavors_file)
flavors.create_flavors_from_list(
client, flavors_list, args.kernel, args.ramdisk)
except Exception:
logging.exception("Unexpected error during command execution")
return 1
return 0

77
os_cloud_config/flavors.py Normal file
View File

@ -0,0 +1,77 @@
# Copyright (c) 2014 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import logging
LOG = logging.getLogger(__name__)
def cleanup_flavors(client, names=('m1.tiny', 'm1.small', 'm1.medium',
'm1.large', 'm1.xlarge')):
LOG.debug('Cleaning up non-baremetal flavors.')
for flavor in client.flavors.list():
if flavor.name in names:
client.flavors.delete(flavor.id)
def create_flavors_from_nodes(client, node_list, kernel, ramdisk, root_disk):
LOG.debug('Populating flavors to create from node list.')
node_details = set()
for node in node_list:
disk = node['disk']
ephemeral = 0
if root_disk:
disk = str(root_disk)
ephemeral = str(int(node['disk']) - int(root_disk))
node_details.add((node['memory'], disk, node['cpu'], node['arch'],
ephemeral))
flavor_list = []
for node in node_details:
new_flavor = {'memory': node[0], 'disk': node[1], 'cpu': node[2],
'arch': node[3], 'ephemeral': node[4]}
name = 'baremetal_%(memory)s_%(disk)s_%(ephemeral)s_%(cpu)s' % (
new_flavor)
new_flavor['name'] = name
flavor_list.append(new_flavor)
create_flavors_from_list(client, flavor_list, kernel, ramdisk)
def create_flavors_from_list(client, flavor_list, kernel, ramdisk):
LOG.debug('Creating flavors from flavors list.')
for flavor in filter_existing_flavors(client, flavor_list):
flavor.update({'kernel': kernel, 'ramdisk': ramdisk})
LOG.debug('Creating %(name)s flavor with memory %(memory)s, '
'disk %(disk)s, cpu %(cpu)s, %(arch)s arch.' % flavor)
_create_flavor(client, flavor)
def filter_existing_flavors(client, flavor_list):
flavors = client.flavors.list()
names_to_create = (set([f['name'] for f in flavor_list]) -
set([f.name for f in flavors]))
flavors_to_create = [f for f in flavor_list
if f['name'] in names_to_create]
return flavors_to_create
def _create_flavor(client, flavor_desc):
flavor = client.flavors.create(flavor_desc['name'], flavor_desc['memory'],
flavor_desc['cpu'], flavor_desc['disk'],
None, ephemeral=flavor_desc['ephemeral'])
bm_prefix = 'baremetal:deploy'
flavor_metadata = {'cpu_arch': flavor_desc['arch'],
'%s_kernel_id' % bm_prefix: flavor_desc['kernel'],
'%s_ramdisk_id' % bm_prefix: flavor_desc['ramdisk']}
flavor.set_keys(metadata=flavor_metadata)

93
os_cloud_config/tests/test_flavors.py Normal file
View File

@ -0,0 +1,93 @@
# Copyright (c) 2014 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import collections
import mock
from os_cloud_config import flavors
from os_cloud_config.tests import base
class FlavorsTest(base.TestCase):
def test_cleanup_flavors(self):
client = mock.MagicMock()
to_del = ('m1.tiny', 'm1.small', 'm1.medium', 'm1.large', 'm1.xlarge')
delete_calls = [mock.call(flavor) for flavor in to_del]
flavors.cleanup_flavors(client=client)
client.flavors.delete.has_calls(delete_calls)
client.flavors.delete.assert_not_called_with('baremetal')
def test_filter_existing_flavors_none(self):
client = mock.MagicMock()
client.flavors.list.return_value = []
flavor_list = [{'name': 'baremetal'}]
self.assertEqual(flavor_list,
flavors.filter_existing_flavors(client, flavor_list))
def test_filter_existing_flavors_one_existing(self):
client = mock.MagicMock()
flavor = collections.namedtuple('flavor', ['name'])
client.flavors.list.return_value = [flavor('baremetal_1')]
flavor_list = [{'name': 'baremetal_0'}, {'name': 'baremetal_1'}]
self.assertEqual([flavor_list[0]],
flavors.filter_existing_flavors(client, flavor_list))
def test_filter_existing_flavors_all_existing(self):
client = mock.MagicMock()
flavor = collections.namedtuple('flavor', ['name'])
client.flavors.list.return_value = [flavor('baremetal_0'),
flavor('baremetal_1')]
flavor_list = [{'name': 'baremetal_0'}, {'name': 'baremetal_1'}]
self.assertEqual([],
flavors.filter_existing_flavors(client, flavor_list))
@mock.patch('os_cloud_config.flavors._create_flavor')
def test_create_flavors_from_nodes(self, create_flavor):
node = {'cpu': '1', 'memory': '2048', 'disk': '30', 'arch': 'i386'}
node_list = [node, node]
client = mock.MagicMock()
flavors.create_flavors_from_nodes(client, node_list, 'aaa', 'bbb',
'10')
expected_flavor = node
expected_flavor.update({'disk': '10', 'ephemeral': '20',
'kernel': 'aaa', 'ramdisk': 'bbb',
'name': 'baremetal_2048_10_20_1'})
create_flavor.assert_called_once_with(client, expected_flavor)
@mock.patch('os_cloud_config.flavors._create_flavor')
def test_create_flavors_from_list(self, create_flavor):
flavor_list = [{'name': 'controller', 'cpu': '1', 'memory': '2048',
'disk': '30', 'arch': 'amd64'}]
client = mock.MagicMock()
flavors.create_flavors_from_list(client, flavor_list, 'aaa', 'bbb')
create_flavor.assert_called_once_with(
client, {'disk': '30', 'cpu': '1', 'arch': 'amd64',
'kernel': 'aaa', 'ramdisk': 'bbb', 'memory': '2048',
'name': 'controller'})
def test_create_flavor(self):
flavor = {'cpu': '1', 'memory': '2048', 'disk': '30', 'arch': 'i386',
'kernel': 'aaa', 'ramdisk': 'bbb', 'name': 'baremetal',
'ephemeral': None}
client = mock.MagicMock()
flavors._create_flavor(client, flavor)
client.flavors.create.assert_called_once_with(
'baremetal', '2048', '1', '30', None, ephemeral=None)
metadata = {'cpu_arch': 'i386', 'baremetal:deploy_kernel_id': 'aaa',
'baremetal:deploy_ramdisk_id': 'bbb'}
client.flavors.create.return_value.set_keys.assert_called_once_with(
metadata=metadata)

1
setup.cfg
View File

@ -29,6 +29,7 @@ console_scripts =
init-keystone = os_cloud_config.cmd.init_keystone:main
register-nodes = os_cloud_config.cmd.register_nodes:main
setup-endpoints = os_cloud_config.cmd.setup_endpoints:main
setup-flavors = os_cloud_config.cmd.setup_flavors:main
setup-neutron = os_cloud_config.cmd.setup_neutron:main
[build_sphinx]