openstack/kolla-ansible
Code Issues Proposed changes

Improve developer documentation for Bifrost

Browse Source 
This change fills out the Bifrost developer documentation and makes a
few corrections.

Change-Id: I14a6eca58c440677d5de638cbc677e9ec05738af
Closes-Bug: #1661013
Co-Authored-By: Sayantani Goswami <sayantani.goswami@intel.com>
This commit is contained in:
Mark Goddard 2017-02-21 08:53:02 -05:00
parent 2cdaefe243
commit ee68bc385f
1 changed files with 273 additions and 134 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

407
doc/bifrost.rst
View File

@ -2,31 +2,87 @@
Bifrost Guide Bifrost Guide
============= =============
From the bifrost developer documentation:
Prep host Bifrost (pronounced bye-frost) is a set of Ansible playbooks that automates
========= the task of deploying a base image onto a set of known hardware using
Ironic. It provides modular utility for one-off operating system
deployment with as few operational requirements as reasonably possible.
Clone kolla Kolla uses bifrost as a mechanism for bootstrapping an OpenStack control plane
----------- on a set of baremetal servers. Kolla provides a container image for bifrost.
Kolla-ansible provides a playbook to configure and deploy the bifrost
container, as well as building a base OS image and provisioning it onto the
baremetal nodes.
:: Hosts in the System
===================
git clone https://github.com/openstack/kolla In a system deployed by bifrost we define a number of classes of hosts.
cd kolla
set up kolla dependencies :doc:`quickstart` Control host
The control host is the host on which kolla and kolla-ansible will be
installed, and is typically where the cloud will be managed from.
Deployment host
The deployment host runs the bifrost deploy container and is used to
provision the cloud hosts.
Cloud hosts
The cloud hosts run the OpenStack control plane, compute and storage
services.
Bare metal compute hosts:
In a cloud providing bare metal compute services to tenants via Ironic,
these hosts will run the bare metal tenant workloads. In a cloud with only
virtualised compute this category of hosts does not exist.
Fix hosts file .. note::
--------------
Docker bind mounts ``/etc/hosts`` into the container from a volume In many cases the control and deployment host will be the same, although
This prevents atomic renames which will prevent ansible from fixing this is not mandatory.
the ``/etc/hosts`` file automatically.
To enable bifrost to be bootstrapped correctly add the deployment .. note::
hosts hostname to 127.0.0.1 line for example:
:: Bifrost supports provisioning of bare metal nodes. While kolla-ansible is
agnostic to whether the host OS runs on bare metal or is virtualised, in a
virtual environment the provisioning of VMs for cloud hosts and their base
OS images is currently out of scope.
Cloud Deployment Procedure
==========================
Cloud deployment using kolla and bifrost follows the following high level
steps:
#. Install and configure kolla and kolla-ansible on the control host.
#. Deploy bifrost on the deployment host.
#. Use bifrost to build a base OS image and provision cloud hosts with this
image.
#. Deploy OpenStack services on the cloud hosts provisioned by bifrost.
Preparation
===========
Prepare the Control Host
------------------------
Follow the **Install dependencies** section of the :doc:`quickstart` guide
instructions to set up kolla and kolla-ansible dependencies. Follow the
instructions in either the **Install kolla for development** section or the
**Install kolla for deployment or evaluation** section to install kolla and
kolla-ansible.
Prepare the Deployment Host
---------------------------
RabbitMQ requires that the system's hostname resolves to the IP address that it
has been configured to use, which with bifrost will be ``127.0.0.1``. Bifrost
will attempt to modify ``/etc/hosts`` on the deployment host to ensure that
this is the case. Docker bind mounts ``/etc/hosts`` into the container from a
volume. This prevents atomic renames which will prevent Ansible from fixing
the
``/etc/hosts`` file automatically.
To enable bifrost to be bootstrapped correctly add an entry to ``/etc/hosts``
resolving the deployment host's hostname to ``127.0.0.1``, for example::
ubuntu@bifrost:/repo/kolla$ cat /etc/hosts ubuntu@bifrost:/repo/kolla$ cat /etc/hosts
127.0.0.1 bifrost localhost 127.0.0.1 bifrost localhost
@ -40,50 +96,70 @@ hosts hostname to 127.0.0.1 line for example:
ff02::3 ip6-allhosts ff02::3 ip6-allhosts
192.168.100.15 bifrost 192.168.100.15 bifrost
Build a Bifrost Container Image
===============================
Enable source build type This section provides instructions on how to build a container image for
======================== bifrost using kolla.
Via config file Enable Source Build Type
--------------- ------------------------
:: Currently kolla only supports the source install type for the bifrost image.
Configuration File
~~~~~~~~~~~~~~~~~~
If required, generate a default configuration file for ``kolla-build``::
cd kolla
tox -e genconfig tox -e genconfig
Modify ``kolla-build.conf`` as follows. Modify ``kolla-build.conf``, setting ``install_type`` to ``source``::
Set ``install_type`` to ``source``
::
install_type = source install_type = source
Command line Command line
------------ ~~~~~~~~~~~~
Alternatively if you do not wish to use the ``kolla-build.conf`` Alternatively, instead of using ``kolla-build.conf``, a source build can be
you can enable a source build by appending ``-t source`` to enabled by appending ``--type source`` to the ``kolla-build`` or
your ``kolla-build`` or ``tools/build.py`` command. ``tools/build.py`` command.
Build container Build Container
=============== ---------------
Development Development
----------- ~~~~~~~~~~~
:: ::
cd kolla
tools/build.py bifrost-deploy tools/build.py bifrost-deploy
Production Production
---------- ~~~~~~~~~~
:: ::
kolla-build bifrost-deploy kolla-build bifrost-deploy
.. note::
By default kolla-build will build all containers using CentOS as the base
image. To change this behavior, use the following parameter with
``kolla-build`` or ``tools/build.py`` command::
--base [ubuntu|centos|oraclelinux]
Configure and Deploy a Bifrost Container
========================================
This section provides instructions for how to configure and deploy a container
running bifrost services.
Prepare Kolla-Ansible Inventory Prepare Kolla-Ansible Inventory
=============================== -------------------------------
Kolla-ansible will deploy bifrost on the hosts in the ``bifrost`` Ansible Kolla-ansible will deploy bifrost on the hosts in the ``bifrost`` Ansible
group. In the ``all-in-one`` and ``multinode`` inventory files, a ``bifrost`` group. In the ``all-in-one`` and ``multinode`` inventory files, a ``bifrost``
@ -100,85 +176,131 @@ services deployed by kolla including OpenStack Ironic, MariaDB, RabbitMQ and
(optionally) OpenStack Keystone. These services should not be deployed on the (optionally) OpenStack Keystone. These services should not be deployed on the
host on which bifrost is deployed. host on which bifrost is deployed.
Prepare bifrost configs Prepare Kolla-Ansible Configuration
======================= -----------------------------------
Create servers.yml Follow the instructions in :doc:`quickstart` to prepare kolla-ansible's global
------------------ configuration file ``globals.yml``. For bifrost, the
``bifrost_network_interface`` variable should be set to the name of the
interface that will be used to provision bare metal cloud hosts if this is
different than ``network_interface``. For example to use ``eth1``:
The ``servers.yml`` will describing your physical nodes and list IPMI .. code-block:: yaml
credentials. See bifrost dynamic inventory examples for more details.
For example ``/etc/kolla/config/bifrost/servers.yml`` bifrost_network_interface: eth1
Note that this interface should typically have L2 network connectivity with the
bare metal cloud hosts in order to provide DHCP leases with PXE boot options.
Since bifrost only supports the source image type, ensure that this is
reflected in ``globals.yml``
.. code-block:: yaml
kolla_install_type: source
Prepare Bifrost Configuration
-----------------------------
Kolla ansible custom configuration files can be placed in a directory given by
the ``node_custom_config`` variable, which defaults do ``/etc/kolla/config``.
Bifrost configuration files should be placed in this directory or in a
``bifrost`` subdirectory of it (e.g. ``/etc/kolla/config/bifrost``). Within
these directories the files ``bifrost.yml``, ``servers.yml`` and ``dib.yml``
can be used to configure Bifrost.
Create a Bifrost Inventory
~~~~~~~~~~~~~~~~~~~~~~~~~~
The file ``servers.yml`` defines the bifrost hardware inventory that will be
used to populate Ironic. See the `bifrost dynamic inventory examples
<https://github.com/openstack/bifrost/tree/master/playbooks/inventory>`_ for
further details.
For example, the following inventory defines a single node managed via the
Ironic ``agent_ipmitool`` driver. The inventory contains credentials required
to access the node's BMC via IPMI, the MAC addresses of the node's NICs, an IP
address to configure the node's configdrive with, a set of scheduling
properties and a logical name.
.. code-block:: yaml .. code-block:: yaml
--- ---
cloud1: cloud1:
uuid: "31303735-3934-4247-3830-333132535336" uuid: "31303735-3934-4247-3830-333132535336"
driver_info: driver_info:
power: power:
ipmi_username: "admin" ipmi_username: "admin"
ipmi_address: "192.168.1.30" ipmi_address: "192.168.1.30"
ipmi_password: "root" ipmi_password: "root"
nics: nics:
- -
mac: "1c:c1:de:1c:aa:53" mac: "1c:c1:de:1c:aa:53"
- -
mac: "1c:c1:de:1c:aa:52" mac: "1c:c1:de:1c:aa:52"
driver: "agent_ipmitool" driver: "agent_ipmitool"
ipv4_address: "192.168.1.10" ipv4_address: "192.168.1.10"
properties: properties:
cpu_arch: "x86_64" cpu_arch: "x86_64"
ram: "24576" ram: "24576"
disk_size: "120" disk_size: "120"
cpus: "16" cpus: "16"
name: "cloud1" name: "cloud1"
adjust as appropriate for your deployment The required inventory will be specific to the hardware and environment in use.
Create bifrost.yml Create Bifrost Configuration
------------------ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
By default kolla mostly use bifrosts default playbook values.
Parameters passed to the bifrost install playbook can be overridden by
creating a ``bifrost.yml`` file in the kolla custom config directory or in a
bifrost sub directory.
For example ``/etc/kolla/config/bifrost/bifrost.yml``
:: The file ``bifrost.yml`` provides global configuration for the bifrost
playbooks. By default kolla mostly uses bifrost's default variable values.
For details on bifrost's variables see the bifrost documentation.
mysql_service_name: mysql For example:
ansible_python_interpreter: /var/lib/kolla/venv/bin/python
network_interface: < add you network interface here >
# uncomment below if needed
# dhcp_pool_start: 192.168.2.200
# dhcp_pool_end: 192.168.2.250
# dhcp_lease_time: 12h
# dhcp_static_mask: 255.255.255.0
Create Disk Image Builder Config .. code-block:: yaml
--------------------------------
By default kolla mostly use bifrosts default playbook values when
building the baremetal os image. The baremetal os image can be customised
by creating a ``dib.yml`` file in the kolla custom config directory or in a
bifrost sub directory.
For example ``/etc/kolla/config/bifrost/dib.yml``
:: mysql_service_name: mysql
ansible_python_interpreter: /var/lib/kolla/venv/bin/python
enabled_drivers: agent_ipmitool,agent_ipminative
# uncomment below if needed
# dhcp_pool_start: 192.168.2.200
# dhcp_pool_end: 192.168.2.250
# dhcp_lease_time: 12h
# dhcp_static_mask: 255.255.255.0
dib_os_element: ubuntu Create Disk Image Builder Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The file ``dib.yml`` provides configuration for bifrost's image build
playbooks. By default kolla mostly uses bifrost's default variable values when
building the baremetal OS and deployment images, and will build an
**Ubuntu-based** image for deployment to nodes. For details on bifrost's
variables see the bifrost documentation.
For example to use the ``debian`` Disk Image Builder OS element:
.. code-block:: yaml
dib_os_element: debian
See the `diskimage-builder documentation
<http://docs.openstack.org/developer/diskimage-builder/>`__ for more details.
Deploy Bifrost Deploy Bifrost
========================= --------------
Ansible The bifrost container can be deployed either using kolla-ansible or manually.
-------
Kolla-Ansible
~~~~~~~~~~~~~
Development Development
___________ ___________
:: ::
cd kolla-ansible
tools/kolla-ansible deploy-bifrost tools/kolla-ansible deploy-bifrost
Production Production
@ -189,16 +311,19 @@ __________
kolla-ansible deploy-bifrost kolla-ansible deploy-bifrost
Manual Manual
------ ~~~~~~
Start Bifrost Container Start Bifrost Container
_______________________ _______________________
:: ::
docker run -it --net=host -v /dev:/dev -d --privileged --name bifrost_deploy kolla/ubuntu-source-bifrost-deploy:3.0.1 docker run -it --net=host -v /dev:/dev -d \
--privileged --name bifrost_deploy \
kolla/ubuntu-source-bifrost-deploy:3.0.1
Copy configs Copy Configuration Files
____________ ________________________
.. code-block:: console .. code-block:: console
@ -207,39 +332,40 @@ ____________
docker cp /etc/kolla/config/bifrost/bifrost.yml bifrost_deploy:/etc/bifrost/bifrost.yml docker cp /etc/kolla/config/bifrost/bifrost.yml bifrost_deploy:/etc/bifrost/bifrost.yml
docker cp /etc/kolla/config/bifrost/dib.yml bifrost_deploy:/etc/bifrost/dib.yml docker cp /etc/kolla/config/bifrost/dib.yml bifrost_deploy:/etc/bifrost/dib.yml
Bootstrap bifrost Bootstrap Bifrost
_________________ _________________
:: ::
docker exec -it bifrost_deploy bash docker exec -it bifrost_deploy bash
Generate ssh key Generate an SSH Key
~~~~~~~~~~~~~~~~ ___________________
:: ::
ssh-keygen ssh-keygen
Source env variables Bootstrap and Start Services
~~~~~~~~~~~~~~~~~~~~ ____________________________
::
cd /bifrost
. env-vars
. /opt/stack/ansible/hacking/env-setup
cd playbooks/
Bootstrap and start services
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: console .. code-block:: console
ansible-playbook -vvvv -i /bifrost/playbooks/inventory/localhost /bifrost/playbooks/install.yaml -e @/etc/bifrost/bifrost.yml cd /bifrost
./scripts/env-setup.sh
. env-vars
cat > /etc/rabbitmq/rabbitmq-env.conf << EOF
HOME=/var/lib/rabbitmq
EOF
ansible-playbook -vvvv \
-i /bifrost/playbooks/inventory/target \
/bifrost/playbooks/install.yaml \
-e @/etc/bifrost/bifrost.yml \
-e @/etc/bifrost/dib.yml \
-e skip_package_install=true
Check ironic is running Validate the Deployed Container
======================= ===============================
.. code-block:: console .. code-block:: console
@ -257,80 +383,93 @@ Running "ironic node-list" should return with no nodes, for example
+------+------+---------------+-------------+--------------------+-------------+ +------+------+---------------+-------------+--------------------+-------------+
+------+------+---------------+-------------+--------------------+-------------+ +------+------+---------------+-------------+--------------------+-------------+
Enroll and Deploy Physical Nodes Enroll and Deploy Physical Nodes
================================ ================================
Ansible Once we have deployed a bifrost container we can use it to provision the bare
------- metal cloud hosts specified in the inventory file. Again, this can be done
either using kolla-ansible or manually.
Kolla-Ansible
-------------
Development Development
___________ ~~~~~~~~~~~
:: ::
tools/kolla-ansible deploy-servers tools/kolla-ansible deploy-servers
Production Production
__________ ~~~~~~~~~~
:: ::
kolla-ansible deploy-servers kolla-ansible deploy-servers
Manual Manual
------ ------
.. code-block:: console .. code-block:: console
docker exec -it bifrost_deploy bash docker exec -it bifrost_deploy bash
cd /bifrost cd /bifrost
. env-vars . env-vars
export BIFROST_INVENTORY_SOURCE=/etc/bifrost/servers.yml export BIFROST_INVENTORY_SOURCE=/etc/bifrost/servers.yml
ansible-playbook -vvvv -i inventory/bifrost_inventory.py enroll-dynamic.yaml -e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" -e network_interface=<provisioning interface> ansible-playbook -vvvv \
-i /bifrost/playbooks/inventory/bifrost_inventory.py \
/bifrost/playbooks/enroll-dynamic.yaml \
-e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" \
-e @/etc/bifrost/bifrost.yml
docker exec -it bifrost_deploy bash docker exec -it bifrost_deploy bash
cd /bifrost cd /bifrost
. env-vars . env-vars
export BIFROST_INVENTORY_SOURCE=/etc/bifrost/servers.yml export BIFROST_INVENTORY_SOURCE=/etc/bifrost/servers.yml
ansible-playbook -vvvv -i inventory/bifrost_inventory.py deploy-dynamic.yaml -e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" -e network_interface=<prvisioning interface> -e @/etc/bifrost/dib.yml ansible-playbook -vvvv \
-i /bifrost/playbooks/inventory/bifrost_inventory.py \
/bifrost/playbooks/deploy-dynamic.yaml \
-e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" \
-e @/etc/bifrost/bifrost.yml
At this point ironic should clean down your nodes and install the default At this point Ironic should clean down the nodes and install the default
os image. OS image.
Advanced configuration Advanced Configuration
====================== ======================
Bring your own image Bring Your Own Image
-------------------- --------------------
TODO TODO
Bring your own ssh key Bring Your Own SSH Key
---------------------- ----------------------
To use your own ssh key after you have generated the ``passwords.yml`` file
update the private and public keys under bifrost_ssh_key. To use your own SSH key after you have generated the ``passwords.yml`` file
update the private and public keys under ``bifrost_ssh_key``.
Known issues Known issues
============ ============
SSH daemon not running SSH daemon not running
---------------------- ----------------------
By default sshd is installed in the image but may not be enabled.
If you encounter this issue you will have to access the server physically in By default ``sshd`` is installed in the image but may not be enabled. If you
recovery mode to enable the ssh service. If your hardware supports it, this encounter this issue you will have to access the server physically in recovery
can be done remotely with ipmitool and serial over lan. For example mode to enable the ``sshd`` service. If your hardware supports it, this can be
done remotely with ``ipmitool`` and Serial Over LAN. For example
.. code-block:: console .. code-block:: console
ipmitool -I lanplus -H 192.168.1.30 -U admin -P root sol activate ipmitool -I lanplus -H 192.168.1.30 -U admin -P root sol activate
References References
========== ==========
Docs: http://docs.openstack.org/developer/bifrost/ Bifrost documentation: http://docs.openstack.org/developer/bifrost/
Troubleshooting: http://docs.openstack.org/developer/bifrost/troubleshooting.html Bifrost troubleshooting guide: http://docs.openstack.org/developer/bifrost/troubleshooting.html
Code: https://github.com/openstack/bifrost Bifrost code repository: https://github.com/openstack/bifrost