Improve developer documentation for Bifrost
This change fills out the Bifrost developer documentation and makes a few corrections. Change-Id: I14a6eca58c440677d5de638cbc677e9ec05738af Closes-Bug: #1661013 Co-Authored-By: Sayantani Goswami <sayantani.goswami@intel.com>
This commit is contained in:
Mark Goddard
parent
2cdaefe243
commit
ee68bc385f
407
doc/bifrost.rst
407
doc/bifrost.rst
@ -2,31 +2,87 @@
|
||||
Bifrost Guide
|
||||
=============
|
||||
|
||||
From the bifrost developer documentation:
|
||||
|
||||
Prep host
|
||||
=========
|
||||
Bifrost (pronounced bye-frost) is a set of Ansible playbooks that automates
|
||||
the task of deploying a base image onto a set of known hardware using
|
||||
Ironic. It provides modular utility for one-off operating system
|
||||
deployment with as few operational requirements as reasonably possible.
|
||||
|
||||
Clone kolla
|
||||
-----------
|
||||
Kolla uses bifrost as a mechanism for bootstrapping an OpenStack control plane
|
||||
on a set of baremetal servers. Kolla provides a container image for bifrost.
|
||||
Kolla-ansible provides a playbook to configure and deploy the bifrost
|
||||
container, as well as building a base OS image and provisioning it onto the
|
||||
baremetal nodes.
|
||||
|
||||
::
|
||||
Hosts in the System
|
||||
===================
|
||||
|
||||
git clone https://github.com/openstack/kolla
|
||||
cd kolla
|
||||
In a system deployed by bifrost we define a number of classes of hosts.
|
||||
|
||||
set up kolla dependencies :doc:`quickstart`
|
||||
Control host
|
||||
The control host is the host on which kolla and kolla-ansible will be
|
||||
installed, and is typically where the cloud will be managed from.
|
||||
Deployment host
|
||||
The deployment host runs the bifrost deploy container and is used to
|
||||
provision the cloud hosts.
|
||||
Cloud hosts
|
||||
The cloud hosts run the OpenStack control plane, compute and storage
|
||||
services.
|
||||
Bare metal compute hosts:
|
||||
In a cloud providing bare metal compute services to tenants via Ironic,
|
||||
these hosts will run the bare metal tenant workloads. In a cloud with only
|
||||
virtualised compute this category of hosts does not exist.
|
||||
|
||||
Fix hosts file
|
||||
--------------
|
||||
.. note::
|
||||
|
||||
Docker bind mounts ``/etc/hosts`` into the container from a volume
|
||||
This prevents atomic renames which will prevent ansible from fixing
|
||||
the ``/etc/hosts`` file automatically.
|
||||
In many cases the control and deployment host will be the same, although
|
||||
this is not mandatory.
|
||||
|
||||
To enable bifrost to be bootstrapped correctly add the deployment
|
||||
hosts hostname to 127.0.0.1 line for example:
|
||||
.. note::
|
||||
|
||||
::
|
||||
Bifrost supports provisioning of bare metal nodes. While kolla-ansible is
|
||||
agnostic to whether the host OS runs on bare metal or is virtualised, in a
|
||||
virtual environment the provisioning of VMs for cloud hosts and their base
|
||||
OS images is currently out of scope.
|
||||
|
||||
Cloud Deployment Procedure
|
||||
==========================
|
||||
|
||||
Cloud deployment using kolla and bifrost follows the following high level
|
||||
steps:
|
||||
|
||||
#. Install and configure kolla and kolla-ansible on the control host.
|
||||
#. Deploy bifrost on the deployment host.
|
||||
#. Use bifrost to build a base OS image and provision cloud hosts with this
|
||||
image.
|
||||
#. Deploy OpenStack services on the cloud hosts provisioned by bifrost.
|
||||
|
||||
Preparation
|
||||
===========
|
||||
|
||||
Prepare the Control Host
|
||||
------------------------
|
||||
|
||||
Follow the **Install dependencies** section of the :doc:`quickstart` guide
|
||||
instructions to set up kolla and kolla-ansible dependencies. Follow the
|
||||
instructions in either the **Install kolla for development** section or the
|
||||
**Install kolla for deployment or evaluation** section to install kolla and
|
||||
kolla-ansible.
|
||||
|
||||
Prepare the Deployment Host
|
||||
---------------------------
|
||||
|
||||
RabbitMQ requires that the system's hostname resolves to the IP address that it
|
||||
has been configured to use, which with bifrost will be ``127.0.0.1``. Bifrost
|
||||
will attempt to modify ``/etc/hosts`` on the deployment host to ensure that
|
||||
this is the case. Docker bind mounts ``/etc/hosts`` into the container from a
|
||||
volume. This prevents atomic renames which will prevent Ansible from fixing
|
||||
the
|
||||
``/etc/hosts`` file automatically.
|
||||
|
||||
To enable bifrost to be bootstrapped correctly add an entry to ``/etc/hosts``
|
||||
resolving the deployment host's hostname to ``127.0.0.1``, for example::
|
||||
|
||||
ubuntu@bifrost:/repo/kolla$ cat /etc/hosts
|
||||
127.0.0.1 bifrost localhost
|
||||
@ -40,50 +96,70 @@ hosts hostname to 127.0.0.1 line for example:
|
||||
ff02::3 ip6-allhosts
|
||||
192.168.100.15 bifrost
|
||||
|
||||
Build a Bifrost Container Image
|
||||
===============================
|
||||
|
||||
Enable source build type
|
||||
========================
|
||||
This section provides instructions on how to build a container image for
|
||||
bifrost using kolla.
|
||||
|
||||
Via config file
|
||||
---------------
|
||||
Enable Source Build Type
|
||||
------------------------
|
||||
|
||||
::
|
||||
Currently kolla only supports the source install type for the bifrost image.
|
||||
|
||||
Configuration File
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If required, generate a default configuration file for ``kolla-build``::
|
||||
|
||||
cd kolla
|
||||
tox -e genconfig
|
||||
|
||||
Modify ``kolla-build.conf`` as follows.
|
||||
Set ``install_type`` to ``source``
|
||||
|
||||
::
|
||||
Modify ``kolla-build.conf``, setting ``install_type`` to ``source``::
|
||||
|
||||
install_type = source
|
||||
|
||||
Command line
|
||||
------------
|
||||
~~~~~~~~~~~~
|
||||
|
||||
Alternatively if you do not wish to use the ``kolla-build.conf``
|
||||
you can enable a source build by appending ``-t source`` to
|
||||
your ``kolla-build`` or ``tools/build.py`` command.
|
||||
Alternatively, instead of using ``kolla-build.conf``, a source build can be
|
||||
enabled by appending ``--type source`` to the ``kolla-build`` or
|
||||
``tools/build.py`` command.
|
||||
|
||||
Build container
|
||||
===============
|
||||
Build Container
|
||||
---------------
|
||||
|
||||
Development
|
||||
-----------
|
||||
~~~~~~~~~~~
|
||||
|
||||
::
|
||||
|
||||
cd kolla
|
||||
tools/build.py bifrost-deploy
|
||||
|
||||
Production
|
||||
----------
|
||||
~~~~~~~~~~
|
||||
|
||||
::
|
||||
|
||||
kolla-build bifrost-deploy
|
||||
|
||||
.. note::
|
||||
|
||||
By default kolla-build will build all containers using CentOS as the base
|
||||
image. To change this behavior, use the following parameter with
|
||||
``kolla-build`` or ``tools/build.py`` command::
|
||||
|
||||
--base [ubuntu|centos|oraclelinux]
|
||||
|
||||
Configure and Deploy a Bifrost Container
|
||||
========================================
|
||||
|
||||
This section provides instructions for how to configure and deploy a container
|
||||
running bifrost services.
|
||||
|
||||
Prepare Kolla-Ansible Inventory
|
||||
===============================
|
||||
-------------------------------
|
||||
|
||||
Kolla-ansible will deploy bifrost on the hosts in the ``bifrost`` Ansible
|
||||
group. In the ``all-in-one`` and ``multinode`` inventory files, a ``bifrost``
|
||||
@ -100,85 +176,131 @@ services deployed by kolla including OpenStack Ironic, MariaDB, RabbitMQ and
|
||||
(optionally) OpenStack Keystone. These services should not be deployed on the
|
||||
host on which bifrost is deployed.
|
||||
|
||||
Prepare bifrost configs
|
||||
=======================
|
||||
Prepare Kolla-Ansible Configuration
|
||||
-----------------------------------
|
||||
|
||||
Create servers.yml
|
||||
------------------
|
||||
Follow the instructions in :doc:`quickstart` to prepare kolla-ansible's global
|
||||
configuration file ``globals.yml``. For bifrost, the
|
||||
``bifrost_network_interface`` variable should be set to the name of the
|
||||
interface that will be used to provision bare metal cloud hosts if this is
|
||||
different than ``network_interface``. For example to use ``eth1``:
|
||||
|
||||
The ``servers.yml`` will describing your physical nodes and list IPMI
|
||||
credentials. See bifrost dynamic inventory examples for more details.
|
||||
.. code-block:: yaml
|
||||
|
||||
For example ``/etc/kolla/config/bifrost/servers.yml``
|
||||
bifrost_network_interface: eth1
|
||||
|
||||
Note that this interface should typically have L2 network connectivity with the
|
||||
bare metal cloud hosts in order to provide DHCP leases with PXE boot options.
|
||||
|
||||
Since bifrost only supports the source image type, ensure that this is
|
||||
reflected in ``globals.yml``
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
kolla_install_type: source
|
||||
|
||||
Prepare Bifrost Configuration
|
||||
-----------------------------
|
||||
|
||||
Kolla ansible custom configuration files can be placed in a directory given by
|
||||
the ``node_custom_config`` variable, which defaults do ``/etc/kolla/config``.
|
||||
Bifrost configuration files should be placed in this directory or in a
|
||||
``bifrost`` subdirectory of it (e.g. ``/etc/kolla/config/bifrost``). Within
|
||||
these directories the files ``bifrost.yml``, ``servers.yml`` and ``dib.yml``
|
||||
can be used to configure Bifrost.
|
||||
|
||||
Create a Bifrost Inventory
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The file ``servers.yml`` defines the bifrost hardware inventory that will be
|
||||
used to populate Ironic. See the `bifrost dynamic inventory examples
|
||||
<https://github.com/openstack/bifrost/tree/master/playbooks/inventory>`_ for
|
||||
further details.
|
||||
|
||||
For example, the following inventory defines a single node managed via the
|
||||
Ironic ``agent_ipmitool`` driver. The inventory contains credentials required
|
||||
to access the node's BMC via IPMI, the MAC addresses of the node's NICs, an IP
|
||||
address to configure the node's configdrive with, a set of scheduling
|
||||
properties and a logical name.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
cloud1:
|
||||
uuid: "31303735-3934-4247-3830-333132535336"
|
||||
driver_info:
|
||||
power:
|
||||
ipmi_username: "admin"
|
||||
ipmi_address: "192.168.1.30"
|
||||
ipmi_password: "root"
|
||||
nics:
|
||||
-
|
||||
mac: "1c:c1:de:1c:aa:53"
|
||||
-
|
||||
mac: "1c:c1:de:1c:aa:52"
|
||||
driver: "agent_ipmitool"
|
||||
ipv4_address: "192.168.1.10"
|
||||
properties:
|
||||
cpu_arch: "x86_64"
|
||||
ram: "24576"
|
||||
disk_size: "120"
|
||||
cpus: "16"
|
||||
name: "cloud1"
|
||||
uuid: "31303735-3934-4247-3830-333132535336"
|
||||
driver_info:
|
||||
power:
|
||||
ipmi_username: "admin"
|
||||
ipmi_address: "192.168.1.30"
|
||||
ipmi_password: "root"
|
||||
nics:
|
||||
-
|
||||
mac: "1c:c1:de:1c:aa:53"
|
||||
-
|
||||
mac: "1c:c1:de:1c:aa:52"
|
||||
driver: "agent_ipmitool"
|
||||
ipv4_address: "192.168.1.10"
|
||||
properties:
|
||||
cpu_arch: "x86_64"
|
||||
ram: "24576"
|
||||
disk_size: "120"
|
||||
cpus: "16"
|
||||
name: "cloud1"
|
||||
|
||||
adjust as appropriate for your deployment
|
||||
The required inventory will be specific to the hardware and environment in use.
|
||||
|
||||
Create bifrost.yml
|
||||
------------------
|
||||
By default kolla mostly use bifrosts default playbook values.
|
||||
Parameters passed to the bifrost install playbook can be overridden by
|
||||
creating a ``bifrost.yml`` file in the kolla custom config directory or in a
|
||||
bifrost sub directory.
|
||||
For example ``/etc/kolla/config/bifrost/bifrost.yml``
|
||||
Create Bifrost Configuration
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
::
|
||||
The file ``bifrost.yml`` provides global configuration for the bifrost
|
||||
playbooks. By default kolla mostly uses bifrost's default variable values.
|
||||
For details on bifrost's variables see the bifrost documentation.
|
||||
|
||||
mysql_service_name: mysql
|
||||
ansible_python_interpreter: /var/lib/kolla/venv/bin/python
|
||||
network_interface: < add you network interface here >
|
||||
# uncomment below if needed
|
||||
# dhcp_pool_start: 192.168.2.200
|
||||
# dhcp_pool_end: 192.168.2.250
|
||||
# dhcp_lease_time: 12h
|
||||
# dhcp_static_mask: 255.255.255.0
|
||||
For example:
|
||||
|
||||
Create Disk Image Builder Config
|
||||
--------------------------------
|
||||
By default kolla mostly use bifrosts default playbook values when
|
||||
building the baremetal os image. The baremetal os image can be customised
|
||||
by creating a ``dib.yml`` file in the kolla custom config directory or in a
|
||||
bifrost sub directory.
|
||||
For example ``/etc/kolla/config/bifrost/dib.yml``
|
||||
.. code-block:: yaml
|
||||
|
||||
::
|
||||
mysql_service_name: mysql
|
||||
ansible_python_interpreter: /var/lib/kolla/venv/bin/python
|
||||
enabled_drivers: agent_ipmitool,agent_ipminative
|
||||
# uncomment below if needed
|
||||
# dhcp_pool_start: 192.168.2.200
|
||||
# dhcp_pool_end: 192.168.2.250
|
||||
# dhcp_lease_time: 12h
|
||||
# dhcp_static_mask: 255.255.255.0
|
||||
|
||||
dib_os_element: ubuntu
|
||||
Create Disk Image Builder Configuration
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The file ``dib.yml`` provides configuration for bifrost's image build
|
||||
playbooks. By default kolla mostly uses bifrost's default variable values when
|
||||
building the baremetal OS and deployment images, and will build an
|
||||
**Ubuntu-based** image for deployment to nodes. For details on bifrost's
|
||||
variables see the bifrost documentation.
|
||||
|
||||
For example to use the ``debian`` Disk Image Builder OS element:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
dib_os_element: debian
|
||||
|
||||
See the `diskimage-builder documentation
|
||||
<http://docs.openstack.org/developer/diskimage-builder/>`__ for more details.
|
||||
|
||||
Deploy Bifrost
|
||||
=========================
|
||||
--------------
|
||||
|
||||
Ansible
|
||||
-------
|
||||
The bifrost container can be deployed either using kolla-ansible or manually.
|
||||
|
||||
Kolla-Ansible
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Development
|
||||
___________
|
||||
|
||||
::
|
||||
|
||||
cd kolla-ansible
|
||||
tools/kolla-ansible deploy-bifrost
|
||||
|
||||
Production
|
||||
@ -189,16 +311,19 @@ __________
|
||||
kolla-ansible deploy-bifrost
|
||||
|
||||
Manual
|
||||
------
|
||||
~~~~~~
|
||||
|
||||
Start Bifrost Container
|
||||
_______________________
|
||||
|
||||
::
|
||||
|
||||
docker run -it --net=host -v /dev:/dev -d --privileged --name bifrost_deploy kolla/ubuntu-source-bifrost-deploy:3.0.1
|
||||
docker run -it --net=host -v /dev:/dev -d \
|
||||
--privileged --name bifrost_deploy \
|
||||
kolla/ubuntu-source-bifrost-deploy:3.0.1
|
||||
|
||||
Copy configs
|
||||
____________
|
||||
Copy Configuration Files
|
||||
________________________
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -207,39 +332,40 @@ ____________
|
||||
docker cp /etc/kolla/config/bifrost/bifrost.yml bifrost_deploy:/etc/bifrost/bifrost.yml
|
||||
docker cp /etc/kolla/config/bifrost/dib.yml bifrost_deploy:/etc/bifrost/dib.yml
|
||||
|
||||
Bootstrap bifrost
|
||||
Bootstrap Bifrost
|
||||
_________________
|
||||
|
||||
::
|
||||
|
||||
docker exec -it bifrost_deploy bash
|
||||
|
||||
Generate ssh key
|
||||
~~~~~~~~~~~~~~~~
|
||||
Generate an SSH Key
|
||||
___________________
|
||||
|
||||
::
|
||||
|
||||
ssh-keygen
|
||||
|
||||
Source env variables
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
Bootstrap and Start Services
|
||||
____________________________
|
||||
|
||||
::
|
||||
|
||||
cd /bifrost
|
||||
. env-vars
|
||||
. /opt/stack/ansible/hacking/env-setup
|
||||
cd playbooks/
|
||||
|
||||
|
||||
Bootstrap and start services
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
.. code-block:: console
|
||||
|
||||
ansible-playbook -vvvv -i /bifrost/playbooks/inventory/localhost /bifrost/playbooks/install.yaml -e @/etc/bifrost/bifrost.yml
|
||||
cd /bifrost
|
||||
./scripts/env-setup.sh
|
||||
. env-vars
|
||||
cat > /etc/rabbitmq/rabbitmq-env.conf << EOF
|
||||
HOME=/var/lib/rabbitmq
|
||||
EOF
|
||||
ansible-playbook -vvvv \
|
||||
-i /bifrost/playbooks/inventory/target \
|
||||
/bifrost/playbooks/install.yaml \
|
||||
-e @/etc/bifrost/bifrost.yml \
|
||||
-e @/etc/bifrost/dib.yml \
|
||||
-e skip_package_install=true
|
||||
|
||||
Check ironic is running
|
||||
=======================
|
||||
Validate the Deployed Container
|
||||
===============================
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -257,80 +383,93 @@ Running "ironic node-list" should return with no nodes, for example
|
||||
+------+------+---------------+-------------+--------------------+-------------+
|
||||
+------+------+---------------+-------------+--------------------+-------------+
|
||||
|
||||
|
||||
Enroll and Deploy Physical Nodes
|
||||
================================
|
||||
|
||||
Ansible
|
||||
-------
|
||||
Once we have deployed a bifrost container we can use it to provision the bare
|
||||
metal cloud hosts specified in the inventory file. Again, this can be done
|
||||
either using kolla-ansible or manually.
|
||||
|
||||
Kolla-Ansible
|
||||
-------------
|
||||
|
||||
Development
|
||||
___________
|
||||
~~~~~~~~~~~
|
||||
|
||||
::
|
||||
|
||||
tools/kolla-ansible deploy-servers
|
||||
|
||||
Production
|
||||
__________
|
||||
~~~~~~~~~~
|
||||
|
||||
::
|
||||
|
||||
kolla-ansible deploy-servers
|
||||
|
||||
|
||||
Manual
|
||||
------
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
docker exec -it bifrost_deploy bash
|
||||
cd /bifrost
|
||||
. env-vars
|
||||
export BIFROST_INVENTORY_SOURCE=/etc/bifrost/servers.yml
|
||||
ansible-playbook -vvvv -i inventory/bifrost_inventory.py enroll-dynamic.yaml -e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" -e network_interface=<provisioning interface>
|
||||
ansible-playbook -vvvv \
|
||||
-i /bifrost/playbooks/inventory/bifrost_inventory.py \
|
||||
/bifrost/playbooks/enroll-dynamic.yaml \
|
||||
-e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" \
|
||||
-e @/etc/bifrost/bifrost.yml
|
||||
|
||||
docker exec -it bifrost_deploy bash
|
||||
cd /bifrost
|
||||
. env-vars
|
||||
export BIFROST_INVENTORY_SOURCE=/etc/bifrost/servers.yml
|
||||
ansible-playbook -vvvv -i inventory/bifrost_inventory.py deploy-dynamic.yaml -e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" -e network_interface=<prvisioning interface> -e @/etc/bifrost/dib.yml
|
||||
ansible-playbook -vvvv \
|
||||
-i /bifrost/playbooks/inventory/bifrost_inventory.py \
|
||||
/bifrost/playbooks/deploy-dynamic.yaml \
|
||||
-e "ansible_python_interpreter=/var/lib/kolla/venv/bin/python" \
|
||||
-e @/etc/bifrost/bifrost.yml
|
||||
|
||||
At this point ironic should clean down your nodes and install the default
|
||||
os image.
|
||||
At this point Ironic should clean down the nodes and install the default
|
||||
OS image.
|
||||
|
||||
Advanced configuration
|
||||
Advanced Configuration
|
||||
======================
|
||||
|
||||
Bring your own image
|
||||
Bring Your Own Image
|
||||
--------------------
|
||||
|
||||
TODO
|
||||
|
||||
Bring your own ssh key
|
||||
Bring Your Own SSH Key
|
||||
----------------------
|
||||
To use your own ssh key after you have generated the ``passwords.yml`` file
|
||||
update the private and public keys under bifrost_ssh_key.
|
||||
|
||||
To use your own SSH key after you have generated the ``passwords.yml`` file
|
||||
update the private and public keys under ``bifrost_ssh_key``.
|
||||
|
||||
Known issues
|
||||
============
|
||||
|
||||
SSH daemon not running
|
||||
----------------------
|
||||
By default sshd is installed in the image but may not be enabled.
|
||||
If you encounter this issue you will have to access the server physically in
|
||||
recovery mode to enable the ssh service. If your hardware supports it, this
|
||||
can be done remotely with ipmitool and serial over lan. For example
|
||||
|
||||
By default ``sshd`` is installed in the image but may not be enabled. If you
|
||||
encounter this issue you will have to access the server physically in recovery
|
||||
mode to enable the ``sshd`` service. If your hardware supports it, this can be
|
||||
done remotely with ``ipmitool`` and Serial Over LAN. For example
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
ipmitool -I lanplus -H 192.168.1.30 -U admin -P root sol activate
|
||||
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
Docs: http://docs.openstack.org/developer/bifrost/
|
||||
Bifrost documentation: http://docs.openstack.org/developer/bifrost/
|
||||
|
||||
Troubleshooting: http://docs.openstack.org/developer/bifrost/troubleshooting.html
|
||||
Bifrost troubleshooting guide: http://docs.openstack.org/developer/bifrost/troubleshooting.html
|
||||
|
||||
Code: https://github.com/openstack/bifrost
|
||||
Bifrost code repository: https://github.com/openstack/bifrost
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user