Update Ironic Documentation

This patch updates OSA Ironic documentation to be more current
and to provide working examples.

Change-Id: I98cb73f49c526d0699446c0325e05a10ffca3e01
This commit is contained in:
James Denton 2021-12-17 16:19:55 -06:00
parent cd684b1175
commit fc6902e811

View File

@ -1,68 +1,211 @@
====================================================== ======================================================
Configuring the Bare Metal (ironic) service (optional) Configuring the Bare Metal (Ironic) Service (optional)
====================================================== ======================================================
.. note:: .. note::
This feature is experimental at this time and it has not been fully This feature is experimental at this time and has not been fully
production tested yet. These implementation instructions assume that production tested.
ironic is being deployed as the sole hypervisor for the region.
Ironic is an OpenStack project which provisions bare metal (as opposed to Ironic is an OpenStack project which provisions bare metal (as opposed to
virtual) machines by leveraging common technologies such as PXE boot and IPMI virtual) machines by leveraging common technologies such as PXE boot and IPMI
to cover a wide range of hardware, while supporting pluggable drivers to allow to cover a wide range of hardware, while supporting pluggable drivers to allow
vendor-specific functionality to be added. vendor-specific functionality to be added.
OpenStack's ironic project makes physical servers as easy to provision as OpenStack's Ironic project makes physical servers as easy to provision as
virtual machines in a cloud. virtual machines in a cloud.
OpenStack-Ansible deployment OpenStack-Ansible Deployment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. Modify the environment files and force ``nova-compute`` to run from The use of Ironic within an OpenStack deployment leverages Nova to
within a container: deploy baremetal instances when an ``openstack server create`` command is
issued using a baremetal flavor. So, in addition to Ironic API services,
a Nova compute service using an Ironic compute driver (as opposed to
libvirt) must be configured. The playbooks can automatically deploy
this service when the inventory is configured accordingly.
.. code-block:: bash To deploy Ironic, populate the respective groups within
``openstack_user_config.yml``:
sed -i '/is_metal.*/d' /etc/openstack_deploy/env.d/nova.yml .. code-block::
Setup a neutron network for use by ironic ironic-infra_hosts: *infrastructure_hosts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ironic-compute_hosts: *infrastructure_hosts
In a general case, neutron networking can be a simple flat network. However, With the inventory updated, Ironic API and conductor services
in a complex case, this can be whatever you need and want. Ensure will be deployed on the infra/controller nodes, along with a ``nova-compute``
you adjust the deployment accordingly. The following is an example: service configured for use with Ironic.
OpenStack-Ansible is configured to support PXE-based deployments by default.
To enable the use of iPXE, which uses HTTP instead of TFTP for the full
deployment, add the following override:
.. code-block::
ironic_ipxe_enabled: yes
.. note::
With iPXE enabled, PXE is used to bootstrap into the iPXE loader.
Deployment times are considerably faster with iPXE vs PXE, and its
configuration is highly recommended. When iPXE is enabled, a web
server is deployed on the conductor node(s) to host images and files.
Some drivers of the Baremetal service (in particular, any drivers using Direct
deploy or Ansible deploy interfaces, and some virtual media drivers) require
target user images to be available over clean HTTP(S) URL with NO
authentication involved (neither username/password-based, nor token-based).
The default deploy method relies on Swift to provide this functionality. If
Swift is not available in your environment, then the following override can
provide similar functionality by using the web server deployed the conductor
node(s) (see ``ironic_ipxe_enabled``):
.. code-block::
ironic_enable_web_server_for_images: yes
The Ironic ``ipmi`` hardware driver is enabled by default. Vendor-specific
drivers, including iLO and DRAC, are available for use with supported
hardware. OpenStack-Ansible provides a set of drivers with pre-configured
hardware, boot, deploy, inspect, management, and power characteristics,
including:
* agent_ilo
* agent_ipmitool
* agent_ipmitool_socat
* agent_irmc
* agent_ucs
* pxe_agent_cimc
* pxe_drac
* pxe_drac_inspector
* pxe_ilo
* pxe_ipmitool
* pxe_ipmitool_socat
* pxe_irmc
* pxe_snmp
* pxe_ucs
.. note::
The characteristics of these drivers can be seen in further details
by reviewing the ``ironic_driver_types`` variable in the Ironic role.
To enable iLO and DRAC drivers, along with IPMI, set the following override:
.. code-block:: bash .. code-block:: bash
neutron net-create cleaning-net --shared \ ironic_drivers_enabled:
--provider:network_type flat \ - agent_ipmitool
--provider:physical_network ironic-net - pxe_ipmitool
- agent_ilo
- pxe_ilo
- pxe_drac
neutron subnet-create ironic-net 172.19.0.0/22 --name ironic-subnet Setup Neutron Networks for Use With Ironic
--ip-version=4 \ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--allocation-pool start=172.19.1.100,end=172.19.1.200 \
--enable-dhcp \
--dns-nameservers list=true 8.8.4.4 8.8.8.8
Building ironic images Ironic supports two main network interfaces: ``flat`` and ``neutron``:
* | The ``flat`` interface places all provisioned nodes and nodes being
| deployed into a single layer 2 network.
* | The ``neutron`` interface provides tenant-defined networking
| (aka multi-tenancy) by integrating with Neutron, while also separating
| tenant networks from the provisioning and cleaning provider networks.
With the ``flat`` network interface, inspection, cleaning, and provisioning
functions will be performed on the same ``flat`` provider network. All
baremetal nodes will share the same VLAN and network/subnet once deployed,
which may present security challenges to tenants and to the control plane.
With the ``neutron`` network interface, inspection, cleaning, provisioning,
and tenant networks can use distict VLANs. However, an ML2 plugin such as
``networking-generic-switch`` must be used to configure the respective
switchports when switching between functions.
`<https://docs.openstack.org/openstack-ansible-os_neutron/latest/app-genericswitch.html>`_
.. note::
Both the ``flat`` and ``neutron`` network interfaces require a cleaning
network to be defined in ``ironic.conf``. For ``flat`` deployments, the
cleaning network will be the same as the deployment network.
Create a network and subnet to be used by the baremetal instance for cleaning,
provisioning, and post-deployment use:
.. code-block:: bash
openstack network create \
--provider-network-type flat \
--provider-physical-network physnet1 \
myBaremetalNetwork
openstack subnet create \
--network myBaremetalNetwork \
--subnet-range 172.17.100.0/24 \
myBaremetalNetworkSubnet
Set an override to define the cleaning network name:
.. code-block:: bash
ironic_neutron_cleaning_network_name: "myBaremetalNetwork"
.. note::
Ironic multi-tenancy is an advanced topic that requires the use of a
compatible ML2 driver such as ``networking-generic-switch``.
.. important::
Provisioning activities on baremetal instances require network access
to the Ironic conductor (web) service and other OpenStack APIs. You must
ensure routing exists between respective networks for deployments to
succeed.
Building Ironic Images
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
Images using the ``diskimage-builder`` must be built outside of a container. Bare Metal provisioning requires two sets of images: the deploy images and the
For this process, use one of the physical hosts within the environment. user images. The deploy images consist of a kernel and ramdisk image that are
used by Ironic to prepare the baremetal server for actual OS deployment,
whereas the user images are installed on the baremetal server to be used by
the end user.
#. Install the necessary packages: For more information on building and uploading disk images for use with
Ironic, refer to the following documentation:
`<https://docs.openstack.org/ironic/latest/user/creating-images.html>`_
`<https://docs.openstack.org/ironic/latest/install/configure-glance-images.html>`_
There are two types of user images:
* Partition Images
* Whole Disk Images
For your convenience, the following steps have been provided to demonstrate
creating partition-based images.
.. note::
Images created using ``diskimage-builder`` must be built outside of an
LXC container. For this process, use one of the physical hosts within
the environment or a virtual machine.
#. Install the necessary pre-requisites:
.. code-block:: bash .. code-block:: bash
apt-get install qemu uuid-runtime curl apt install qemu uuid-runtime curl
#. Install the ``disk-imagebuilder`` package: #. Install the ``disk-imagebuilder`` package:
.. code-block:: bash .. code-block:: bash
pip install diskimage-builder --isolated pip install diskimage-builder
.. important:: .. important::
@ -70,150 +213,307 @@ For this process, use one of the physical hosts within the environment.
deployed by OpenStack-Ansible, otherwise pip will not deployed by OpenStack-Ansible, otherwise pip will not
resolve the external package. resolve the external package.
#. Optional: Force the ubuntu ``image-create`` process to use a modern kernel: #. Create Ubuntu Focal kernel, ramdisk, and user images:
.. code-block:: bash .. code-block:: bash
echo 'linux-image-generic-lts-xenial:' > \ export IMAGE_NAME=my-image
/usr/local/share/diskimage-builder/elements/ubuntu/package-installs.yaml export DIB_RELEASE=focal
export DIB_CLOUD_INIT_DATASOURCES="Ec2, ConfigDrive, OpenStack"
disk-image-create ubuntu baremetal dhcp-all-interfaces grub2 -o ${IMAGE_NAME}
#. Create Ubuntu ``initramfs``: #. Upload the created user images into the Image (Glance) Service:
.. code-block:: bash .. code-block:: bash
disk-image-create ironic-agent ubuntu -o ${IMAGE_NAME} # Kernel image:
openstack image create my-image.kernel \
--public \
--disk-format aki \
--container-format aki \
--file my-image.vmlinuz
#. Upload the created deploy images into the Image (glance) Service: # Ramdisk image
openstack image create my-image.initrd \
--public \
--disk-format ari \
--container-format ari \
--file my-image.initrd
.. code-block:: bash # User image
openstack image create my-image \
--public \
--disk-format qcow2 \
--container-format bare \
--property kernel_id=<kernel image uuid> \
--property ramdisk_id=<ramdisk image uuid> \
--file my-image.qcow2
# Upload the deploy image kernel .. note::
glance image-create --name ${IMAGE_NAME}.kernel --visibility public \
--disk-format aki --container-format aki < ${IMAGE_NAME}.kernel
# Upload the user image initramfs When a baremetal instance is provisioned using a partition-based
glance image-create --name ${IMAGE_NAME}.initramfs --visibility public \ image, the kernel and ramdisk images will be used for PXE when the
--disk-format ari --container-format ari < ${IMAGE_NAME}.initramfs ``local`` boot capability is not available.
#. Create Ubuntu user image: Creating an Ironic Flavor
.. code-block:: bash
disk-image-create ubuntu baremetal localboot local-config dhcp-all-interfaces grub2 -o ${IMAGE_NAME}
#. Upload the created user images into the Image (glance) Service:
.. code-block:: bash
# Upload the user image vmlinuz and store uuid
VMLINUZ_UUID="$(glance image-create --name ${IMAGE_NAME}.vmlinuz --visibility public --disk-format aki --container-format aki < ${IMAGE_NAME}.vmlinuz | awk '/\| id/ {print $4}')"
# Upload the user image initrd and store uuid
INITRD_UUID="$(glance image-create --name ${IMAGE_NAME}.initrd --visibility public --disk-format ari --container-format ari < ${IMAGE_NAME}.initrd | awk '/\| id/ {print $4}')"
# Create image
glance image-create --name ${IMAGE_NAME} --visibility public --disk-format qcow2 --container-format bare --property kernel_id=${VMLINUZ_UUID} --property ramdisk_id=${INITRD_UUID} < ${IMAGE_NAME}.qcow2
Creating an ironic flavor
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
#. Create a new flavor called ``my-baremetal-flavor``. The use of flavors are necessary when creating instances using Nova,
and baremetal flavors should be used when targeting baremetal nodes
for instances. The properties of the flavor, along with the defined
resource class, are useful to the scheduler when scheduling against
libvirt or ironic compute services.
.. note:: As an example, imagine an Ironic deployment has the following nodes:
The following example sets the CPU architecture for the newly created .. code-block:: bash
flavor to be `x86_64`.
.. code-block:: bash - node-1:
resource_class: ironic-gold
properties:
cpus: 32
memory_mb: 32768
capabilities:
boot_mode: uefi,bios
- node-2:
resource_class: ironic-silver
properties:
cpus: 16
memory_mb: 16384
nova flavor-create ${FLAVOR_NAME} ${FLAVOR_ID} ${FLAVOR_RAM} ${FLAVOR_DISK} ${FLAVOR_CPU} The operator might define the flavors as such:
nova flavor-key ${FLAVOR_NAME} set cpu_arch=x86_64
nova flavor-key ${FLAVOR_NAME} set capabilities:boot_option="local" .. code-block:: bash
- baremetal-gold
resources:
ironic-gold: 1
extra_specs:
capabilities: boot_mode:bios
- baremetal-gold-uefi
resources:
ironic-gold: 1
extra_specs:
capabilities: boot_mode:uefi
- baremetal-silver
resources:
ironic-silver: 1
A user booting an instance with either the baremetal-gold or
baremetal-gold-uefi flavor would land on node-1, because capabilities can
still be passed down to ironic, and the resource_class on the node matche
what is required by flavor. The baremetal-silver flavor would match node-2.
.. note:: .. note::
Ensure the flavor and nodes match when enrolling into ironic. A flavor can request exactly one instance of a bare metal resource class.
See the documentation on flavors for more information:
https://docs.openstack.org/nova/queens/admin/flavors.html
After successfully deploying the ironic node on subsequent boots, the instance When creating a baremetal flavor, its useful to add the RAM and
boots from your local disk as first preference. This speeds up the deployed CPU properties as a convenience to users, although they are not used for
node's boot time. Alternatively, if this is not set, the ironic node PXE boots scheduling. In addition, the DISK property is also not used for scheduling,
first and allows for operator-initiated image updates and other operations. but is still used to determine the root partition size.
.. code-block:: bash
openstack flavor create \
--ram 32768 \
--vcpu 32 \
--disk 120 \
baremetal-gold
After creation, associate each flavor with one custom resource class. The name
of a custom resource class that corresponds to a nodes resource class
(in the Bare Metal service) is:
* the bare metal nodes resource class all upper-cased
* prefixed with ``CUSTOM_``
* all punctuation replaced with an underscore
.. code-block:: bash
openstack flavor set \
--property resources:CUSTOM_IRONIC_GOLD=1 \
baremetal-gold
.. note:: .. note::
The operational reasoning and building an environment to support this Ensure the resource class defined in the flavor matches that
use case is not covered here. of the baremetal node, otherwise, the scheduler will not find eligible
hosts. In the example provided, the resource class is ``ironic-gold``.
Enroll ironic nodes Another set of flavor properties must be used to disable scheduling based on
------------------- standard properties for a bare metal flavor:
#. From the utility container, enroll a new baremetal node by executing the .. code-block:: bash
following:
.. code-block:: bash openstack flavor set --property resources:VCPU=0 baremetal-gold
openstack flavor set --property resources:MEMORY_MB=0 baremetal-gold
openstack flavor set --property resources:DISK_GB=0 baremetal-gold
# Source credentials Lastly, a ``boot_option`` capability can be set to speed up booting after
. ~/openrc the deployment:
# Create the node .. code-block:: bash
NODE_HOSTNAME="myfirstnodename"
IPMI_ADDRESS="10.1.2.3"
IPMI_USER="my-ipmi-user"
IPMI_PASSWORD="my-ipmi-password"
KERNEL_IMAGE=$(glance image-list | awk "/${IMAGE_NAME}.kernel/ {print \$2}")
INITRAMFS_IMAGE=$(glance image-list | awk "/${IMAGE_NAME}.initramfs/ {print \$2}")
ironic node-create \
-d agent_ipmitool \
-i ipmi_address="${IPMI_ADDRESS}" \
-i ipmi_username="${IPMI_USER}" \
-i ipmi_password="${IPMI_PASSWORD}" \
-i deploy_ramdisk="${INITRAMFS_IMAGE}" \
-i deploy_kernel="${KERNEL_IMAGE}" \
-n ${NODE_HOSTNAME}
# Create a port for the node openstack flavor set --property capabilities:'boot_option=local' baremetal-gold
NODE_MACADDRESS="aa:bb:cc:dd:ee:ff"
ironic port-create \
-n $(ironic node-list | awk "/${NODE_HOSTNAME}/ {print \$2}") \
-a ${NODE_MACADDRESS}
# Associate an image to the node .. note::
ROOT_DISK_SIZE_GB=40
ironic node-update $(ironic node-list | awk "/${IMAGE_NAME}/ {print \$2}") add \
driver_info/deploy_kernel=$KERNEL_IMAGE \
driver_info/deploy_ramdisk=$INITRAMFS_IMAGE \
instance_info/deploy_kernel=$KERNEL_IMAGE \
instance_info/deploy_ramdisk=$INITRAMFS_IMAGE \
instance_info/root_gb=${ROOT_DISK_SIZE_GB}
# Add node properties Specifying the ``local`` boot option allows the deployed baremetal
# The property values used here should match the hardware used instance to boot directly to disk instead of network.
ironic node-update $(ironic node-list | awk "/${NODE_HOSTNAME}/ {print \$2}") add \
properties/cpus=48 \
properties/memory_mb=254802 \
properties/local_gb=80 \
properties/size=3600 \
properties/cpu_arch=x86_64 \
properties/capabilities=memory_mb:254802,local_gb:80,cpu_arch:x86_64,cpus:48,boot_option:local
Deploy a baremetal node kicked with ironic Enrolling Ironic Nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
Enrolling baremetal nodes makes then available to the Ironic service. The
properties of a given node will allow Ironic to determine how an image should
be deployed on the node, including using IPMI or vendor-specific out-of-band
interfaces. Some properties are optional, and may rely on defaults set by
the operator or within OpenStack-Ansible. Others are required, and may be
noted as such.
Some things should be known about the baremetal node prior to enrollment,
including:
* Node Name
* Driver
* Deploy Interface (based on driver)
* Provisioning Interface (MAC Address)
* IPMI or OOB Credentials
* OOB Management IP
* Deploy Kernel Image UUID (from Glance)
* Deploy Ramdisk Image UUID (from Glance)
* Boot Mode (bios or uefi)
* Network Interface (flat or neutron)
.. note::
Kernel and ramdisk images may be provided by the ``diskimage-builder``
process, or may be downloaded from opendev.org:
`<https://tarballs.opendev.org/openstack/ironic-python-agent/dib/>`_
`<https://docs.openstack.org/ironic/latest/install/deploy-ramdisk.html>`_
.. important:: .. important::
You will not have access unless you have a key set within nova before The deploy kernel and ramdisk should be updated on a regular basis
your ironic deployment. If you do not have an ssh key readily to match the OpenStack release of the underlying infrastructure. The
available, set one up with ``ssh-keygen``. Ironic Python Agent that runs on the ramdisk interfaces with Ironic
APIs, and should be kept in sync.
To enroll a node, use the ``openstack baremetal node create`` command. The
example below demonstrates the creation of a baremetal node with the
following characteristics:
.. code-block:: bash .. code-block:: bash
nova keypair-add --pub-key ~/.ssh/id_rsa.pub admin node_name=baremetal01
node_mac="f0:92:1c:0c:1f:88" # MAC address of PXE interface (em1 as example)
deploy_aki=ironic-deploy-aki # Kernel image
deploy_ari=ironic-deploy-ari # Ramdisk image
resource=ironic-gold # Ironic resource class (matches flavor as CUSTOM_IRONIC_GOLD)
phys_arch=x86_64
phys_cpus=32
phys_ram=32768
phys_disk=270
ipmi_username=root
ipmi_password=calvin
ipmi_address=172.19.0.22
boot_mode=bios
network_interface=flat
Now boot a node: .. important::
The Ironic conductor service must be able to communicate with the OOB IP
address to perform provisioning functions.
.. code-block:: bash .. code-block:: bash
nova boot --flavor ${FLAVOR_NAME} --image ${IMAGE_NAME} --key-name admin ${NODE_NAME} openstack baremetal node create \
--driver ipmi \
--deploy-interface direct \
--driver-info ipmi_username=$ipmi_username \
--driver-info ipmi_password=$ipmi_password \
--driver-info ipmi_address=$ipmi_address \
--driver-info deploy_kernel=`openstack image show $deploy_aki -c id |awk '/id / {print $4}'` \
--driver-info deploy_ramdisk=`openstack image show $deploy_ari -c id |awk '/id / {print $4}'` \
--property cpus=$phys_cpus \
--property memory_mb=$phys_ram \
--property local_gb=$phys_disk \
--property cpu_arch=$phys_arch \
--property capabilities='boot_option:local,disk_label:gpt' \
--resource-class $resource \
--network-interface $network_interface \
--name $node_name
The node will first appear in an ``enroll`` state. To make it available for
provisioning, set the state to ``manage``, then ``available``:
.. code-block:: bash
openstack baremetal node manage baremetal01
openstack baremetal node provide baremetal01
openstack baremetal node list --fit
+--------------------------------------+-------------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+-------------+---------------+-------------+--------------------+-------------+
| c362890d-5d7a-4dc3-ad29-7dac0bf49344 | baremetal01 | None | power off | available | False |
+--------------------------------------+-------------+---------------+-------------+--------------------+-------------+
Next, create a baremetal port using the ``openstack baremetal port create``
command:
.. code-block:: bash
node_name=baremetal01
node_mac="f0:92:1c:0c:1f:88"
openstack baremetal port create $node_mac \
--node `openstack baremetal node show $node_name -c uuid |awk -F "|" '/ uuid / {print $3}'`
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| address | f0:92:1c:0c:1f:88 |
| created_at | 2021-12-17T20:36:19+00:00 |
| extra | {} |
| internal_info | {} |
| is_smartnic | False |
| local_link_connection | {} |
| node_uuid | c362890d-5d7a-4dc3-ad29-7dac0bf49344 |
| physical_network | None |
| portgroup_uuid | None |
| pxe_enabled | True |
| updated_at | None |
| uuid | 44e5d872-ffa5-45f5-a5aa-7147c523e593 |
+-----------------------+--------------------------------------+
.. note::
The baremetal port is used to setup Neutron to provide DHCP services
during provisioning. When the ``neutron`` network interface is used,
the respective switchport can be managed by OpenStack.
Deploy a Baremetal Node Using Ironic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Baremetal instances can be deployed using the ``openstack server create``
command and a baremetal flavor. Unless the image has been created with
support for passwords, an SSH key must be provided. The baremetal instance
relies on Neutron DHCP and metadata services, just like a virtual instance.
.. code-block:: bash
openstack server create \
--flavor baremetal-gold \
--image focal-server-cloudimg-amd64 \
--key-name myKey \
--network myBaremetalNetwork \
myBaremetalInstance
.. important::
If you do not have an ssh key readily available, set one up with
``ssh-keygen`` and/or create one with ``openstack keypair create``.
Otherwise, you will not be able to connect to the deployed instance.