Self-Service via Runbooks

Adds runbooks; the new API feature that makes it possible for
project members to self-serve maintenance tasks through curated step
lists associated with target nodes via traits.

In addition to basic CRUD support, runbook extends current API flow for
performing manual cleaning and servicing to support runbooks in lieu of
an explicit/arbitrary ``clean_steps`` and ``service_steps`` user-defined
lists.

Demo Video: https://youtu.be/00PJS4SXFYQ
Closes-Bug: #2027690

Change-Id: I43555ef72cb882adcada2ed875fda40eed0dd034

api-ref/source/baremetal-api-v1-runbooks.inc

@@ -0,0 +1,245 @@
+.. -*- rst -*-
+
+===================
+Runbooks (runbooks)
+===================
+
+The Runbook resource represents a collection of steps that define a
+series of actions to be executed on a node. Runbooks enable users to perform
+complex operations in a predefined, automated manner. A runbook is
+matched for a node if the runbook's name matches a trait in the node.
+
+.. versionadded:: 1.92
+    Runbook API was introduced.
+
+Create Runbook
+==============
+
+.. rest_method::  POST /v1/runbooks
+
+Creates a runbook.
+
+.. versionadded:: 1.92
+    Runbook API was introduced.
+
+Normal response codes: 201
+
+Error response codes: 400, 401, 403, 409
+
+Request
+-------
+
+.. rest_parameters:: parameters.yaml
+
+   - name: runbook_name
+   - steps: runbook_steps
+   - disable_ramdisk: req_disable_ramdisk
+   - uuid: req_uuid
+   - extra: req_extra
+
+Request Step
+------------
+
+.. rest_parameters:: parameters.yaml
+
+   - interface: runbook_step_interface
+   - step: runbook_step_step
+   - args: runbook_step_args
+   - order: runbook_step_order
+
+Request Example
+---------------
+
+.. literalinclude:: samples/runbook-create-request.json
+   :language: javascript
+
+Response Parameters
+-------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - uuid: uuid
+   - name: runbook_name
+   - steps: runbook_steps
+   - disable_ramdisk: disable_ramdisk
+   - extra: extra
+   - public: runbook_public
+   - owner: runbook_owner
+   - created_at: created_at
+   - updated_at: updated_at
+   - links: links
+
+Response Example
+----------------
+
+.. literalinclude:: samples/runbook-create-response.json
+   :language: javascript
+
+List Runbooks
+=============
+
+.. rest_method::  GET /v1/runbooks
+
+Lists all runbooks.
+
+.. versionadded:: 1.92
+    Runbook API was introduced.
+
+Normal response codes: 200
+
+Error response codes: 400, 401, 403, 404
+
+Request
+-------
+
+.. rest_parameters:: parameters.yaml
+
+   - fields: fields
+   - limit: limit
+   - marker: marker
+   - sort_dir: sort_dir
+   - sort_key: sort_key
+   - detail: detail
+
+Response Parameters
+-------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - uuid: uuid
+   - name: runbook_name
+   - disable_ramdisk: disable_ramdisk
+   - steps: runbook_steps
+   - extra: extra
+   - public: runbook_public
+   - owner: runbook_owner
+   - created_at: created_at
+   - updated_at: updated_at
+   - links: links
+
+Response Example
+----------------
+
+**Example runbook list response:**
+
+.. literalinclude:: samples/runbook-list-response.json
+   :language: javascript
+
+**Example detailed runbook list response:**
+
+.. literalinclude:: samples/runbook-detail-response.json
+   :language: javascript
+
+Show Runbook Details
+====================
+
+.. rest_method::  GET /v1/runbooks/{runbook_id}
+
+Shows details for a runbook.
+
+.. versionadded:: 1.92
+    Runbook API was introduced.
+
+Normal response codes: 200
+
+Error response codes: 400, 401, 403, 404
+
+Request
+-------
+
+.. rest_parameters:: parameters.yaml
+
+   - fields: fields
+   - runbook_id: runbook_ident
+
+Response Parameters
+-------------------
+
+.. rest_parameters:: parameters.yaml
+
+   - uuid: uuid
+   - name: runbook_name
+   - steps: runbook_steps
+   - disable_ramdisk: disable_ramdisk
+   - extra: extra
+   - public: runbook_public
+   - owner: runbook_owner
+   - created_at: created_at
+   - updated_at: updated_at
+   - links: links
+
+Response Example
+----------------
+
+.. literalinclude:: samples/runbook-show-response.json
+   :language: javascript
+
+Update a Runbook
+================
+
+.. rest_method:: PATCH /v1/runbooks/{runbook_id}
+
+Update a runbook.
+
+.. versionadded:: 1.92
+    Runbook API was introduced.
+
+Normal response code: 200
+
+Error response codes: 400, 401, 403, 404, 409
+
+Request
+-------
+
+The BODY of the PATCH request must be a JSON PATCH document, adhering to
+`RFC 6902 <https://tools.ietf.org/html/rfc6902>`_.
+
+Request
+-------
+
+.. rest_parameters:: parameters.yaml
+
+    - runbook_id: runbook_ident
+
+.. literalinclude:: samples/runbook-update-request.json
+   :language: javascript
+
+Response
+--------
+
+.. rest_parameters:: parameters.yaml
+
+   - uuid: uuid
+   - name: runbook_name
+   - steps: runbook_steps
+   - disable_ramdisk: disable_ramdisk
+   - extra: extra
+   - public: runbook_public
+   - owner: runbook_owner
+   - created_at: created_at
+   - updated_at: updated_at
+   - links: links
+
+.. literalinclude:: samples/runbook-update-response.json
+   :language: javascript
+
+Delete Runbook
+==============
+
+.. rest_method::  DELETE /v1/runbooks/{runbook_id}
+
+Deletes a runbook.
+
+.. versionadded:: 1.92
+    Runbook API was introduced.
+
+Normal response codes: 204
+
+Error response codes: 400, 401, 403, 404
+
+Request
+-------
+
+.. rest_parameters:: parameters.yaml
+
+  - runbook_id: runbook_ident

doc/source/admin/cleaning.rst

@@ -209,6 +209,15 @@ In the above example, the node's RAID interface would configure hardware
 RAID without non-root volumes, and then all devices would be erased
 (in that order).
 
+Alternatively, you can specify a runbook instead of clean_steps::
+
+  {
+    "target":"clean",
+    "runbook": "<runbook_name_or_uuid>"
+  }
+
+The specified runbook must match one of the node's traits to be used.
+
 Starting manual cleaning via "openstack metal" CLI
 ------------------------------------------------------
 
@@ -246,6 +255,24 @@ Or with stdin::
     cat my-clean-steps.txt | baremetal node clean <node> \
         --clean-steps -
 
+To use a runbook instead of specifying clean steps:
+
+    baremetal node clean <node> --runbook <runbook_name_or_uuid>
+
+Runbooks for Manual Cleaning
+----------------------------
+Instead of passing a list of clean steps, operators can now use runbooks.
+Runbooks are curated lists of steps that can be associated with nodes via
+traits which simplifies the process of performing consistent cleaning
+operations across similar nodes.
+
+To use a runbook for manual cleaning:
+
+    baremetal node clean <node> --runbook <runbook_name_or_uuid>
+
+Runbooks must be created and associated with nodes beforehand. Only runbooks
+that match the node's traits can be used for cleaning that node.
+
 Cleaning Network
 ================
 

doc/source/admin/servicing.rst

@@ -109,6 +109,15 @@ configuration, and then the vendor interface's ``send_raw`` step would be
 called to send a raw command to the BMC. Please note, ``send_raw`` is only
 available for the ``ipmi`` hardware type.
 
+Alternatively, you can specify a runbook instead of service_steps::
+
+  {
+    "target":"service",
+    "runbook": "<runbook_name_or_uuid>"
+  }
+
+The specified runbook must match one of the node's traits to be used.
+
 Starting servicing via "openstack baremetal" CLI
 ------------------------------------------------
 
@@ -137,6 +146,23 @@ Or with stdin::
     cat my-clean-steps.txt | baremetal node service <node> \
         --service-steps -
 
+To use a runbook instead of specifying service steps:
+
+    baremetal node service <node> --runbook <runbook_name_or_uuid>
+
+Using Runbooks for Servicing
+----------------------------
+Similar to manual cleaning, you can use runbooks for node servicing.
+Runbooks provide a predefined list of service steps associated with nodes
+via traits.
+
+To use a runbook for servicing:
+
+    baremetal node service <node> --runbook <runbook_name_or_uuid>
+
+Ensure that the runbook matches one of the node's traits before using it
+for servicing.
+
 Available Steps in Ironic
 -------------------------
 

ironic/api/controllers/v1/__init__.py

@@ -36,6 +36,7 @@ from ironic.api.controllers.v1 import node
 from ironic.api.controllers.v1 import port
 from ironic.api.controllers.v1 import portgroup
 from ironic.api.controllers.v1 import ramdisk
+from ironic.api.controllers.v1 import runbook
 from ironic.api.controllers.v1 import shard
 from ironic.api.controllers.v1 import utils
 from ironic.api.controllers.v1 import versions
@@ -77,6 +78,7 @@ VERSIONED_CONTROLLERS = {
     'events': utils.allow_expose_events,
     'deploy_templates': utils.allow_deploy_templates,
     'shards': utils.allow_shards_endpoint,
+    'runbooks': utils.allow_runbooks,
     # NOTE(dtantsur): continue_inspection is available in 1.1 as a
     # compatibility hack to make it usable with IPA without changes.
     # Hide this fact from consumers since it was not actually available
@@ -131,6 +133,7 @@ class Controller(object):
         'deploy_templates': deploy_template.DeployTemplatesController(),
         'shards': shard.ShardController(),
         'continue_inspection': ramdisk.ContinueInspectionController(),
+        'runbooks': runbook.RunbooksController()
     }
 
     @method.expose()

ironic/api/controllers/v1/deploy_template.py

@@ -10,7 +10,6 @@
 # License for the specific language governing permissions and limitations
 # under the License.
 
-import collections
 from http import client as http_client
 
 from ironic_lib import metrics_utils
@@ -57,46 +56,13 @@ PATCH_ALLOWED_FIELDS = ['extra', 'name', 'steps', 'description']
 STEP_PATCH_ALLOWED_FIELDS = ['args', 'interface', 'priority', 'step']
 
 
-def duplicate_steps(name, value):
-    """Argument validator to check template for duplicate steps"""
-    # TODO(mgoddard): Determine the consequences of allowing duplicate
-    # steps.
-    # * What if one step has zero priority and another non-zero?
-    # * What if a step that is enabled by default is included in a
-    #   template? Do we override the default or add a second invocation?
-
-    # Check for duplicate steps. Each interface/step combination can be
-    # specified at most once.
-    counter = collections.Counter((step['interface'], step['step'])
-                                  for step in value['steps'])
-    duplicates = {key for key, count in counter.items() if count > 1}
-    if duplicates:
-        duplicates = {"interface: %s, step: %s" % (interface, step)
-                      for interface, step in duplicates}
-        err = _("Duplicate deploy steps. A deploy template cannot have "
-                "multiple deploy steps with the same interface and step. "
-                "Duplicates: %s") % "; ".join(duplicates)
-        raise exception.InvalidDeployTemplate(err=err)
-    return value
-
-
 TEMPLATE_VALIDATOR = args.and_valid(
     args.schema(TEMPLATE_SCHEMA),
-    duplicate_steps,
+    api_utils.duplicate_steps,
     args.dict_valid(uuid=args.uuid)
 )
 
 
-def convert_steps(rpc_steps):
-    for step in rpc_steps:
-        yield {
-            'interface': step['interface'],
-            'step': step['step'],
-            'args': step['args'],
-            'priority': step['priority'],
-        }
-
-
 def convert_with_links(rpc_template, fields=None, sanitize=True):
     """Add links to the deploy template."""
     template = api_utils.object_to_dict(
@@ -104,7 +70,7 @@ def convert_with_links(rpc_template, fields=None, sanitize=True):
         fields=('name', 'extra'),
         link_resource='deploy_templates',
     )
-    template['steps'] = list(convert_steps(rpc_template.steps))
+    template['steps'] = list(api_utils.convert_steps(rpc_template.steps))
 
     if fields is not None:
         api_utils.check_for_invalid_fields(fields, template)

ironic/api/controllers/v1/node.py

@@ -86,6 +86,10 @@ _STEPS_SCHEMA = {
                 "type": "object",
                 "properties": {}
             },
+            'order': {'anyOf': [
+                {'type': 'integer', 'minimum': 0},
+                {'type': 'string', 'minLength': 1, 'pattern': '^[0-9]+$'}
+            ]},
             "execute_on_child_nodes": {
                 "description": "Boolean if the step should be executed "
                                "on child nodes.",
@@ -988,6 +992,41 @@ class NodeStatesController(rest.RestController):
         url_args = '/'.join([node_ident, 'states'])
         api.response.location = link.build_url('nodes', url_args)
 
+    def _handle_runbook(self, rpc_node, target, runbook, clean_steps,
+                        service_steps):
+        if not api_utils.allow_runbooks():
+            raise exception.NotAcceptable()
+
+        rpc_runbook = api_utils.check_runbook_policy_and_retrieve(
+            policy_name='baremetal:runbook:use',
+            runbook_ident=runbook)
+
+        node_traits = rpc_node.traits.get_trait_names() or []
+        if rpc_runbook.name not in node_traits:
+            msg = (_('This runbook has not been approved for '
+                     'use on this node %s. Please ask an administrator '
+                     'to add it to your node traits.') % rpc_node.uuid)
+            raise exception.ClientSideError(
+                msg, status_code=http_client.BAD_REQUEST)
+
+        disable_ramdisk = rpc_runbook.disable_ramdisk
+        if target == ir_states.VERBS['clean']:
+            if clean_steps:
+                msg = (_('Please provide either "clean_steps" or a '
+                         'runbook, but not both.'))
+                raise exception.ClientSideError(
+                    msg, status_code=http_client.BAD_REQUEST)
+            clean_steps = list(api_utils.convert_steps(rpc_runbook.steps))
+        elif target == ir_states.VERBS['service']:
+            if service_steps:
+                msg = (_('Please provide either "service_steps" or a '
+                         'runbook, but not both.'))
+                raise exception.ClientSideError(
+                    msg, status_code=http_client.BAD_REQUEST)
+            service_steps = list(api_utils.convert_steps(
+                rpc_runbook.steps))
+        return clean_steps, service_steps, disable_ramdisk
+
     def _do_provision_action(self, rpc_node, target, configdrive=None,
                              clean_steps=None, deploy_steps=None,
                              rescue_password=None, disable_ramdisk=None,
@@ -1061,11 +1100,12 @@ class NodeStatesController(rest.RestController):
                    deploy_steps=args.types(type(None), list),
                    rescue_password=args.string,
                    disable_ramdisk=args.boolean,
-                   service_steps=args.types(type(None), list))
+                   service_steps=args.types(type(None), list),
+                   runbook=args.types(type(None), str))
     def provision(self, node_ident, target, configdrive=None,
                   clean_steps=None, deploy_steps=None,
                   rescue_password=None, disable_ramdisk=None,
-                  service_steps=None):
+                  service_steps=None, runbook=None):
         """Asynchronous trigger the provisioning of the node.
 
         This will set the target provision state of the node, and a
@@ -1142,6 +1182,7 @@ class NodeStatesController(rest.RestController):
                 'args': {'force': True},
                 'priority': 90 }
 
+        :param runbook: UUID or logical name of a runbook.
         :raises: NodeLocked (HTTP 409) if the node is currently locked.
         :raises: ClientSideError (HTTP 409) if the node is already being
                  provisioned.
@@ -1187,9 +1228,26 @@ class NodeStatesController(rest.RestController):
         api_utils.check_allow_configdrive(target, configdrive)
         api_utils.check_allow_clean_disable_ramdisk(target, disable_ramdisk)
 
+        if runbook:
+            clean_steps, service_steps, disable_ramdisk = self._handle_runbook(
+                rpc_node, target, runbook, clean_steps, service_steps
+            )
+        else:
+            if clean_steps:
+                api_utils.check_policy(
+                    'baremetal:node:set_provision_state:clean_steps')
+            if service_steps:
+                api_utils.check_policy(
+                    'baremetal:node:set_provision_state:service_steps')
+
         if clean_steps and target != ir_states.VERBS['clean']:
             msg = (_('"clean_steps" is only valid when setting target '
                      'provision state to %s') % ir_states.VERBS['clean'])
+            if runbook:
+                rb_allowed_targets = [ir_states.VERBS['clean'],
+                                      ir_states.VERBS['service']]
+                msg = (_('"runbooks" is only valid when setting target '
+                         'provision state to any of %s') % rb_allowed_targets)
             raise exception.ClientSideError(
                 msg, status_code=http_client.BAD_REQUEST)
 
@@ -1214,6 +1272,17 @@ class NodeStatesController(rest.RestController):
             if not api_utils.allow_unhold_verb():
                 raise exception.NotAcceptable()
 
+        if service_steps and target != ir_states.VERBS['service']:
+            msg = (_('"service_steps" is only valid when setting target '
+                     'provision state to %s') % ir_states.VERBS['service'])
+            if runbook:
+                rb_allowed_targets = [ir_states.VERBS['clean'],
+                                      ir_states.VERBS['service']]
+                msg = (_('"runbooks" is only valid when setting target '
+                         'provision state to any of %s') % rb_allowed_targets)
+            raise exception.ClientSideError(
+                msg, status_code=http_client.BAD_REQUEST)
+
         if target == ir_states.VERBS['service']:
             if not api_utils.allow_service_verb():
                 raise exception.NotAcceptable()

ironic/api/controllers/v1/notification_utils.py

@@ -28,6 +28,7 @@ from ironic.objects import node as node_objects
 from ironic.objects import notification
 from ironic.objects import port as port_objects
 from ironic.objects import portgroup as portgroup_objects
+from ironic.objects import runbook as runbook_objects
 from ironic.objects import volume_connector as volume_connector_objects
 from ironic.objects import volume_target as volume_target_objects
 
@@ -48,6 +49,8 @@ CRUD_NOTIFY_OBJ = {
              port_objects.PortCRUDPayload),
     'portgroup': (portgroup_objects.PortgroupCRUDNotification,
                   portgroup_objects.PortgroupCRUDPayload),
+    'runbook': (runbook_objects.RunbookCRUDNotification,
+                runbook_objects.RunbookCRUDPayload),
     'volumeconnector':
         (volume_connector_objects.VolumeConnectorCRUDNotification,
          volume_connector_objects.VolumeConnectorCRUDPayload),

ironic/api/controllers/v1/runbook.py

@@ -0,0 +1,391 @@
+# Licensed under the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License. You may obtain
+# a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations
+# under the License.
+
+from http import client as http_client
+
+from ironic_lib import metrics_utils
+from oslo_log import log
+from oslo_utils import strutils
+from oslo_utils import uuidutils
+import pecan
+from pecan import rest
+from webob import exc as webob_exc
+
+from ironic import api
+from ironic.api.controllers import link
+from ironic.api.controllers.v1 import collection
+from ironic.api.controllers.v1 import notification_utils as notify
+from ironic.api.controllers.v1 import utils as api_utils
+from ironic.api import method
+from ironic.common import args
+from ironic.common import exception
+from ironic.common.i18n import _
+import ironic.conf
+from ironic import objects
+
+
+CONF = ironic.conf.CONF
+LOG = log.getLogger(__name__)
+METRICS = metrics_utils.get_metrics_logger(__name__)
+
+DEFAULT_RETURN_FIELDS = ['uuid', 'name']
+
+RUNBOOK_SCHEMA = {
+    'type': 'object',
+    'properties': {
+        'uuid': {'type': ['string', 'null']},
+        'name': api_utils.TRAITS_SCHEMA,
+        'description': {'type': ['string', 'null'], 'maxLength': 255},
+        'steps': {
+            'type': 'array',
+            'items': api_utils.RUNBOOK_STEP_SCHEMA,
+            'minItems': 1},
+        'disable_ramdisk': {'type': ['boolean', 'null']},
+        'extra': {'type': ['object', 'null']},
+        'public': {'type': ['boolean', 'null']},
+        'owner': {'type': ['string', 'null'], 'maxLength': 255}
+    },
+    'required': ['steps', 'name'],
+    'additionalProperties': False,
+}
+
+PATCH_ALLOWED_FIELDS = [
+    'extra',
+    'name',
+    'steps',
+    'description',
+    'public',
+    'owner'
+]
+STEP_PATCH_ALLOWED_FIELDS = ['args', 'interface', 'order', 'step']
+
+
+RUNBOOK_VALIDATOR = args.and_valid(
+    args.schema(RUNBOOK_SCHEMA),
+    api_utils.duplicate_steps,
+    args.dict_valid(uuid=args.uuid)
+)
+
+
+def convert_with_links(rpc_runbook, fields=None, sanitize=True):
+    """Add links to the runbook."""
+    runbook = api_utils.object_to_dict(
+        rpc_runbook,
+        fields=('name', 'extra', 'public', 'owner', 'disable_ramdisk'),
+        link_resource='runbooks',
+    )
+    runbook['steps'] = list(api_utils.convert_steps(rpc_runbook.steps))
+
+    if fields is not None:
+        api_utils.check_for_invalid_fields(fields, runbook)
+
+    if sanitize:
+        runbook_sanitize(runbook, fields)
+
+    return runbook
+
+
+def runbook_sanitize(runbook, fields):
+    """Removes sensitive and unrequested data.
+
+    Will only keep the fields specified in the ``fields`` parameter.
+
+    :param fields:
+        list of fields to preserve, or ``None`` to preserve them all
+    :type fields: list of str
+    """
+    api_utils.sanitize_dict(runbook, fields)
+    if runbook.get('steps'):
+        for step in runbook['steps']:
+            step_sanitize(step)
+
+
+def step_sanitize(step):
+    if step.get('args'):
+        step['args'] = strutils.mask_dict_password(step['args'], "******")
+
+
+def list_convert_with_links(rpc_runbooks, limit, fields=None, **kwargs):
+    return collection.list_convert_with_links(
+        items=[convert_with_links(t, fields=fields, sanitize=False)
+               for t in rpc_runbooks],
+        item_name='runbooks',
+        url='runbooks',
+        limit=limit,
+        fields=fields,
+        sanitize_func=runbook_sanitize,
+        **kwargs
+    )
+
+
+class RunbooksController(rest.RestController):
+    """REST controller for runbooks."""
+
+    invalid_sort_key_list = ['extra', 'steps']
+
+    @pecan.expose()
+    def _route(self, args, request=None):
+        if not api_utils.allow_runbooks():
+            msg = _("The API version does not allow runbooks")
+            if api.request.method == "GET":
+                raise webob_exc.HTTPNotFound(msg)
+            else:
+                raise webob_exc.HTTPMethodNotAllowed(msg)
+        return super(RunbooksController, self)._route(args, request)
+
+    @METRICS.timer('RunbooksController.get_all')
+    @method.expose()
+    @args.validate(marker=args.name, limit=args.integer, sort_key=args.string,
+                   sort_dir=args.string, fields=args.string_list,
+                   detail=args.boolean, project=args.boolean)
+    def get_all(self, marker=None, limit=None, sort_key='id', sort_dir='asc',
+                fields=None, detail=None, project=None):
+        """Retrieve a list of runbooks.
+
+        :param marker: pagination marker for large data sets.
+        :param limit: maximum number of resources to return in a single result.
+                      This value cannot be larger than the value of max_limit
+                      in the [api] section of the ironic configuration, or only
+                      max_limit resources will be returned.
+        :param project: Optional string value that set the project
+                        whose runbooks are to be returned.
+        :param sort_key: column to sort results by. Default: id.
+        :param sort_dir: direction to sort. "asc" or "desc". Default: asc.
+        :param fields: Optional, a list with a specified set of fields
+                       of the resource to be returned.
+        :param detail: Optional, boolean to indicate whether retrieve a list
+                       of runbooks with detail.
+        """
+        if not api_utils.allow_runbooks():
+            raise exception.NotFound()
+
+        project_id = api_utils.check_list_policy('runbook', project)
+
+        api_utils.check_allowed_fields(fields)
+        api_utils.check_allowed_fields([sort_key])
+
+        fields = api_utils.get_request_return_fields(fields, detail,
+                                                     DEFAULT_RETURN_FIELDS)
+
+        limit = api_utils.validate_limit(limit)
+        sort_dir = api_utils.validate_sort_dir(sort_dir)
+
+        if sort_key in self.invalid_sort_key_list:
+            raise exception.InvalidParameterValue(
+                _("The sort_key value %(key)s is an invalid field for "
+                  "sorting") % {'key': sort_key})
+
+        filters = {}
+        if project_id:
+            filters['project'] = project_id
+
+        marker_obj = None
+        if marker:
+            marker_obj = objects.Runbook.get_by_uuid(
+                api.request.context, marker)
+
+        runbooks = objects.Runbook.list(
+            api.request.context, limit=limit, marker=marker_obj,
+            sort_key=sort_key, sort_dir=sort_dir, filters=filters)
+
+        parameters = {'sort_key': sort_key, 'sort_dir': sort_dir}
+
+        if detail is not None:
+            parameters['detail'] = detail
+
+        return list_convert_with_links(
+            runbooks, limit, fields=fields, **parameters)
+
+    @METRICS.timer('RunbooksController.get_one')
+    @method.expose()
+    @args.validate(runbook_ident=args.uuid_or_name, fields=args.string_list)
+    def get_one(self, runbook_ident, fields=None):
+        """Retrieve information about the given runbook.
+
+        :param runbook_ident: UUID or logical name of a runbook.
+        :param fields: Optional, a list with a specified set of fields
+            of the resource to be returned.
+        """
+        if not api_utils.allow_runbooks():
+            raise exception.NotFound()
+
+        try:
+            rpc_runbook = api_utils.check_runbook_policy_and_retrieve(
+                'baremetal:runbook:get', runbook_ident)
+        except exception.NotAuthorized:
+            # If the user is not authorized to access the runbook,
+            # check also, if the runbook is public
+            rpc_runbook = api_utils.check_and_retrieve_public_runbook(
+                runbook_ident)
+
+        api_utils.check_allowed_fields(fields)
+        return convert_with_links(rpc_runbook, fields=fields)
+
+    @METRICS.timer('RunbooksController.post')
+    @method.expose(status_code=http_client.CREATED)
+    @method.body('runbook')
+    @args.validate(runbook=RUNBOOK_VALIDATOR)
+    def post(self, runbook):
+        """Create a new runbook.
+
+        :param runbook: a runbook within the request body.
+        """
+        if not api_utils.allow_runbooks():
+            raise exception.NotFound()
+
+        context = api.request.context
+        api_utils.check_policy('baremetal:runbook:create')
+
+        cdict = context.to_policy_values()
+        if cdict.get('system_scope') != 'all':
+            project_id = None
+            requested_owner = runbook.get('owner', None)
+            if cdict.get('project_id', False):
+                project_id = cdict.get('project_id')
+
+            if requested_owner and requested_owner != project_id:
+                # Translation: If project scoped, and an owner has been
+                # requested, and that owner does not match the requester's
+                # project ID value.
+                msg = _("Cannot create a runbook as a project scoped admin "
+                        "with an owner other than your own project.")
+                raise exception.Invalid(msg)
+
+            if project_id and runbook.get('public', False):
+                msg = _("Cannot create a public runbook as a project scoped "
+                        "admin.")
+                raise exception.Invalid(msg)
+            # Finally, note the project ID
+            runbook['owner'] = project_id
+
+        if not runbook.get('uuid'):
+            runbook['uuid'] = uuidutils.generate_uuid()
+        new_runbook = objects.Runbook(context, **runbook)
+
+        notify.emit_start_notification(context, new_runbook, 'create')
+        with notify.handle_error_notification(context, new_runbook, 'create'):
+            new_runbook.create()
+
+        # Set the HTTP Location Header
+        api.response.location = link.build_url('runbooks', new_runbook.uuid)
+        api_runbook = convert_with_links(new_runbook)
+        notify.emit_end_notification(context, new_runbook, 'create')
+        return api_runbook
+
+    def _authorize_patch_and_get_runbook(self, runbook_ident, patch):
+        # deal with attribute-specific policy rules
+        policy_checks = []
+        generic_update = False
+
+        paths_to_policy = (
+            ('/owner', 'baremetal:runbook:update:owner'),
+            ('/public', 'baremetal:runbook:update:public'),
+        )
+        for p in patch:
+            # Process general direct path to policy map
+            rule_match_found = False
+            for check_path, policy_name in paths_to_policy:
+                if p['path'].startswith(check_path):
+                    policy_checks.append(policy_name)
+                    # Break, policy found
+                    rule_match_found = True
+                    break
+            if not rule_match_found:
+                generic_update = True
+
+        if generic_update or not policy_checks:
+            # If we couldn't find specific policy to apply,
+            # apply the update policy check.
+            policy_checks.append('baremetal:runbook:update')
+        return api_utils.check_multiple_runbook_policies_and_retrieve(
+            policy_checks, runbook_ident)
+
+    @METRICS.timer('RunbooksController.patch')
+    @method.expose()
+    @method.body('patch')
+    @args.validate(runbook_ident=args.uuid_or_name, patch=args.patch)
+    def patch(self, runbook_ident, patch=None):
+        """Update an existing runbook.
+
+        :param runbook_ident: UUID or logical name of a runbook.
+        :param patch: a json PATCH document to apply to this runbook.
+        """
+        if not api_utils.allow_runbooks():
+            raise exception.NotFound()
+
+        api_utils.patch_validate_allowed_fields(patch, PATCH_ALLOWED_FIELDS)
+
+        context = api.request.context
+
+        rpc_runbook = self._authorize_patch_and_get_runbook(runbook_ident,
+                                                            patch)
+        runbook = rpc_runbook.as_dict()
+
+        owner = api_utils.get_patch_values(patch, '/owner')
+        public = api_utils.get_patch_values(patch, '/public')
+
+        if owner:
+            # NOTE(cid): There should not be an owner for a public runbook,
+            # but an owned runbook can be set to non-public and assigned an
+            # owner atomically
+            public_value = public[0] if public else False
+            if runbook.get('public') and (not public) or public_value:
+                msg = _("There cannot be an owner for a public runbook")
+                raise exception.PatchError(patch=patch, reason=msg)
+
+        if public:
+            runbook['owner'] = None
+
+        # apply the patch
+        runbook = api_utils.apply_jsonpatch(runbook, patch)
+
+        # validate the result with the patch schema
+        for step in runbook.get('steps', []):
+            api_utils.patched_validate_with_schema(
+                step, api_utils.RUNBOOK_STEP_SCHEMA)
+        api_utils.patched_validate_with_schema(
+            runbook, RUNBOOK_SCHEMA, RUNBOOK_VALIDATOR)
+
+        api_utils.patch_update_changed_fields(
+            runbook, rpc_runbook, fields=objects.Runbook.fields,
+            schema=RUNBOOK_SCHEMA
+        )
+
+        notify.emit_start_notification(context, rpc_runbook, 'update')
+        with notify.handle_error_notification(context, rpc_runbook, 'update'):
+            rpc_runbook.save()
+
+        api_runbook = convert_with_links(rpc_runbook)
+        notify.emit_end_notification(context, rpc_runbook, 'update')
+
+        return api_runbook
+
+    @METRICS.timer('RunbooksController.delete')
+    @method.expose(status_code=http_client.NO_CONTENT)
+    @args.validate(runbook_ident=args.uuid_or_name)
+    def delete(self, runbook_ident):
+        """Delete a runbook.
+
+        :param runbook_ident: UUID or logical name of a runbook.
+        """
+        if not api_utils.allow_runbooks():
+            raise exception.NotFound()
+
+        rpc_runbook = api_utils.check_runbook_policy_and_retrieve(
+            policy_name='baremetal:runbook:delete',
+            runbook_ident=runbook_ident)
+
+        context = api.request.context
+        notify.emit_start_notification(context, rpc_runbook, 'delete')
+        with notify.handle_error_notification(context, rpc_runbook, 'delete'):
+            rpc_runbook.destroy()
+        notify.emit_end_notification(context, rpc_runbook, 'delete')

ironic/api/controllers/v1/utils.py

@@ -13,6 +13,7 @@
 #    License for the specific language governing permissions and limitations
 #    under the License.
 
+import collections
 import copy
 from http import client as http_client
 import inspect
@@ -158,6 +159,24 @@ DEPLOY_STEP_SCHEMA = {
     'additionalProperties': False,
 }
 
+RUNBOOK_STEP_SCHEMA = {
+    'type': 'object',
+    'properties': {
+        'args': {'type': 'object'},
+        'interface': {
+            'type': 'string',
+            'enum': list(conductor_steps.CLEANING_INTERFACE_PRIORITY)
+        },
+        'step': {'type': 'string', 'minLength': 1},
+        'order': {'anyOf': [
+            {'type': 'integer', 'minimum': 0},
+            {'type': 'string', 'minLength': 1, 'pattern': '^[0-9]+$'}
+        ]}
+    },
+    'required': ['interface', 'step', 'order'],
+    'additionalProperties': False,
+}
+
 
 def local_link_normalize(name, value):
     if not value:
@@ -685,6 +704,43 @@ def get_rpc_deploy_template_with_suffix(template_ident):
                             exception.DeployTemplateNotFound)
 
 
+def get_rpc_runbook(runbook_ident):
+    """Get the RPC runbook from the UUID or logical name.
+
+    :param runbook_ident: the UUID or logical name of a runbook.
+
+    :returns: The RPC runbook.
+    :raises: InvalidUuidOrName if the name or uuid provided is not valid.
+    :raises: RunbookNotFound if the runbook is not found.
+    """
+    # If runbook_ident is instead a valid UUID, treat it as a UUID.
+    if uuidutils.is_uuid_like(runbook_ident):
+        return objects.Runbook.get_by_uuid(api.request.context,
+                                           runbook_ident)
+
+    # Else, we can refer to runbooks by their name too
+    if utils.is_valid_logical_name(runbook_ident):
+        return objects.Runbook.get_by_name(api.request.context,
+                                           runbook_ident)
+    raise exception.InvalidUuidOrName(name=runbook_ident)
+
+
+def check_runbook_policy_and_retrieve(policy_name, runbook_ident):
+    """Check if the specified policy authorizes this request on a node.
+
+    :param: policy_name: Name of the policy to check.
+    :param: runbook_ident: the UUID or logical name of a runbook.
+
+    :raises: HTTPForbidden if the policy forbids access.
+    :raises: RunbookNotFound if the runbook is not found.
+    :return: a runbook object
+    """
+    rpc_runbook = get_rpc_runbook(runbook_ident)
+    check_owner_policy(object_type='runbook', policy_name=policy_name,
+                       owner=rpc_runbook['owner'])
+    return rpc_runbook
+
+
 def is_valid_node_name(name):
     """Determine if the provided name is a valid node name.
 
@@ -1517,6 +1573,53 @@ def check_policy_true(policy_name):
     return policy.check_policy(policy_name, cdict, api.request.context)
 
 
+def duplicate_steps(name, value):
+    """Argument validator to check template for duplicate steps"""
+    # TODO(mgoddard): Determine the consequences of allowing duplicate
+    # steps.
+    # * What if one step has zero priority and another non-zero?
+    # * What if a step that is enabled by default is included in a
+    #   template? Do we override the default or add a second invocation?
+
+    # Check for duplicate steps. Each interface/step combination can be
+    # specified at most once.
+    counter = collections.Counter((step['interface'], step['step'])
+                                  for step in value['steps'])
+    duplicates = {key for key, count in counter.items() if count > 1}
+    if duplicates:
+        duplicates = {"interface: %s, step: %s" % (interface, step)
+                      for interface, step in duplicates}
+        err = _("Duplicate deploy steps. A template cannot have multiple "
+                "deploy steps with the same interface and step. "
+                "Duplicates: %s") % "; ".join(duplicates)
+        raise exception.InvalidDeployTemplate(err=err)
+    return value
+
+
+def convert_steps(rpc_steps):
+    for step in rpc_steps:
+        result = {
+            'interface': step['interface'],
+            'step': step['step'],
+            'args': step['args'],
+        }
+
+        if 'priority' in step:
+            result['priority'] = step['priority']
+        elif 'order' in step:
+            result['order'] = step['order']
+
+        yield result
+
+
+def allow_runbooks():
+    """Check if accessing runbook endpoints is allowed.
+
+    Version 1.92 of the API exposed runbook endpoints.
+    """
+    return api.request.version.minor >= versions.MINOR_92_RUNBOOKS
+
+
 def check_owner_policy(object_type, policy_name, owner, lessee=None,
                        conceal_node=False):
     """Check if the policy authorizes this request on an object.
@@ -1547,6 +1650,19 @@ def check_owner_policy(object_type, policy_name, owner, lessee=None,
             raise
 
 
+def check_and_retrieve_public_runbook(runbook_ident):
+    """If policy authorization check fails, check if runbook is public.
+
+    :param: runbook_ident: the UUID or logical name of a runbook.
+    :raises: HTTPForbidden if runbook is not public.
+    :return: RPC runbook identified by runbook_ident
+    """
+    rpc_runbook = get_rpc_runbook(runbook_ident)
+    if not rpc_runbook.public:
+        raise exception.HTTPForbidden
+    return rpc_runbook
+
+
 def check_node_policy_and_retrieve(policy_name, node_ident,
                                    with_suffix=False):
     """Check if the specified policy authorizes this request on a node.
@@ -1635,6 +1751,27 @@ def check_multiple_node_policies_and_retrieve(policy_names,
     return rpc_node
 
 
+def check_multiple_runbook_policies_and_retrieve(policy_names,
+                                                 runbook_ident):
+    """Check if the specified policies authorize this request on a runbook.
+
+    :param: policy_names: List of policy names to check.
+    :param: runbook_ident: the UUID or logical name of a runbook.
+
+    :raises: HTTPForbidden if the policy forbids access.
+    :raises: RunbookNotFound if the runbook is not found.
+    :return: RPC runbook identified by runbook_ident
+    """
+    rpc_runbook = None
+    for policy_name in policy_names:
+        if rpc_runbook is None:
+            rpc_runbook = check_runbook_policy_and_retrieve(policy_names[0],
+                                                            runbook_ident)
+        else:
+            check_owner_policy('runbook', policy_name, rpc_runbook['owner'])
+    return rpc_runbook
+
+
 def check_list_policy(object_type, owner=None):
     """Check if the list policy authorizes this request on an object.
 

ironic/api/controllers/v1/versions.py

@@ -129,6 +129,7 @@ BASE_VERSION = 1
 # v1.89: Add API for attaching/detaching virtual media
 # v1.90: Accept ovn vtep switch metadata schema to port.local_link_connection
 # v1.91: Remove special treatment of .json for API objects
+# v1.92: Add runbooks API
 
 MINOR_0_JUNO = 0
 MINOR_1_INITIAL_VERSION = 1
@@ -222,6 +223,7 @@ MINOR_88_PORT_NAME = 88
 MINOR_89_ATTACH_DETACH_VMEDIA = 89
 MINOR_90_OVN_VTEP = 90
 MINOR_91_DOT_JSON = 91
+MINOR_92_RUNBOOKS = 92
 
 # When adding another version, update:
 # - MINOR_MAX_VERSION
@@ -229,7 +231,7 @@ MINOR_91_DOT_JSON = 91
 #   explanation of what changed in the new version
 # - common/release_mappings.py, RELEASE_MAPPING['master']['api']
 
-MINOR_MAX_VERSION = MINOR_91_DOT_JSON
+MINOR_MAX_VERSION = MINOR_92_RUNBOOKS
 
 # String representations of the minor and maximum versions
 _MIN_VERSION_STRING = '{}.{}'.format(BASE_VERSION, MINOR_1_INITIAL_VERSION)

ironic/common/exception.py

@@ -716,6 +716,22 @@ class InvalidDeployTemplate(Invalid):
     _msg_fmt = _("Deploy template invalid: %(err)s.")
 
 
+class RunbookDuplicateName(Conflict):
+    _msg_fmt = _("A runbook with name %(name)s already exists.")
+
+
+class RunbookAlreadyExists(Conflict):
+    _msg_fmt = _("A runbook with UUID %(uuid)s already exists.")
+
+
+class RunbookNotFound(NotFound):
+    _msg_fmt = _("Runbook %(runbook)s could not be found.")
+
+
+class InvalidRunbook(Invalid):
+    _msg_fmt = _("Runbook invalid: %(err)s.")
+
+
 class InvalidKickstartTemplate(Invalid):
     _msg_fmt = _("The kickstart template is missing required variables")
 

ironic/common/policy.py

@@ -49,7 +49,7 @@ SYSTEM_ADMIN = 'role:admin and system_scope:all'
 
 # Generic policy check string for system users who don't require all the
 # authorization that system administrators typically have. This persona, or
-# check string, typically isn't used by default, but it's existence it useful
+# check string, typically isn't used by default, but it's existence is useful
 # in the event a deployment wants to offload some administrative action from
 # system administrator to system members.
 # The rule:service_role match here is to enable an elevated level of API
@@ -59,7 +59,7 @@ SYSTEM_MEMBER = '(role:member and system_scope:all) or rule:service_role'  # noq
 
 # Generic policy check string for read-only access to system-level
 # resources. This persona is useful for someone who needs access
-# for auditing or even support. These uses are also able to view
+# for auditing or even support. These users are also able to view
 # project-specific resources where applicable (e.g., listing all
 # volumes in the deployment, regardless of the project they belong to).
 # The rule:service_role match here is to enable an elevated level of API
@@ -126,6 +126,24 @@ ALLOCATION_OWNER_MANAGER = ('role:manager and project_id:%(allocation.owner)s')
 ALLOCATION_OWNER_MEMBER = ('role:member and project_id:%(allocation.owner)s')
 ALLOCATION_OWNER_READER = ('role:reader and project_id:%(allocation.owner)s')
 
+# Members can create/destroy their runbooks.
+RUNBOOK_OWNER_ADMIN = ('role:admin and project_id:%(runbook.owner)s')
+RUNBOOK_OWNER_MANAGER = ('role:manager and project_id:%(runbook.owner)s')
+RUNBOOK_OWNER_MEMBER = ('role:member and project_id:%(runbook.owner)s')
+RUNBOOK_OWNER_READER = ('role:reader and project_id:%(runbook.owner)s')
+
+RUNBOOK_ADMIN = (
+    '(' + SYSTEM_MEMBER + ') or (' + RUNBOOK_OWNER_MANAGER + ') or role:service' # noqa
+)
+
+RUNBOOK_READER = (
+    '(' + SYSTEM_READER + ') or (' + RUNBOOK_OWNER_READER + ') or role:service' # noqa
+)
+
+RUNBOOK_CREATOR = (
+    '(' + SYSTEM_MEMBER + ') or role:manager or role:service' # noqa
+)
+
 # Used for general operations like changing provision state.
 SYSTEM_OR_OWNER_MEMBER_AND_LESSEE_ADMIN = (
     '(' + SYSTEM_MEMBER + ') or (' + SYSTEM_SERVICE + ') or (' + PROJECT_OWNER_MEMBER + ') or (' + PROJECT_LESSEE_ADMIN + ') or (' + PROJECT_LESSEE_MANAGER + ') or (' + PROJECT_SERVICE + ')'  # noqa
@@ -862,6 +880,24 @@ node_policies = [
         ],
         deprecated_rule=deprecated_node_set_provision_state
     ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:node:set_provision_state:clean_steps',
+        check_str=SYSTEM_OR_OWNER_MEMBER_AND_LESSEE_ADMIN,
+        scope_types=['system', 'project'],
+        description='Allow execution of arbitrary steps on a node',
+        operations=[
+            {'path': '/nodes/{node_ident}/states/provision', 'method': 'PUT'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:node:set_provision_state:service_steps',
+        check_str=SYSTEM_OR_OWNER_MEMBER_AND_LESSEE_ADMIN,
+        scope_types=['system', 'project'],
+        description='Allow execution of arbitrary steps on a node',
+        operations=[
+            {'path': '/nodes/{node_ident}/states/provision', 'method': 'PUT'}
+        ],
+    ),
     policy.DocumentedRuleDefault(
         name='baremetal:node:set_raid_state',
         check_str=SYSTEM_MEMBER_OR_OWNER_MEMBER,
@@ -1880,6 +1916,89 @@ deploy_template_policies = [
     ),
 ]
 
+runbook_policies = [
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:get',
+        check_str=RUNBOOK_READER,
+        scope_types=['system', 'project'],
+        description='Retrieve a single runbook record',
+        operations=[
+            {'path': '/runbooks/{runbook_ident}', 'method': 'GET'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:list',
+        check_str=API_READER,
+        scope_types=['system', 'project'],
+        description='Retrieve multiple runbook records, filtered by '
+                    'an explicit owner or the client project_id',
+        operations=[
+            {'path': '/runbooks', 'method': 'GET'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:list_all',
+        check_str=SYSTEM_READER,
+        scope_types=['system', 'project'],
+        description='Retrieve all runbook records',
+        operations=[
+            {'path': '/runbooks', 'method': 'GET'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:create',
+        check_str=RUNBOOK_CREATOR,
+        scope_types=['system', 'project'],
+        description='Create Runbook records',
+        operations=[{'path': '/runbooks', 'method': 'POST'}],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:delete',
+        check_str=RUNBOOK_ADMIN,
+        scope_types=['system', 'project'],
+        description='Delete a runbook record',
+        operations=[
+            {'path': '/runbooks/{runbook_ident}', 'method': 'DELETE'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:update',
+        check_str=RUNBOOK_ADMIN,
+        scope_types=['system', 'project'],
+        description='Update a runbook record',
+        operations=[
+            {'path': '/runbooks/{runbook_ident}', 'method': 'PATCH'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:update:public',
+        check_str=SYSTEM_MEMBER,
+        scope_types=['system', 'project'],
+        description='Set and unset a runbook as public',
+        operations=[
+            {'path': '/runbooks/{runbook_ident}/public', 'method': 'PATCH'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:update:owner',
+        check_str=SYSTEM_MEMBER,
+        scope_types=['system', 'project'],
+        description='Set and unset the owner of a runbook',
+        operations=[
+            {'path': '/runbooks/{runbook_ident}/owner', 'method': 'PATCH'}
+        ],
+    ),
+    policy.DocumentedRuleDefault(
+        name='baremetal:runbook:use',
+        check_str=RUNBOOK_ADMIN,
+        scope_types=['system', 'project'],
+        description='Allowed to use a runbook for node operations',
+        operations=[
+            {'path': '/nodes/{node_ident}/states/provision', 'method': 'PUT'}
+        ],
+    )
+]
+
 
 def list_policies():
     policies = itertools.chain(
@@ -1896,6 +2015,7 @@ def list_policies():
         allocation_policies,
         event_policies,
         deploy_template_policies,
+        runbook_policies,
     )
     return policies
 

ironic/common/release_mappings.py

@@ -709,7 +709,7 @@ RELEASE_MAPPING = {
     # make it below. To release, we will preserve a version matching
     # the release as a separate block of text, like above.
     'master': {
-        'api': '1.91',
+        'api': '1.92',
         'rpc': '1.60',
         'objects': {
             'Allocation': ['1.1'],
@@ -728,6 +728,7 @@ RELEASE_MAPPING = {
             'VolumeConnector': ['1.0'],
             'VolumeTarget': ['1.0'],
             'FirmwareComponent': ['1.0'],
+            'Runbook': ['1.0'],
         }
     },
 }

ironic/db/api.py

@@ -1347,6 +1347,101 @@ class Connection(object, metaclass=abc.ABCMeta):
         :returns: A list of deploy templates.
         """
 
+    @abc.abstractmethod
+    def create_runbook(self, values):
+        """Create a runbook.
+
+        :param values: A dict describing the runbook. For example:
+
+                     ::
+
+                      {
+                       'uuid': uuidutils.generate_uuid(),
+                       'name': 'CUSTOM_DT1',
+                      }
+        :raises: RunbookDuplicateName if a runbook with the same
+            name exists.
+        :raises: RunbookAlreadyExists if a runbook with the same
+            UUID exists.
+        :returns: A runbook.
+        """
+
+    @abc.abstractmethod
+    def update_runbook(self, runbook_id, values):
+        """Update a runbook.
+
+        :param runbook_id: ID of the runbook to update.
+        :param values: A dict describing the runbook. For example:
+
+                     ::
+
+                      {
+                       'uuid': uuidutils.generate_uuid(),
+                       'name': 'CUSTOM_DT1',
+                      }
+        :raises: RunbookDuplicateName if a runbook with the same
+            name exists.
+        :raises: RunbookNotFound if the runbook does not exist.
+        :returns: A runbook.
+        """
+
+    @abc.abstractmethod
+    def destroy_runbook(self, runbook_id):
+        """Destroy a runbook.
+
+        :param runbook_id: ID of the runbook to destroy.
+        :raises: RunbookNotFound if the runbook does not exist.
+        """
+
+    @abc.abstractmethod
+    def get_runbook_by_id(self, runbook_id):
+        """Retrieve a runbook by ID.
+
+        :param runbook_id: ID of the runbook to retrieve.
+        :raises: RunbookNotFound if the runbook does not exist.
+        :returns: A runbook.
+        """
+
+    @abc.abstractmethod
+    def get_runbook_by_uuid(self, runbook_uuid):
+        """Retrieve a runbook by UUID.
+
+        :param runbook_uuid: UUID of the runbook to retrieve.
+        :raises: RunbookNotFound if the runbook does not exist.
+        :returns: A runbook.
+        """
+
+    @abc.abstractmethod
+    def get_runbook_by_name(self, runbook_name):
+        """Retrieve a runbook by name.
+
+        :param runbook_name: name of the runbook to retrieve.
+        :raises: RunbookNotFound if the runbook does not exist.
+        :returns: A runbook.
+        """
+
+    @abc.abstractmethod
+    def get_runbook_list(self, limit=None, marker=None, filters=None,
+                         sort_key=None, sort_dir=None):
+        """Retrieve a list of runbooks.
+
+        :param limit: Maximum number of runbooks to return.
+        :param marker: The last item of the previous page; we return the next
+                       result set.
+        :param sort_key: Attribute by which results should be sorted.
+        :param sort_dir: Direction in which results should be sorted.
+                         (asc, desc)
+        :returns: A list of runbooks.
+        """
+
+    @abc.abstractmethod
+    def get_runbook_list_by_names(self, names):
+        """Return a list of runbooks with one of a list of names.
+
+        :param names: List of names to filter by.
+        :returns: A list of runbooks.
+        """
+
     @abc.abstractmethod
     def create_node_history(self, values):
         """Create a new history record.

ironic/db/sqlalchemy/alembic/versions/66bd9c5604d5_add_runbook_and_runbook_step.py

@@ -0,0 +1,70 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Create runbooks and runbook_steps tables
+
+Revision ID: 66bd9c5604d5
+Revises: 01f21d5e5195
+Create Date: 2024-05-29 19:33:53.268794
+
+"""
+
+from alembic import op
+import sqlalchemy as sa
+
+# revision identifiers, used by Alembic.
+revision = '66bd9c5604d5'
+down_revision = '01f21d5e5195'
+
+
+def upgrade():
+    op.create_table(
+        'runbooks',
+        sa.Column('version', sa.String(length=15), nullable=True),
+        sa.Column('created_at', sa.DateTime(), nullable=True),
+        sa.Column('updated_at', sa.DateTime(), nullable=True),
+        sa.Column('id', sa.Integer(), nullable=False,
+                  autoincrement=True),
+        sa.Column('uuid', sa.String(length=36)),
+        sa.Column('name', sa.String(length=255), nullable=False),
+        sa.Column('disable_ramdisk', sa.Boolean, default=False),
+        sa.Column('public', sa.Boolean, default=False),
+        sa.Column('owner', sa.String(length=255), nullable=True),
+        sa.Column('extra', sa.Text(), nullable=True),
+        sa.PrimaryKeyConstraint('id'),
+        sa.UniqueConstraint('uuid', name='uniq_runbooks0uuid'),
+        sa.UniqueConstraint('name', name='uniq_runbooks0name'),
+        mysql_engine='InnoDB',
+        mysql_charset='UTF8MB3'
+    )
+    op.create_table(
+        'runbook_steps',
+        sa.Column('version', sa.String(length=15), nullable=True),
+        sa.Column('created_at', sa.DateTime(), nullable=True),
+        sa.Column('updated_at', sa.DateTime(), nullable=True),
+        sa.Column('id', sa.Integer(), nullable=False,
+                  autoincrement=True),
+        sa.Column('runbook_id', sa.Integer(), nullable=False,
+                  autoincrement=False),
+        sa.Column('interface', sa.String(length=255), nullable=False),
+        sa.Column('step', sa.String(length=255), nullable=False),
+        sa.Column('args', sa.Text, nullable=False),
+        sa.Column('order', sa.Integer, nullable=False),
+        sa.PrimaryKeyConstraint('id'),
+        sa.ForeignKeyConstraint(['runbook_id'],
+                                ['runbooks.id']),
+        sa.Index('runbook_id', 'runbook_id'),
+        sa.Index('runbook_steps_interface_idx', 'interface'),
+        sa.Index('runbook_steps_step_idx', 'step'),
+        mysql_engine='InnoDB',
+        mysql_charset='UTF8MB3'
+    )

ironic/db/sqlalchemy/api.py

@@ -169,6 +169,16 @@ def _get_deploy_template_select_with_steps():
     ).options(selectinload(models.DeployTemplate.steps))
 
 
+def _get_runbook_select_with_steps():
+    """Return a select object for the Runbook joined with steps.
+
+    :returns: a select object.
+    """
+    return sa.select(
+        models.Runbook
+    ).options(selectinload(models.Runbook.steps))
+
+
 def model_query(model, *args, **kwargs):
     """Query helper for simpler session usage.
 
@@ -471,6 +481,13 @@ class Connection(api.Connection):
                      | set(_NODE_IN_QUERY_FIELDS)
                      | set(_NODE_NON_NULL_FILTERS))
 
+    _RUNBOOK_QUERY_FIELDS = {'id', 'uuid', 'name', 'public', 'owner',
+                             'disable_ramdisk'}
+    _RUNBOOK_IN_QUERY_FIELDS = {'%s_in' % field: field
+                                for field in ('id', 'uuid', 'name')}
+    _RUNBOOK_FILTERS = ({'project'} | _RUNBOOK_QUERY_FIELDS
+                        | set(_RUNBOOK_IN_QUERY_FIELDS))
+
     def __init__(self):
         pass
 
@@ -541,6 +558,31 @@ class Connection(api.Connection):
         # a full list of both parents and children being conveyed.
         return query
 
+    def _validate_runbooks_filters(self, filters):
+        if filters is None:
+            filters = dict()
+        unsupported_filters = set(filters).difference(self._RUNBOOK_FILTERS)
+        if unsupported_filters:
+            msg = _("SqlAlchemy API does not support "
+                    "filtering by %s") % ', '.join(unsupported_filters)
+            raise ValueError(msg)
+        return filters
+
+    def _add_runbooks_filters(self, query, filters):
+        filters = self._validate_runbooks_filters(filters)
+        for field in self._RUNBOOK_QUERY_FIELDS:
+            if field in filters:
+                query = query.filter_by(**{field: filters[field]})
+        for key, field in self._RUNBOOK_IN_QUERY_FIELDS.items():
+            if key in filters:
+                query = query.filter(
+                    getattr(models.Runbook, field).in_(filters[key]))
+        if 'project' in filters:
+            project = filters['project']
+            query = query.filter((models.Runbook.owner == project)
+                                 | (models.Runbook.public))
+        return query
+
     def _add_allocations_filters(self, query, filters):
         if filters is None:
             filters = dict()
@@ -2628,6 +2670,171 @@ class Connection(api.Connection):
             ).all()
             return [r[0] for r in res]
 
+    @staticmethod
+    def _get_runbook_steps(steps, runbook_id=None):
+        results = []
+        for values in steps:
+            step = models.RunbookStep()
+            step.update(values)
+            if runbook_id:
+                step['runbook_id'] = runbook_id
+            results.append(step)
+        return results
+
+    @oslo_db_api.retry_on_deadlock
+    def create_runbook(self, values):
+        steps = values.get('steps', [])
+        values['steps'] = self._get_runbook_steps(steps)
+
+        runbook = models.Runbook()
+        runbook.update(values)
+        with _session_for_write() as session:
+            try:
+                session.add(runbook)
+                session.flush()
+            except db_exc.DBDuplicateEntry as e:
+                if 'name' in e.columns:
+                    raise exception.RunbookDuplicateName(
+                        name=values['name'])
+                raise exception.RunbookAlreadyExists(
+                    uuid=values['uuid'])
+        return runbook
+
+    def _update_runbook_steps(self, session, runbook_id, steps):
+        """Update the steps for a runbook.
+
+        :param session: DB session object.
+        :param runbook_id: runbook ID.
+        :param steps: list of steps that should exist for the runbook.
+        """
+
+        def _step_key(step):
+            """Compare two runbook steps."""
+            # NOTE(mgoddard): In python 3, dicts are not orderable so cannot be
+            # used as a sort key. Serialise the step arguments to a JSON string
+            # for comparison. Taken from https://stackoverflow.com/a/22003440.
+            sortable_args = json.dumps(step.args, sort_keys=True)
+            return step.interface, step.step, sortable_args, step.order
+
+        # List all existing steps for the runbook.
+        current_steps = (session.query(models.RunbookStep)
+                         .filter_by(runbook_id=runbook_id))
+
+        # List the new steps for the runbook.
+        new_steps = self._get_runbook_steps(steps, runbook_id)
+
+        # The following is an efficient way to ensure that the steps in the
+        # database match those that have been requested. We compare the current
+        # and requested steps in a single pass using the _zip_matching
+        # function.
+        steps_to_create = []
+        step_ids_to_delete = []
+        for current_step, new_step in _zip_matching(current_steps, new_steps,
+                                                    _step_key):
+            if current_step is None:
+                # No matching current step found for this new step - create.
+                steps_to_create.append(new_step)
+            elif new_step is None:
+                # No matching new step found for this current step - delete.
+                step_ids_to_delete.append(current_step.id)
+            # else: steps match, no work required.
+
+        # Delete and create steps in bulk as necessary.
+        if step_ids_to_delete:
+            ((session.query(models.RunbookStep)
+              .filter(models.RunbookStep.id.in_(step_ids_to_delete)))
+             .delete(synchronize_session=False))
+        if steps_to_create:
+            session.bulk_save_objects(steps_to_create)
+
+    @oslo_db_api.retry_on_deadlock
+    def update_runbook(self, runbook_id, values):
+        if 'uuid' in values:
+            msg = _("Cannot overwrite UUID for an existing runbook.")
+            raise exception.InvalidParameterValue(err=msg)
+
+        try:
+            with _session_for_write() as session:
+                # NOTE(mgoddard): Don't issue a joined query for the update as
+                # this does not work with PostgreSQL.
+                query = session.query(models.Runbook)
+                query = add_identity_filter(query, runbook_id)
+                ref = query.with_for_update().one()
+                # First, update non-step columns.
+                steps = values.pop('steps', None)
+                ref.update(values)
+                # If necessary, update steps.
+                if steps is not None:
+                    self._update_runbook_steps(session, ref.id, steps)
+                session.flush()
+
+            with _session_for_read() as session:
+                # Return the updated runbook joined with all relevant fields.
+                query = _get_runbook_select_with_steps()
+                query = add_identity_filter(query, runbook_id)
+                res = session.execute(query).one()[0]
+            return res
+        except db_exc.DBDuplicateEntry as e:
+            if 'name' in e.columns:
+                raise exception.RunbookDuplicateName(
+                    name=values['name'])
+            raise
+        except NoResultFound:
+            # TODO(TheJulia): What would unified core raise?!?
+            raise exception.RunbookNotFound(
+                runbook=runbook_id)
+
+    @oslo_db_api.retry_on_deadlock
+    def destroy_runbook(self, runbook_id):
+        with _session_for_write() as session:
+            session.query(models.RunbookStep).filter_by(
+                runbook_id=runbook_id).delete()
+            count = session.query(models.Runbook).filter_by(
+                id=runbook_id).delete()
+            if count == 0:
+                raise exception.RunbookNotFound(runbook=runbook_id)
+
+    def _get_runbook(self, field, value):
+        """Helper method for retrieving a runbook."""
+        query = (_get_runbook_select_with_steps()
+                 .where(field == value))
+        try:
+            with _session_for_read() as session:
+                res = session.execute(query).one()[0]
+            return res
+        except NoResultFound:
+            raise exception.RunbookNotFound(runbook=value)
+
+    def get_runbook_by_id(self, runbook_id):
+        return self._get_runbook(models.Runbook.id,
+                                 runbook_id)
+
+    def get_runbook_by_uuid(self, runbook_uuid):
+        return self._get_runbook(models.Runbook.uuid,
+                                 runbook_uuid)
+
+    def get_runbook_by_name(self, runbook_name):
+        return self._get_runbook(models.Runbook.name,
+                                 runbook_name)
+
+    def get_runbook_list(self, limit=None, marker=None, filters=None,
+                         sort_key=None, sort_dir=None):
+        query = (sa.select(models.Runbook)
+                 .options(selectinload(models.Runbook.steps)))
+        query = self._add_runbooks_filters(query, filters)
+        return _paginate_query(models.Runbook, limit, marker,
+                               sort_key, sort_dir, query)
+
+    def get_runbook_list_by_names(self, names):
+        query = _get_runbook_select_with_steps()
+        with _session_for_read() as session:
+            res = session.execute(
+                query.where(
+                    models.Runbook.name.in_(names)
+                )
+            ).all()
+            return [r[0] for r in res]
+
     @oslo_db_api.retry_on_deadlock
     def create_node_history(self, values):
         values['uuid'] = uuidutils.generate_uuid()

ironic/db/sqlalchemy/models.py

@@ -516,6 +516,51 @@ class FirmwareComponent(Base):
     last_version_flashed = Column(String(255), nullable=True)
 
 
+class Runbook(Base):
+    """Represents a runbook."""
+
+    __tablename__ = 'runbooks'
+    __table_args__ = (
+        schema.UniqueConstraint('uuid', name='uniq_runbooks0uuid'),
+        schema.UniqueConstraint('name', name='uniq_runbooks0name'),
+        table_args())
+    id = Column(Integer, primary_key=True)
+    uuid = Column(String(36))
+    name = Column(String(255), nullable=False)
+    public = Column(Boolean, default=False)
+    owner = Column(String(255), nullable=True)
+    disable_ramdisk = Column(Boolean, default=False)
+    extra = Column(db_types.JsonEncodedDict)
+    steps: orm.Mapped[List['RunbookStep']] = orm.relationship(  # noqa
+        "RunbookStep",
+        back_populates="runbook",
+        lazy="selectin")
+
+
+class RunbookStep(Base):
+    """Represents a deployment step in a runbook."""
+
+    __tablename__ = 'runbook_steps'
+    __table_args__ = (
+        Index('runbook_id', 'runbook_id'),
+        Index('runbook_steps_interface_idx', 'interface'),
+        Index('runbook_steps_step_idx', 'step'),
+        table_args())
+    id = Column(Integer, primary_key=True)
+    runbook_id = Column(Integer, ForeignKey('runbooks.id'), nullable=False)
+    interface = Column(String(255), nullable=False)
+    step = Column(String(255), nullable=False)
+    args = Column(db_types.JsonEncodedDict, nullable=False)
+    order = Column(Integer, nullable=False)
+    runbook = orm.relationship(
+        "Runbook",
+        primaryjoin=(
+            'and_(RunbookStep.runbook_id == '
+            'Runbook.id)'),
+        foreign_keys=runbook_id
+    )
+
+
 def get_class(model_name):
     """Returns the model class with the specified name.
 

ironic/objects/__init__.py

@@ -36,6 +36,7 @@ def register_all():
     __import__('ironic.objects.node_inventory')
     __import__('ironic.objects.port')
     __import__('ironic.objects.portgroup')
+    __import__('ironic.objects.runbook')
     __import__('ironic.objects.trait')
     __import__('ironic.objects.volume_connector')
     __import__('ironic.objects.volume_target')

ironic/objects/runbook.py

@@ -0,0 +1,252 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from oslo_versionedobjects import base as object_base
+
+from ironic.db import api as db_api
+from ironic.objects import base
+from ironic.objects import fields as object_fields
+from ironic.objects import notification
+
+
+@base.IronicObjectRegistry.register
+class Runbook(base.IronicObject, object_base.VersionedObjectDictCompat):
+    # Version 1.0: Initial version
+    VERSION = '1.0'
+
+    dbapi = db_api.get_instance()
+
+    fields = {
+        'id': object_fields.IntegerField(),
+        'uuid': object_fields.UUIDField(nullable=False),
+        'name': object_fields.StringField(nullable=False),
+        'steps': object_fields.ListOfFlexibleDictsField(nullable=False),
+        'disable_ramdisk': object_fields.BooleanField(default=False),
+        'extra': object_fields.FlexibleDictField(nullable=True),
+        'public': object_fields.BooleanField(default=False),
+        'owner': object_fields.StringField(nullable=True),
+    }
+
+    def create(self, context=None):
+        """Create a Runbook record in the DB.
+
+        :param context: security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context).
+        :raises: RunbookDuplicateName if a runbook with the same
+            name exists.
+        :raises: RunbookAlreadyExists if a runbook with the same
+            UUID exists.
+        """
+        values = self.do_version_changes_for_db()
+        db_template = self.dbapi.create_runbook(values)
+        self._from_db_object(self._context, self, db_template)
+
+    def save(self, context=None):
+        """Save updates to this Runbook.
+
+        Column-wise updates will be made based on the result of
+        self.what_changed().
+
+        :param context: Security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context)
+        :raises: RunbookDuplicateName if a runbook with the same
+            name exists.
+        :raises: RunbookNotFound if the runbook does not exist.
+        """
+        updates = self.do_version_changes_for_db()
+        db_template = self.dbapi.update_runbook(self.uuid, updates)
+        self._from_db_object(self._context, self, db_template)
+
+    def destroy(self):
+        """Delete the Runbook from the DB.
+
+        :param context: security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context).
+        :raises: RunbookNotFound if the runbook no longer
+            appears in the database.
+        """
+        self.dbapi.destroy_runbook(self.id)
+        self.obj_reset_changes()
+
+    @classmethod
+    def get_by_id(cls, context, runbook_id):
+        """Find a runbook based on its integer ID.
+
+        :param context: security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context).
+        :param runbook_id: The ID of a runbook.
+        :raises: RunbookNotFound if the runbook no longer
+            appears in the database.
+        :returns: a :class:`Runbook` object.
+        """
+        db_template = cls.dbapi.get_runbook_by_id(runbook_id)
+        template = cls._from_db_object(context, cls(), db_template)
+        return template
+
+    @classmethod
+    def get_by_uuid(cls, context, uuid):
+        """Find a runbook based on its UUID.
+
+        :param context: security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context).
+        :param uuid: The UUID of a runbook.
+        :raises: RunbookNotFound if the runbook no longer
+            appears in the database.
+        :returns: a :class:`Runbook` object.
+        """
+        db_template = cls.dbapi.get_runbook_by_uuid(uuid)
+        template = cls._from_db_object(context, cls(), db_template)
+        return template
+
+    @classmethod
+    def get_by_name(cls, context, name):
+        """Find a runbook based on its name.
+
+        :param context: security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context).
+        :param name: The name of a runbook.
+        :raises: RunbookNotFound if the runbook no longer
+            appears in the database.
+        :returns: a :class:`Runbook` object.
+        """
+        db_template = cls.dbapi.get_runbook_by_name(name)
+        template = cls._from_db_object(context, cls(), db_template)
+        return template
+
+    @classmethod
+    def list(cls, context, limit=None, marker=None, sort_key=None,
+             sort_dir=None, filters=None):
+        """Return a list of Runbook objects.
+
+        :param context: security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context).
+        :param limit: maximum number of resources to return in a single result.
+        :param marker: pagination marker for large data sets.
+        :param sort_key: column to sort results by.
+        :param sort_dir: direction to sort. "asc" or "desc".
+        :param filters: Filters to apply.
+        :returns: a list of :class:`Runbook` objects.
+        """
+        db_templates = cls.dbapi.get_runbook_list(limit=limit, marker=marker,
+                                                  sort_key=sort_key,
+                                                  sort_dir=sort_dir,
+                                                  filters=filters)
+        return cls._from_db_object_list(context, db_templates)
+
+    @classmethod
+    def list_by_names(cls, context, names):
+        """Return a list of Runbook objects matching a set of names.
+
+        :param context: security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Runbook(context).
+        :param names: a list of names to filter by.
+        :returns: a list of :class:`Runbook` objects.
+        """
+        db_templates = cls.dbapi.get_runbook_list_by_names(names)
+        return cls._from_db_object_list(context, db_templates)
+
+    def refresh(self, context=None):
+        """Loads updates for this runbook.
+
+        Loads a runbook with the same uuid from the database and
+        checks for updated attributes. Updates are applied from
+        the loaded template column by column, if there are any updates.
+
+        :param context: Security context. NOTE: This should only
+                        be used internally by the indirection_api,
+                        but, RPC requires context as the first
+                        argument, even though we don't use it.
+                        A context should be set when instantiating the
+                        object, e.g.: Port(context)
+        :raises: RunbookNotFound if the runbook no longer
+            appears in the database.
+        """
+        current = self.get_by_uuid(self._context, uuid=self.uuid)
+        self.obj_refresh(current)
+        self.obj_reset_changes()
+
+
+@base.IronicObjectRegistry.register
+class RunbookCRUDNotification(notification.NotificationBase):
+    """Notification emitted on runbook API operations."""
+    # Version 1.0: Initial version
+    VERSION = '1.0'
+
+    fields = {
+        'payload': object_fields.ObjectField('RunbookCRUDPayload')
+    }
+
+
+@base.IronicObjectRegistry.register
+class RunbookCRUDPayload(notification.NotificationPayloadBase):
+    # Version 1.0: Initial version
+    VERSION = '1.0'
+
+    SCHEMA = {
+        'created_at': ('runbook', 'created_at'),
+        'disable_ramdisk': ('runbook', 'disable_ramdisk'),
+        'extra': ('runbook', 'extra'),
+        'name': ('runbook', 'name'),
+        'owner': ('runbook', 'owner'),
+        'public': ('runbook', 'public'),
+        'steps': ('runbook', 'steps'),
+        'updated_at': ('runbook', 'updated_at'),
+        'uuid': ('runbook', 'uuid')
+    }
+
+    fields = {
+        'created_at': object_fields.DateTimeField(nullable=True),
+        'disable_ramdisk': object_fields.BooleanField(default=False),
+        'extra': object_fields.FlexibleDictField(nullable=True),
+        'name': object_fields.StringField(nullable=False),
+        'owner': object_fields.StringField(nullable=True),
+        'public': object_fields.BooleanField(default=False),
+        'steps': object_fields.ListOfFlexibleDictsField(nullable=False),
+        'updated_at': object_fields.DateTimeField(nullable=True),
+        'uuid': object_fields.UUIDField()
+    }
+
+    def __init__(self, runbook, **kwargs):
+        super(RunbookCRUDPayload, self).__init__(**kwargs)
+        self.populate_schema(runbook=runbook)

ironic/tests/unit/api/controllers/v1/test_node.py

@@ -7019,6 +7019,113 @@ ORHMKeXMO8fcK0By7CiMKwHSXCoEQgfQhWwpMdSsO8LgHCjh87DQc= """
         self.assertEqual('application/json', ret.content_type)
         self.assertEqual(http_client.BAD_REQUEST, ret.status_code)
 
+    @mock.patch.object(api_utils, 'check_runbook_policy_and_retrieve',
+                       autospec=True)
+    @mock.patch.object(rpcapi.ConductorAPI, 'do_node_service',
+                       autospec=True)
+    def test_service_with_runbooks(self, mock_dns, mock_policy):
+        objects.TraitList.create(self.context, self.node.id, ['CUSTOM_1'])
+        self.node.refresh
+
+        self.node.provision_state = states.SERVICEHOLD
+        self.node.save()
+
+        runbook = mock.Mock()
+        runbook.name = 'CUSTOM_1'
+        runbook.steps = [{"step": "upgrade_firmware", "interface": "deploy",
+                          "args": {}}]
+        mock_policy.return_value = runbook
+        ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
+                            {'target': states.VERBS['service'],
+                             'runbook': runbook.name},
+                            headers={api_base.Version.string:
+                                     str(api_v1.max_version())})
+        self.assertEqual(http_client.ACCEPTED, ret.status_code)
+        self.assertEqual(b'', ret.body)
+        mock_policy.assert_has_calls([mock.call('baremetal:runbook:use',
+                                                runbook.name)]),
+        mock_dns.assert_called_once_with(mock.ANY, mock.ANY,
+                                         self.node.uuid, runbook.steps,
+                                         mock.ANY, topic='test-topic')
+
+    @mock.patch.object(api_utils, 'check_runbook_policy_and_retrieve',
+                       autospec=True)
+    @mock.patch.object(rpcapi.ConductorAPI, 'do_node_clean', autospec=True)
+    @mock.patch.object(api_node, '_check_clean_steps', autospec=True)
+    def test_clean_with_runbooks(self, mock_check, mock_rpcapi, mock_policy):
+        objects.TraitList.create(self.context, self.node.id, ['CUSTOM_1'])
+        self.node.refresh
+
+        self.node.provision_state = states.MANAGEABLE
+        self.node.save()
+
+        step = {"step": "configure raid", "interface": "raid", "args": {},
+                "order": 1}
+
+        runbook = mock.Mock()
+        runbook.name = 'CUSTOM_1'
+        runbook.steps = [step]
+        mock_policy.return_value = runbook
+        ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
+                            {'target': states.VERBS['clean'],
+                             'runbook': runbook.name},
+                            headers={api_base.Version.string:
+                                     str(api_v1.max_version())})
+        self.assertEqual(http_client.ACCEPTED, ret.status_code)
+        self.assertEqual(b'', ret.body)
+        mock_policy.assert_has_calls([mock.call('baremetal:runbook:use',
+                                                runbook.name)]),
+        mock_check.assert_called_once_with(runbook.steps)
+        mock_rpcapi.assert_called_once_with(mock.ANY, mock.ANY, self.node.uuid,
+                                            runbook.steps, mock.ANY,
+                                            topic='test-topic')
+
+    @mock.patch.object(api_utils, 'check_runbook_policy_and_retrieve',
+                       autospec=True)
+    def test_service_with_runbooks_unapproved(self, mock_policy):
+        objects.TraitList.create(self.context, self.node.id, ['CUSTOM_2'])
+        self.node.refresh
+
+        self.node.provision_state = states.SERVICEHOLD
+        self.node.save()
+
+        runbook = mock.Mock()
+        runbook.name = 'CUSTOM_1'
+        runbook.steps = [{'step': 'meow', 'interface': 'raid', 'args': {},
+                          'order': 1}]
+        mock_policy.return_value = runbook
+
+        ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
+                            {'target': states.VERBS['service'],
+                             'runbook': runbook.name},
+                            expect_errors=True,
+                            headers={api_base.Version.string:
+                                     str(api_v1.max_version())})
+        self.assertEqual(http_client.BAD_REQUEST, ret.status_int)
+
+    @mock.patch.object(api_utils, 'check_runbook_policy_and_retrieve',
+                       autospec=True)
+    def test_clean_with_runbooks_unapproved(self, mock_policy):
+        objects.TraitList.create(self.context, self.node.id, ['CUSTOM_2'])
+        self.node.refresh
+
+        self.node.provision_state = states.MANAGEABLE
+        self.node.save()
+
+        runbook = mock.Mock()
+        runbook.name = 'CUSTOM_1'
+        runbook.steps = [{'step': 'meow', 'interface': 'deploy', 'args': {},
+                          'order': 1}]
+        mock_policy.return_value = runbook
+
+        ret = self.put_json('/nodes/%s/states/provision' % self.node.uuid,
+                            {'target': states.VERBS['clean'],
+                             'runbook': runbook.name},
+                            expect_errors=True,
+                            headers={api_base.Version.string:
+                                     str(api_v1.max_version())})
+        self.assertEqual(http_client.BAD_REQUEST, ret.status_int)
+
 
 class TestCheckCleanSteps(db_base.DbTestCase):
     def test__check_clean_steps_not_list(self):

ironic/tests/unit/api/controllers/v1/test_root.py

@@ -160,6 +160,12 @@ class TestV1Routing(api_base.BaseApiTest):
             'volume': [
                 {'href': 'http://localhost/v1/volume/', 'rel': 'self'},
                 {'href': 'http://localhost/volume/', 'rel': 'bookmark'}
+            ],
+            'runbooks': [
+                {'href': 'http://localhost/v1/runbooks/',
+                 'rel': 'self'},
+                {'href': 'http://localhost/runbooks/',
+                 'rel': 'bookmark'}
             ]
         }, response)
 

ironic/tests/unit/api/test_acl.py

@@ -298,6 +298,7 @@ class TestRBACModelBeforeScopesBase(TestACLBase):
         # false positives with test runners.
         db_utils.create_test_node(
             uuid='18a552fb-dcd2-43bf-9302-e4c93287be11')
+        fake_db_runbook = db_utils.create_test_runbook()
         self.format_data.update({
             'node_ident': fake_db_node['uuid'],
             'allocated_node_ident': fake_db_node_alloced['uuid'],
@@ -314,6 +315,7 @@ class TestRBACModelBeforeScopesBase(TestACLBase):
             'driver_name': 'fake-driverz',
             'bios_setting': fake_setting,
             'trait': fake_trait,
+            'runbook_ident': fake_db_runbook['uuid'],
             'volume_target_ident': fake_db_volume_target['uuid'],
             'volume_connector_ident': fake_db_volume_connector['uuid'],
             'history_ident': fake_history['uuid'],
@@ -391,6 +393,9 @@ class TestRBACProjectScoped(TestACLBase):
         lessee_project_id = 'f11853c7-fa9c-4db3-a477-c9d8e0dbbf13'
         unowned_node = db_utils.create_test_node(chassis_id=None)
 
+        fake_db_runbook = db_utils.create_test_runbook(
+            owner='70e5e25a-2ca2-4cb1-8ae8-7d8739cee205')
+
         # owned node - since the tests use the same node for
         # owner/lesse checks
         owned_node = db_utils.create_test_node(
@@ -496,6 +501,7 @@ class TestRBACProjectScoped(TestACLBase):
             'vif_ident': fake_vif_port_id,
             'ind_component': 'component',
             'ind_ident': 'magic_light',
+            'runbook_ident': fake_db_runbook['uuid'],
             'owner_port_ident': owned_node_port['uuid'],
             'other_port_ident': other_port['uuid'],
             'owner_portgroup_ident': owner_pgroup['uuid'],

ironic/tests/unit/api/test_rbac_project_scoped.yaml

@@ -3978,3 +3978,314 @@ service_cannot_get_firmware_components:
   method: get
   headers: *service_headers
   assert_status: 404
+
+# Runbooks - https://docs.openstack.org/api-ref/baremetal/#runbooks-templates
+
+runbooks_post_admin:
+  path: '/v1/runbooks'
+  method: post
+  body: &runbook_body
+    name: 'CUSTOM_NAME'
+    steps:
+      - interface: 'raid'
+        step: 'noop'
+        args: {}
+        order: 0
+  headers: *owner_admin_headers
+  assert_status: 201
+
+runbooks_post_manager:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *owner_manager_headers
+  assert_status: 201
+
+service_post_runbook:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *service_headers_owner_project
+  assert_status: 201
+
+third_party_admin_post_runbook:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *third_party_admin_headers
+  assert_status: 201
+
+runbooks_post_public_admin:
+  path: '/v1/runbooks'
+  method: post
+  body: &runbook_body_public
+    name: 'CUSTOM_NAME'
+    public: true
+    steps:
+      - interface: 'raid'
+        step: 'noop'
+        args: {}
+        order: 0
+  headers: *owner_admin_headers
+  assert_status: 400
+
+runbooks_post_public_admin:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body_public
+  headers: *owner_manager_headers
+  assert_status: 400
+
+runbooks_post_public_service:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body_public
+  headers: *owner_admin_headers
+  assert_status: 400
+
+runbooks_patch_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: &runbook_patch
+    - op: replace
+      path: /name
+      value: 'CUSTOM_NAME'
+  headers: *owner_admin_headers
+  assert_status: 200
+
+runbooks_patch_manager:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_patch
+  headers: *owner_manager_headers
+  assert_status: 200
+
+service_patch_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_patch
+  headers: *service_headers_owner_project
+  assert_status: 200
+
+project_admin_delete_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: delete
+  headers: *owner_admin_headers
+  assert_status: 204
+
+project_manager_delete_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: delete
+  headers: *owner_manager_headers
+  assert_status: 204
+
+service_get_runbooks:
+  path: '/v1/runbooks'
+  method: get
+  headers: *service_headers_owner_project
+  assert_status: 200
+
+service_patch_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_patch
+  headers: *service_headers_owner_project
+  assert_status: 200
+
+runbooks_project_admin:
+  path: '/v1/runbooks'
+  method: get
+  headers: *owner_admin_headers
+  assert_status: 200
+
+runbooks_runbook_id_get_project_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: get
+  headers: *owner_admin_headers
+  assert_status: 200
+
+project_admin_patch_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_patch
+  headers: *owner_admin_headers
+  assert_status: 200
+
+runbooks_project_manager:
+  path: '/v1/runbooks'
+  method: get
+  headers: *owner_manager_headers
+  assert_status: 200
+
+runbooks_runbook_id_get_project_manager:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: get
+  headers: *owner_manager_headers
+  assert_status: 200
+
+project_manager_patch_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_patch
+  headers: *owner_manager_headers
+  assert_status: 200
+
+runbooks_project_member:
+  path: '/v1/runbooks'
+  method: get
+  headers: *owner_member_headers
+  assert_status: 200
+
+runbooks_runbook_id_get_project_member:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: get
+  headers: *owner_member_headers
+  assert_status: 200
+
+runbooks_list_project_reader:
+  path: '/v1/runbooks'
+  method: get
+  headers: *owner_reader_headers
+  assert_status: 200
+
+runbooks_runbook_id_get_project_reader:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: get
+  headers: *owner_reader_headers
+  assert_status: 200
+
+runbooks_list_third_party_admin:
+  path: '/v1/runbooks'
+  method: get
+  headers: *third_party_admin_headers
+  assert_status: 200
+
+project_reader_cannot_post_runbook:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *owner_reader_headers
+  assert_status: 403
+
+project_reader_cannot_patch_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_patch
+  headers: *owner_reader_headers
+  assert_status: 403
+
+project_reader_cannot_set_runbook_owner:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: &runbook_owner_patch
+    - op: replace
+      path: /owner
+      value: 'new_owner'
+  headers: *owner_reader_headers
+  assert_status: 403
+
+project_reader_cannot_set_runbook_public:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: &runbook_public_patch
+    - op: replace
+      path: /public
+      value: true
+  headers: *owner_reader_headers
+  assert_status: 403
+
+project_reader_cannot_delete_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: delete
+  headers: *owner_reader_headers
+  assert_status: 403
+
+project_member_cannot_post_runbook:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *owner_member_headers
+  assert_status: 403
+
+project_member_cannot_patch_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_patch
+  headers: *owner_member_headers
+  assert_status: 403
+
+project_member_cannot_set_runbook_owner:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_owner_patch
+  headers: *owner_member_headers
+  assert_status: 403
+
+project_member_cannot_set_runbook_public:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *owner_member_headers
+  assert_status: 403
+
+project_member_cannot_delete_runbook:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: delete
+  headers: *owner_member_headers
+  assert_status: 403
+
+project_manager_cannot_set_runbook_owner:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_owner_patch
+  headers: *owner_manager_headers
+  assert_status: 403
+
+project_manager_cannot_set_runbook_public:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *owner_manager_headers
+  assert_status: 403
+
+project_admin_cannot_set_runbook_owner:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_owner_patch
+  headers: *owner_admin_headers
+  assert_status: 403
+
+project_admin_cannot_set_runbook_public:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *owner_admin_headers
+  assert_status: 403
+
+service_cannot_patch_runbook_owner:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_owner_patch
+  headers: *service_headers_owner_project
+  assert_status: 403
+
+service_cannot_patch_runbook_public:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *service_headers_owner_project
+  assert_status: 403
+
+third_party_admin_cannot_patch_runbook_owner:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_owner_patch
+  headers: *third_party_admin_headers
+  assert_status: 403
+
+third_party_admin_cannot_patch_runbook_public:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *third_party_admin_headers
+  assert_status: 403

ironic/tests/unit/api/test_rbac_system_scoped.yaml

@@ -1,5 +1,10 @@
 values:
   skip_reason: "These are fake reference values for YAML templating"
+  # Project scoped admin token
+  project_admin_headers: &project_admin_headers
+    X-Auth-Token: 'owner-admin-token'
+    X-Roles: admin,manager,member,reader
+    X-Project-Id: 70e5e25a-2ca2-4cb1-8ae8-7d8739cee205
   # System scoped admin token
   admin_headers: &admin_headers
     X-Auth-Token: 'baremetal-admin-token'
@@ -2584,3 +2589,186 @@ nodes_firmware_component_get_reader:
   method: get
   headers: *reader_headers
   assert_status: 200
+
+# Runbooks - https://docs.openstack.org/api-ref/baremetal/#runbooks-templates
+
+runbooks_post_admin:
+  path: '/v1/runbooks'
+  method: post
+  body: &runbook_body
+    name: 'CUSTOM_NAME'
+    steps:
+      - interface: 'raid'
+        step: 'noop'
+        args: {}
+        order: 0
+  headers: *admin_headers
+  assert_status: 201
+
+runbooks_post_member:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *scoped_member_headers
+  assert_status: 201
+
+runbooks_post_reader:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *reader_headers
+  assert_status: 403
+
+runbooks_get_admin:
+  path: '/v1/runbooks'
+  method: get
+  headers: *admin_headers
+  assert_status: 200
+
+runbooks_get_member:
+  path: '/v1/runbooks'
+  method: get
+  headers: *scoped_member_headers
+  assert_status: 200
+
+runbooks_get_reader:
+  path: '/v1/runbooks'
+  method: get
+  headers: *reader_headers
+  assert_status: 200
+
+runbooks_runbook_id_get_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: get
+  headers: *admin_headers
+  assert_status: 200
+
+runbooks_runbook_id_get_member:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: get
+  headers: *scoped_member_headers
+  assert_status: 200
+
+runbooks_runbook_id_get_reader:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: get
+  headers: *reader_headers
+  assert_status: 200
+
+runbooks_runbook_id_patch_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: &runbook_name_patch
+    - op: replace
+      path: /name
+      value: 'CUSTOM_NAME'
+  headers: *admin_headers
+  assert_status: 200
+
+runbooks_runbook_id_patch_member:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_name_patch
+  headers: *scoped_member_headers
+  assert_status: 200
+
+runbooks_runbook_id_patch_reader:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_name_patch
+  headers: *reader_headers
+  assert_status: 403
+
+runbooks_runbook_id_patch_public_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: &runbook_public_patch
+    - op: replace
+      path: /public
+      value: true
+  headers: *admin_headers
+  assert_status: 200
+
+runbooks_runbook_id_patch_public_member:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *scoped_member_headers
+  assert_status: 200
+
+runbooks_runbook_id_patch_public_reader:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *reader_headers
+  assert_status: 403
+
+runbooks_runbook_id_patch_owner_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: &runbook_owner_patch
+    - op: replace
+      path: /owner
+      value: 'new_owner'
+  headers: *admin_headers
+  assert_status: 200
+
+runbooks_runbook_id_patch_owner_member:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_owner_patch
+  headers: *scoped_member_headers
+  assert_status: 200
+
+runbooks_runbook_id_patch_owner_reader:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_owner_patch
+  headers: *reader_headers
+  assert_status: 403
+
+runbooks_runbook_id_delete_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: delete
+  headers: *admin_headers
+  assert_status: 204
+
+runbooks_runbook_id_delete_member:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: delete
+  headers: *scoped_member_headers
+  assert_status: 204
+
+runbooks_runbook_id_delete_reader:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: delete
+  headers: *reader_headers
+  assert_status: 403
+
+runbooks_post_project_admin:
+  path: '/v1/runbooks'
+  method: post
+  body: *runbook_body
+  headers: *project_admin_headers
+  assert_status: 201
+
+runbooks_runbook_id_patch_public_admin:
+  path: '/v1/runbooks/{runbook_ident}'
+  method: patch
+  body: *runbook_public_patch
+  headers: *admin_headers
+  assert_status: 200
+
+public_runbooks_post_admin:
+  path: '/v1/runbooks'
+  method: post
+  body: &runbook_body_public
+    name: 'CUSTOM_NAME'
+    public: true
+    steps:
+      - interface: 'raid'
+        step: 'noop'
+        args: {}
+        order: 0
+  headers: *admin_headers
+  assert_status: 201

ironic/tests/unit/api/utils.py

@@ -27,6 +27,7 @@ from ironic.api.controllers.v1 import deploy_template as dt_controller
 from ironic.api.controllers.v1 import node as node_controller
 from ironic.api.controllers.v1 import port as port_controller
 from ironic.api.controllers.v1 import portgroup as portgroup_controller
+from ironic.api.controllers.v1 import runbook as rb_controller
 from ironic.api.controllers.v1 import utils as api_utils
 from ironic.api.controllers.v1 import volume_connector as vc_controller
 from ironic.api.controllers.v1 import volume_target as vt_controller
@@ -201,6 +202,25 @@ def deploy_template_post_data(**kw):
         template, dt_controller.TEMPLATE_SCHEMA['properties'])
 
 
+def runbook_post_data(**kw):
+    """Return a Runbook object without internal attributes."""
+    runbook = db_utils.get_test_runbook(**kw)
+    # These values are not part of the API object
+    runbook.pop('version')
+    # Remove internal attributes from each step.
+    step_internal = api_utils.RUNBOOK_STEP_SCHEMA['properties']
+    runbook['steps'] = [remove_other_fields(step, step_internal)
+                        for step in runbook['steps']]
+    # Remove internal attributes from the runbook.
+    return remove_other_fields(
+        runbook, rb_controller.RUNBOOK_SCHEMA['properties'])
+
+
 def post_get_test_deploy_template(**kw):
     """Return a DeployTemplate object with appropriate attributes."""
     return deploy_template_post_data(**kw)
+
+
+def post_get_test_runbook(**kw):
+    """Return a Runbook object with appropriate attributes."""
+    return runbook_post_data(**kw)

ironic/tests/unit/common/test_release_mappings.py

@@ -101,7 +101,7 @@ class ReleaseMappingsTestCase(base.TestCase):
         # NodeBase is also excluded as it is covered by Node.
         exceptions = set(['NodeTag', 'ConductorHardwareInterfaces',
                           'NodeTrait', 'DeployTemplateStep',
-                          'NodeBase'])
+                          'NodeBase', 'RunbookStep'])
         model_names -= exceptions
         # NodeTrait maps to two objects
         model_names |= set(['Trait', 'TraitList'])

ironic/tests/unit/db/sqlalchemy/test_migrations.py

@@ -1169,6 +1169,152 @@ class MigrationCheckersMixin(object):
         self.assertIsInstance(deploy_templates.c.extra.type,
                               sqlalchemy.types.TEXT)
 
+    def _check_dada631878c4(self, engine, data):
+        # Runbooks.
+        runbooks = db_utils.get_table(engine, 'runbooks')
+        col_names = [column.name for column in runbooks.c]
+        expected = ['created_at', 'updated_at', 'version',
+                    'id', 'uuid', 'name']
+        self.assertEqual(sorted(expected), sorted(col_names))
+        self.assertIsInstance(runbooks.c.created_at.type,
+                              sqlalchemy.types.DateTime)
+        self.assertIsInstance(runbooks.c.updated_at.type,
+                              sqlalchemy.types.DateTime)
+        self.assertIsInstance(runbooks.c.version.type,
+                              sqlalchemy.types.String)
+        self.assertIsInstance(runbooks.c.id.type,
+                              sqlalchemy.types.Integer)
+        self.assertIsInstance(runbooks.c.uuid.type,
+                              sqlalchemy.types.String)
+        self.assertIsInstance(runbooks.c.name.type,
+                              sqlalchemy.types.String)
+
+        # Runbook steps.
+        runbook_steps = db_utils.get_table(engine, 'runbook_steps')
+        col_names = [column.name for column in runbook_steps.c]
+        expected = ['created_at', 'updated_at', 'version', 'id',
+                    'runbook_id', 'interface', 'step', 'args',
+                    'order']
+        self.assertEqual(sorted(expected), sorted(col_names))
+
+        self.assertIsInstance(runbook_steps.c.created_at.type,
+                              sqlalchemy.types.DateTime)
+        self.assertIsInstance(runbook_steps.c.updated_at.type,
+                              sqlalchemy.types.DateTime)
+        self.assertIsInstance(runbook_steps.c.version.type,
+                              sqlalchemy.types.String)
+        self.assertIsInstance(runbook_steps.c.id.type,
+                              sqlalchemy.types.Integer)
+        self.assertIsInstance(runbook_steps.c.runbook_id.type,
+                              sqlalchemy.types.Integer)
+        self.assertIsInstance(runbook_steps.c.interface.type,
+                              sqlalchemy.types.String)
+        self.assertIsInstance(runbook_steps.c.step.type,
+                              sqlalchemy.types.String)
+        self.assertIsInstance(runbook_steps.c.args.type,
+                              sqlalchemy.types.Text)
+        self.assertIsInstance(runbook_steps.c.order.type,
+                              sqlalchemy.types.Integer)
+
+        with engine.begin() as connection:
+            # Insert a Runbook.
+            uuid = uuidutils.generate_uuid()
+            name = 'CUSTOM_DT1'
+            runbook = {'name': name, 'uuid': uuid}
+            insert_dpt = runbooks.insert().values(runbook)
+            connection.execute(insert_dpt)
+            # Query by UUID.
+            dpt_uuid_stmt = sqlalchemy.select(
+                models.Runbook.id,
+                models.Runbook.name,
+            ).where(
+                models.Runbook.uuid == uuid
+            )
+            result = connection.execute(dpt_uuid_stmt).first()
+            runbook_id = result.id
+            self.assertEqual(name, result.name)
+            # Query by name.
+            dpt_name_stmt = sqlalchemy.select(
+                models.Runbook.id
+            ).where(
+                models.Runbook.name == name
+            )
+            result = connection.execute(dpt_name_stmt).first()
+            self.assertEqual(runbook_id, result.id)
+            # Query by ID.
+            dpt_id_stmt = sqlalchemy.select(
+                models.Runbook.uuid,
+                models.Runbook.name
+            ).where(
+                models.Runbook.id == runbook_id
+            )
+            result = connection.execute(dpt_id_stmt).first()
+            self.assertEqual(uuid, result.uuid)
+            self.assertEqual(name, result.name)
+            savepoint_uuid = connection.begin_nested()
+            # UUID is unique.
+            runbook = {'name': 'CUSTOM_DT2', 'uuid': uuid}
+            self.assertRaises(db_exc.DBDuplicateEntry, connection.execute,
+                              runbooks.insert(), runbook)
+            savepoint_uuid.rollback()
+            savepoint_uuid.close()
+            # Name is unique.
+            savepoint_name = connection.begin_nested()
+            runbook = {'name': name, 'uuid': uuidutils.generate_uuid()}
+            self.assertRaises(db_exc.DBDuplicateEntry, connection.execute,
+                              runbooks.insert(), runbook)
+            savepoint_name.rollback()
+            savepoint_name.close()
+
+            # Insert a Runbook step.
+            interface = 'raid'
+            step_name = 'create_configuration'
+            # The line below is JSON.
+            args = '{"logical_disks": []}'
+            order = 1
+            step = {'runbook_id': runbook_id, 'interface': interface,
+                    'step': step_name, 'args': args, 'order': order}
+            insert_dpts = runbook_steps.insert().values(step)
+            connection.execute(insert_dpts)
+            # Query by Runbook ID.
+            query_id_stmt = sqlalchemy.select(
+                models.RunbookStep.runbook_id,
+                models.RunbookStep.interface,
+                models.RunbookStep.step,
+                models.RunbookStep.args,
+                models.RunbookStep.order,
+            ).where(
+                models.RunbookStep.runbook_id == runbook_id
+            )
+            result = connection.execute(query_id_stmt).first()
+            self.assertEqual(runbook_id, result.runbook_id)
+            self.assertEqual(interface, result.interface)
+            self.assertEqual(step_name, result.step)
+            if isinstance(result.args, dict):
+                # Postgres testing results in a dict being returned
+                # at this level which if you str() it, you get a dict,
+                # so comparing string to string fails.
+                result_args = json.dumps(result.args)
+            else:
+                # Mysql/MariaDB appears to be actually hand us
+                # a string back so we should be able to compare it.
+                result_args = result.args
+            self.assertEqual(args, result_args)
+            self.assertEqual(order, result.order)
+            # Insert another step for the same runbook.
+            insert_step = runbook_steps.insert().values(step)
+            connection.execute(insert_step)
+
+    def _check_245c3e54b247(self, engine, data):
+        # Runbook 'extra' field.
+        runbooks = db_utils.get_table(engine, 'runbooks')
+        col_names = [column.name for column in runbooks.c]
+        expected = ['created_at', 'updated_at', 'version',
+                    'id', 'uuid', 'name', 'extra']
+        self.assertEqual(sorted(expected), sorted(col_names))
+        self.assertIsInstance(runbooks.c.extra.type,
+                              sqlalchemy.types.TEXT)
+
     def _check_ce6c4b3cf5a2(self, engine, data):
         allocations = db_utils.get_table(engine, 'allocations')
         col_names = [column.name for column in allocations.c]

ironic/tests/unit/db/test_runbooks.py

@@ -0,0 +1,207 @@
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+"""Tests for manipulating Runbooks via the DB API"""
+
+from oslo_db import exception as db_exc
+from oslo_utils import uuidutils
+
+from ironic.common import exception
+from ironic.tests.unit.db import base
+from ironic.tests.unit.db import utils as db_utils
+
+
+class DbRunbookTestCase(base.DbTestCase):
+
+    def setUp(self):
+        super(DbRunbookTestCase, self).setUp()
+        self.runbook = db_utils.create_test_runbook()
+
+    def test_create(self):
+        self.assertEqual('CUSTOM_DT1', self.runbook.name)
+        self.assertEqual(1, len(self.runbook.steps))
+        step = self.runbook.steps[0]
+        self.assertEqual(self.runbook.id, step.runbook_id)
+        self.assertEqual('raid', step.interface)
+        self.assertEqual('create_configuration', step.step)
+        self.assertEqual({'logical_disks': []}, step.args)
+        self.assertEqual({}, self.runbook.extra)
+
+    def test_create_no_steps(self):
+        uuid = uuidutils.generate_uuid()
+        runbook = db_utils.create_test_runbook(
+            uuid=uuid, name='CUSTOM_DT2', steps=[])
+        self.assertEqual([], runbook.steps)
+
+    def test_create_duplicate_uuid(self):
+        self.assertRaises(exception.RunbookAlreadyExists,
+                          db_utils.create_test_runbook,
+                          uuid=self.runbook.uuid, name='CUSTOM_DT2')
+
+    def test_create_duplicate_name(self):
+        uuid = uuidutils.generate_uuid()
+        self.assertRaises(exception.RunbookDuplicateName,
+                          db_utils.create_test_runbook,
+                          uuid=uuid, name=self.runbook.name)
+
+    def test_create_invalid_step_no_interface(self):
+        uuid = uuidutils.generate_uuid()
+        runbook = db_utils.get_test_runbook(uuid=uuid,
+                                            name='CUSTOM_DT2')
+        del runbook['steps'][0]['interface']
+        self.assertRaises(db_exc.DBError,
+                          self.dbapi.create_runbook,
+                          runbook)
+
+    def test_update_name(self):
+        values = {'name': 'CUSTOM_DT2'}
+        runbook = self.dbapi.update_runbook(self.runbook.id, values)
+        self.assertEqual('CUSTOM_DT2', runbook.name)
+
+    def test_update_steps_replace(self):
+        step = {'interface': 'bios', 'step': 'apply_configuration',
+                'args': {}, 'order': 1}
+        values = {'steps': [step]}
+        runbook = self.dbapi.update_runbook(self.runbook.id, values)
+        self.assertEqual(1, len(runbook.steps))
+        step = runbook.steps[0]
+        self.assertEqual('bios', step.interface)
+        self.assertEqual('apply_configuration', step.step)
+        self.assertEqual({}, step.args)
+        self.assertEqual(1, step.order)
+
+    def test_update_steps_add(self):
+        step = {'interface': 'bios', 'step': 'apply_configuration',
+                'args': {}, 'order': 1}
+        values = {'steps': [self.runbook.steps[0], step]}
+        runbook = self.dbapi.update_runbook(self.runbook.id, values)
+        self.assertEqual(2, len(runbook.steps))
+        step0 = runbook.steps[0]
+        self.assertEqual(self.runbook.steps[0].id, step0.id)
+        self.assertEqual('raid', step0.interface)
+        self.assertEqual('create_configuration', step0.step)
+        self.assertEqual({'logical_disks': []}, step0.args)
+        step1 = runbook.steps[1]
+        self.assertNotEqual(self.runbook.steps[0].id, step1.id)
+        self.assertEqual('bios', step1.interface)
+        self.assertEqual('apply_configuration', step1.step)
+        self.assertEqual({}, step1.args)
+        self.assertEqual(1, step1.order)
+
+    def test_update_steps_replace_args(self):
+        step = self.runbook.steps[0]
+        step['args'] = {'foo': 'bar'}
+        values = {'steps': [step]}
+        runbook = self.dbapi.update_runbook(self.runbook.id, values)
+        self.assertEqual(1, len(runbook.steps))
+        step = runbook.steps[0]
+        self.assertEqual({'foo': 'bar'}, step.args)
+
+    def test_update_steps_remove_all(self):
+        values = {'steps': []}
+        runbook = self.dbapi.update_runbook(self.runbook.id, values)
+        self.assertEqual([], runbook.steps)
+
+    def test_update_extra(self):
+        values = {'extra': {'foo': 'bar'}}
+        runbook = self.dbapi.update_runbook(self.runbook.id, values)
+        self.assertEqual({'foo': 'bar'}, runbook.extra)
+
+    def test_update_duplicate_name(self):
+        uuid = uuidutils.generate_uuid()
+        runbook2 = db_utils.create_test_runbook(uuid=uuid,
+                                                name='CUSTOM_DT2')
+        values = {'name': self.runbook.name}
+        self.assertRaises(exception.RunbookDuplicateName,
+                          self.dbapi.update_runbook, runbook2.id,
+                          values)
+
+    def test_update_not_found(self):
+        self.assertRaises(exception.RunbookNotFound,
+                          self.dbapi.update_runbook, 123, {})
+
+    def test_update_uuid_not_allowed(self):
+        uuid = uuidutils.generate_uuid()
+        self.assertRaises(exception.InvalidParameterValue,
+                          self.dbapi.update_runbook,
+                          self.runbook.id, {'uuid': uuid})
+
+    def test_destroy(self):
+        self.dbapi.destroy_runbook(self.runbook.id)
+        # Attempt to retrieve the runbook to verify it is gone.
+        self.assertRaises(exception.RunbookNotFound,
+                          self.dbapi.get_runbook_by_id,
+                          self.runbook.id)
+        # Ensure that the destroy_runbook returns the
+        # expected exception.
+        self.assertRaises(exception.RunbookNotFound,
+                          self.dbapi.destroy_runbook,
+                          self.runbook.id)
+
+    def test_get_runbook_by_id(self):
+        res = self.dbapi.get_runbook_by_id(self.runbook.id)
+        self.assertEqual(self.runbook.id, res.id)
+        self.assertEqual(self.runbook.name, res.name)
+        self.assertEqual(1, len(res.steps))
+        self.assertEqual(self.runbook.id, res.steps[0].runbook_id)
+        self.assertRaises(exception.RunbookNotFound,
+                          self.dbapi.get_runbook_by_id, -1)
+
+    def test_get_runbook_by_uuid(self):
+        res = self.dbapi.get_runbook_by_uuid(self.runbook.uuid)
+        self.assertEqual(self.runbook.id, res.id)
+        invalid_uuid = uuidutils.generate_uuid()
+        self.assertRaises(exception.RunbookNotFound,
+                          self.dbapi.get_runbook_by_uuid, invalid_uuid)
+
+    def test_get_runbook_by_name(self):
+        res = self.dbapi.get_runbook_by_name(self.runbook.name)
+        self.assertEqual(self.runbook.id, res.id)
+        self.assertRaises(exception.RunbookNotFound,
+                          self.dbapi.get_runbook_by_name, 'bogus')
+
+    def _runbook_list_preparation(self):
+        uuids = [str(self.runbook.uuid)]
+        for i in range(1, 3):
+            runbook = db_utils.create_test_runbook(
+                uuid=uuidutils.generate_uuid(),
+                name='CUSTOM_DT%d' % (i + 1))
+            uuids.append(str(runbook.uuid))
+        return uuids
+
+    def test_get_runbook_list(self):
+        uuids = self._runbook_list_preparation()
+        res = self.dbapi.get_runbook_list()
+        res_uuids = [r.uuid for r in res]
+        self.assertCountEqual(uuids, res_uuids)
+
+    def test_get_runbook_list_sorted(self):
+        uuids = self._runbook_list_preparation()
+        res = self.dbapi.get_runbook_list(sort_key='uuid')
+        res_uuids = [r.uuid for r in res]
+        self.assertEqual(sorted(uuids), res_uuids)
+
+        self.assertRaises(exception.InvalidParameterValue,
+                          self.dbapi.get_runbook_list, sort_key='foo')
+
+    def test_get_runbook_list_by_names(self):
+        self._runbook_list_preparation()
+        names = ['CUSTOM_DT2', 'CUSTOM_DT3']
+        res = self.dbapi.get_runbook_list_by_names(names=names)
+        res_names = [r.name for r in res]
+        self.assertCountEqual(names, res_names)
+
+    def test_get_runbook_list_by_names_no_match(self):
+        self._runbook_list_preparation()
+        names = ['CUSTOM_FOO']
+        res = self.dbapi.get_runbook_list_by_names(names=names)
+        self.assertEqual([], res)

ironic/tests/unit/db/utils.py

@@ -32,6 +32,7 @@ from ironic.objects import node_history
 from ironic.objects import node_inventory
 from ironic.objects import port
 from ironic.objects import portgroup
+from ironic.objects import runbook
 from ironic.objects import trait
 from ironic.objects import volume_connector
 from ironic.objects import volume_target
@@ -673,6 +674,59 @@ def create_test_deploy_template(**kw):
     return dbapi.create_deploy_template(template)
 
 
+def get_test_runbook(**kw):
+    default_uuid = uuidutils.generate_uuid()
+    return {
+        'version': kw.get('version', runbook.Runbook.VERSION),
+        'created_at': kw.get('created_at'),
+        'updated_at': kw.get('updated_at'),
+        'id': kw.get('id', 234),
+        'name': kw.get('name', u'CUSTOM_DT1'),
+        'uuid': kw.get('uuid', default_uuid),
+        'steps': kw.get('steps', [get_test_runbook_step(
+            runbook_id=kw.get('id', 234))]),
+        'disable_ramdisk': kw.get('disable_ramdisk', False),
+        'extra': kw.get('extra', {}),
+        'public': kw.get('public', False),
+        'owner': kw.get('owner', None),
+    }
+
+
+def get_test_runbook_step(**kw):
+    return {
+        'created_at': kw.get('created_at'),
+        'updated_at': kw.get('updated_at'),
+        'id': kw.get('id', 345),
+        'runbook_id': kw.get('runbook_id', 234),
+        'interface': kw.get('interface', 'raid'),
+        'step': kw.get('step', 'create_configuration'),
+        'args': kw.get('args', {'logical_disks': []}),
+        'order': kw.get('order', 1)
+    }
+
+
+def create_test_runbook(**kw):
+    """Create a runbook in the DB and return Runbook model.
+
+    :param kw: kwargs with overriding values for the runbook.
+    :returns: Test Runbook DB object.
+    """
+    runbook = get_test_runbook(**kw)
+    dbapi = db_api.get_instance()
+    # Let DB generate an ID if one isn't specified explicitly.
+    if 'id' not in kw:
+        del runbook['id']
+    if 'steps' not in kw:
+        for step in runbook['steps']:
+            del step['id']
+            del step['runbook_id']
+    else:
+        for kw_step, runbook_step in zip(kw['steps'], runbook['steps']):
+            if 'id' not in kw_step:
+                del runbook_step['id']
+    return dbapi.create_runbook(runbook)
+
+
 def get_test_history(**kw):
     return {
         'id': kw.get('id', 345),

ironic/tests/unit/objects/test_objects.py

@@ -724,6 +724,9 @@ expected_object_fingerprints = {
     'NodeInventory': '1.0-97692fec24e20ab02022b9db54e8f539',
     'FirmwareComponent': '1.0-0e0720dab959e20247bbcfd5f28958c5',
     'FirmwareComponentList': '1.0-33a2e1bb91ad4082f9f63429b77c1244',
+    'Runbook': '1.0-7a9c65b49b5f7b45686b6a674e703629',
+    'RunbookCRUDNotification': '1.0-59acc533c11d306f149846f922739c15',
+    'RunbookCRUDPayload': '1.0-f0c97f4ff29eb3401e53b34550a95e30',
 }
 
 

ironic/tests/unit/objects/utils.py

@@ -358,6 +358,41 @@ def get_payloads_with_schemas(from_module):
     return payloads
 
 
+def get_test_runbook(ctxt, **kw):
+    """Return a Runbook object with appropriate attributes.
+
+    NOTE: The object leaves the attributes marked as changed, such
+    that a create() could be used to commit it to the DB.
+    """
+    db_runbook = db_utils.get_test_runbook(**kw)
+    # Let DB generate ID if it isn't specified explicitly
+    if 'id' not in kw:
+        del db_runbook['id']
+    if 'steps' not in kw:
+        for step in db_runbook['steps']:
+            del step['id']
+            del step['runbook_id']
+    else:
+        for kw_step, runbook_step in zip(kw['steps'], db_runbook['steps']):
+            if 'id' not in kw_step and 'id' in runbook_step:
+                del runbook_step['id']
+    runbook = objects.Runbook(ctxt)
+    for key in db_runbook:
+        setattr(runbook, key, db_runbook[key])
+    return runbook
+
+
+def create_test_runbook(ctxt, **kw):
+    """Create and return a test runbook object.
+
+    NOTE: The object leaves the attributes marked as changed, such
+    that a create() could be used to commit it to the DB.
+    """
+    runbook = get_test_runbook(ctxt, **kw)
+    runbook.create()
+    return runbook
+
+
 class SchemasTestMixIn(object):
     def _check_payload_schemas(self, from_module, fields):
         """Assert that the Payload SCHEMAs have the expected properties.

releasenotes/notes/add-runbooks-38c3efa97ace8c67.yaml

@@ -0,0 +1,19 @@
+---
+features:
+  - |
+    Adds a new API concept, runbooks, to enable self-service of maintenance
+    items on nodes by project members.
+
+    Runbooks are curated lists of steps that can be run on nodes only
+    associated via traits and used in lieu of an explicit list of steps
+    for manual cleaning or servicing.
+  - |
+    Adds a new top-level REST API endpoint `/v1/runbooks/` with basic CRUD
+    support.
+  - |
+    Extends the `/v1/nodes/<node>/states/provision` API to accept a runbook
+    ident (name or UUID) instead of `clean_steps` or `service_steps` for
+    servicing or manual cleaning.
+  - |
+    Implements RBAC-aware lifecycle management for runbooks, allowing projects
+    to limit who can CRUD and use a runbook.