deckhand/doc/source/documents.rst

Documents

All configuration data is stored entirely as structured documents, for which schemas must be registered. Documents satisfy the following use cases:

• layering - helps reduce duplication in configuration while maintaining auditability across many sites
• substitution - provides separation between secret data and other configuration data, while allowing a simple interface for clients
• revision history - improves auditability and enables services to provide functional validation of a well-defined collection of documents that are meant to operate together
• validation - allows services to implement and register different kinds of validations and report errors

Detailed documentation for layering, substitution, revision-history and validation should be reviewed for a more thorough understanding of each concept.

Document Format

The document format is modeled loosely after Kubernetes practices. The top level of each document is a dictionary with 3 keys: schema, metadata, and data.

• schema - Defines the name of the JSON schema to be used for validation. Must have the form: <namespace>/<kind>/<version>, where the meaning of each component is: * namespace - Identifies the owner of this type of document. The values deckhand and metadata are reserved for internal use.
  • kind - Identifies a type of configuration resource in the namespace.
  • version - Describe the version of this resource, e.g. "v1".
• metadata - Defines details that Deckhand will inspect and understand. There are multiple schemas for this section as discussed below. All the various types of metadata include a name field which must be unique for each document schema.
• data - Data to be validated by the schema described by the schema field. Deckhand only interacts with content here as instructed to do so by the metadata section. The form of this section is considered to be completely owned by the namespace in the schema.

Below is a fictitious example of a complete document, which illustrates all the valid fields in the metadata section:

---
schema: some-service/ResourceType/v1
metadata:
  schema: metadata/Document/v1
  name: unique-name-given-schema
  storagePolicy: cleartext
  labels:
    genesis: enabled
    master: enabled
  layeringDefinition:
    abstract: true
    layer: region
    parentSelector:
      required_key_a: required_label_a
      required_key_b: required_label_b
    actions:
      - method: merge
        path: .path.to.merge.into.parent
      - method: delete
        path: .path.to.delete
  substitutions:
    - dest:
        path: .substitution.target
      src:
        schema: another-service/SourceType/v1
        name: name-of-source-document
        path: .source.path
data:
  path:
    to:
      merge:
        into:
          parent:
            foo: bar
          ignored:
            data: here
  substitution:
    target: null

Document Metadata

There are 2 supported kinds of document metadata. Documents with Document metadata are the most common, and are used for normal configuration data. Documents with Control metadata are used to customize the behavior of Deckhand.

schema: metadata/Document/v1

This type of metadata allows the following metadata hierarchy:

• name - string, required - Unique within a revision for a given schema.
• storagePolicy - string, required - Either cleartext or encrypted. If encyrpted is specified, then the data section of the document will be stored in an secure backend (likely via OpenStack Barbican). metadata and schema fields are always stored in cleartext.
• layeringDefinition - dict, required - Specifies layering details. See the Layering section below for details.
  • abstract - boolean, required - An abstract document is not expected to pass schema validation after layering and substitution are applied. Non-abstract (concrete) documents are.
  • layer - string, required - References a layer in the LayeringPolicy control document.
  • parentSelector - labels, optional - Used to construct document chains for executing merges.
  • actions - list, optional - A sequence of actions to apply this documents data during the merge process.
    • method - string, required - How to layer this content.
    * path - string, required - What content in this document to layer onto parent content.
• substitutions - list, optional - A sequence of substitutions to apply. See the Substitutions section for additional details.
  • dest - dict, required - A description of the inserted content destination.
    • path - string, required - The JSON path where the data will be placed into the data section of this document.
    • pattern - string, optional - A regex to search for in the string specified at path in this document and replace with the source data
  • src - dict, required - A description of the inserted content source.
    • schema - string, required - The schema of the source document.
    • name - string, required - The metadata.name of the source document.
    • path - string, required - The JSON path from which to extract data in the source document relative to its data section.

schema: metadata/Control/v1

This schema is the same as the Document schema, except it omits the storagePolicy, layeringDefinition, and substitutions keys, as these actions are not supported on Control documents.

The complete list of valid Control document kinds is specified below along with descriptions of each document kind.