Addresses: BP decoupled-source-sink-discoverable-resources
Add support for a new decoupled model of pipeline config
which does not conflate the elements specific to sources
and sinks.
Instead of the basic unit of config being a consolidated
pipeline, the sources and sinks may now be specified as
separate lists of dictionaries, before being linked together
once parsed.
This allows source-specific configuration, such as resource
discovery, to be kept focused only on the fine-grained source
while avoiding the necessity for wide duplication of sink-
related config.
The shape of the new config format is best illustrated with
an example:
---
sources:
- name: meter_source
interval: 600
meters:
- "*"
sinks:
- meter_sink
- name: host_cpu_source
interval: 120
meters:
- "cpu.util.*min"
resources:
- "snmp://ip1"
- "snmp://ip2"
- "snmp://ip3"
sinks:
- meter_sink
- lossy_sink
- name: instance_cpu_source
interval: 60
meters:
- "cpu"
sinks:
- cpu_sink
sinks:
- name: meter_sink
transformers:
publishers:
- rpc://
- name: lossy_sink
transformers:
publishers:
- udp://addr
- name: cpu_sink
transformers:
- name: "rate_of_change"
parameters:
target:
name: "cpu_util"
unit: "%"
type: "gauge"
scale: "100.0 / (10**9 * (resource_metadata.cpu_number or 1))"
publishers:
- rpc://
The old pipeline.yaml format continues to be supported as
a deprecated feature to avoid breaking existing deployments.
The semantics of the common individual configuration elements
are identical in the deprecated and decoupled versions.
Change-Id: Ide86c0feba88ae736f2a913b5faa95e640c4ceaf
9114e135b6