Merge "Improves pipeline transformer documentation"

doc/source/configuration.rst

@@ -374,7 +374,22 @@ common interval is passed to individual pollsters matching those pipelines.
 
 The *transformers* section of a pipeline sink provides the possibility to add a
 list of transformer definitions. The names of the transformers should be the same
-as the names of the related extensions in setup.cfg.
+as the names of the related extensions in setup.cfg. For a more detailed
+description, please see the :ref:`transformers` section.
+
+The *publishers* section contains the list of publishers, where the samples
+data should be sent after the possible transformations. The names of the
+publishers should be the same as the related names of the plugins in
+setup.cfg.
+
+The default configuration can be found in `pipeline.yaml`_.
+
+.. _pipeline.yaml: https://git.openstack.org/cgit/openstack/ceilometer/tree/etc/ceilometer/pipeline.yaml
+
+.. _transformers:
+
+Transformers
+************
 
 The definition of transformers can contain the following fields::
 
@@ -384,9 +399,15 @@ The definition of transformers can contain the following fields::
 
 The *parameters* section can contain transformer specific fields, like source
 and target fields with different subfields in case of the rate_of_change,
-which depends on the implementation of the transformer. In case of the
+which depends on the implementation of the transformer.
-transformer, which creates the *cpu_util* meter, the definition looks like the
+
-following::
+.. _rate_of_change_transformer:
+
+Rate of change transformer
+++++++++++++++++++++++++++
+
+In the case of the transformer that creates the *cpu_util* meter, the definition
+looks like the following::
 
     transformers:
         - name: "rate_of_change"
@@ -420,11 +441,83 @@ The definition for the disk I/O rate, which is also generated by the
                       unit: "\\1/s"
                   type: "gauge"
 
-The *publishers* section contains the list of publishers, where the samples
+Unit conversion transformer
-data should be sent after the possible transformations. The names of the
++++++++++++++++++++++++++++
-publishers should be the same as the related names of the plugins in
-setup.cfg.
 
-The default configuration can be found in `pipeline.yaml`_.
+Transformer to apply a unit conversion. It takes the volume of the meter
+and multiplies it with the given 'scale' expression. Also supports *map_from*
+and *map_to* like the :ref:`rate_of_change_transformer`.
+
+Sample configuration::
+
+    transformers:
+    - name: "unit_conversion"
+      parameters:
+          target:
+              name: "disk.kilobytes"
+              unit: "KB"
+              scale: "1.0 / 1024.0"
+
+With the *map_from* and *map_to*::
+
+    transformers:
+        - name: "unit_conversion"
+          parameters:
+              source:
+                  map_from:
+                      name: "disk\\.(read|write)\\.bytes"
+              target:
+                  map_to:
+                      name: "disk.\\1.kilobytes"
+                  scale: "1.0 / 1024.0"
+                  unit: "KB"
+
+Aggregator transformer
+++++++++++++++++++++++
+
+A transformer that sums up the incoming samples until enough samples have
+come in or a timeout has been reached.
+
+Timeout can be specified with the *retention_time* parameter. If we want to
+flush the aggregation after a set number of samples have been aggregated,
+we can specify the *size* parameter.
+
+The volume of the created sample is the sum of the volumes of samples that
+came into the transformer. Samples can be aggregated by the attributes
+*project_id*, *user_id* and *resource_metadata*. To aggregate by the chosen
+attributes, specify them in the configuration and set which value of the
+attribute to take for the new sample (*first* to take the first sample's
+attribute, *last* to take the last sample's attribute, and *drop* to discard
+the attribute).
+
+To aggregate 60s worth of samples by resource_metadata and keep the
+resource_metadata of the latest received sample::
+
+    transformers:
+    - name: "aggregator"
+      parameters:
+          retention_time: 60
+          resource_metadata: last
+
+To aggregate each 15 samples by user_id and resource_metadata and keep the
+user_id of the first received sample and drop the resource_metadata::
+
+    transformers:
+    - name: "aggregator"
+      parameters:
+          size: 15
+          user_id: first
+          resource_metadata: drop
+
+Accumulator transformer
++++++++++++++++++++++++
+
+This transformer simply caches the samples until enough samples have arrived
+and then flushes them all down the pipeline at once.
+::
+
+    transformers:
+    - name: "accumulator"
+      parameters:
+          size: 15
 
-.. _pipeline.yaml: https://git.openstack.org/cgit/openstack/ceilometer/tree/etc/ceilometer/pipeline.yaml