openstack-helm-infra/fluent-logging
Steve Wilkerson 70afe83c16 Helm-Toolkit: Add snippet for kubernetes tolerations
This adds a helm-toolkit template for injecting pod tolerations
via values, similar to how container resources are handled. This
allows for custom definition of tolerations instead of defining
tolerations for pods directly into the pod templates

Change-Id: Ice520fcece425b14ae890ca5980fec9d7428a34d
2018-09-18 13:10:54 +00:00
..
templates Helm-Toolkit: Add snippet for kubernetes tolerations 2018-09-18 13:10:54 +00:00
Chart.yaml Fluent-logging helm chart 2017-12-15 10:52:16 -06:00
README.rst Remove the duplicated word 2018-06-10 19:04:54 -04:00
requirements.yaml fluent-logging: yaml indentation fixes 2018-05-11 08:48:21 +00:00
values.yaml Helm-Toolkit: Add snippet for kubernetes tolerations 2018-09-18 13:10:54 +00:00

Fluentd-logging

OpenStack-Helm defines a centralized logging mechanism to provide insight into the state of the OpenStack services and infrastructure components as well as underlying kubernetes platform. Among the requirements for a logging platform, where log data can come from and where log data need to be delivered are very variable. To support various logging scenarios, OpenStack-Helm should provide a flexible mechanism to meet with certain operation needs. This chart proposes fast and lightweight log forwarder and full featured log aggregator complementing each other providing a flexible and reliable solution. Especially, Fluent-bit is proposed as a log forwarder and Fluentd is proposed as a main log aggregator and processor.

Mechanism

Fluent-bit, Fluentd meet OpenStack-Helm's logging requirements for gathering, aggregating, and delivering of logged events. Flunt-bit runs as a daemonset on each node and mounts the /var/lib/docker/containers directory. The Docker container runtime engine directs events posted to stdout and stderr to this directory on the host. Fluent-bit then forward the contents of that directory to Fluentd. Fluentd runs as deployment at the designated nodes and expose service for Fluent-bit to foward logs. Fluentd should then apply the Logstash format to the logs. Fluentd can also write kubernetes and OpenStack metadata to the logs. Fluentd will then forward the results to Elasticsearch and to optionally kafka. Elasticsearch indexes the logs in a logstash-* index by default. kafka stores the logs in a 'logs' topic by default. Any external tool can then consume the 'logs' topic.