62c2ebbfa1
This adds additional playbooks to deploy:
- carbon-cache
- graphite-web
- grafana
as containers. The data should map to the host running dockerd
for data persistence across reboots.
Ensure:
- persistent data if containers are restarted
- systemd scripts to ensure containers startup
- new ports defined in the group_vars/all.yml
- fixed mismatch between templates and variables
- minor space issues to keep folks happy
- Use a help script for docker ps to avoid weirdness with
'{{.Names}}' in the command line when using the command plugin.
These have been tested on RHEL7 hosts, as well as undercloud
deployed via RDO quickstart. The following variables are
of note in the group_vars/all.yml and how to use them:
Location on the target docker host where whisper data is kept:
persistent_carbon_data_path: /data/carbon/whisper
Location on the target docker host where grafana runs to ensure
persistence of grafana related data across reboots (e.g. dashboards, etc):
persistent_grafana_data_path: /data/grafana
This port is mapped from the docker host to the standard carbon-cache port
on the launched container (basically makes it easy to ensure ports don't
clash in case there is anothre service listening here):
docker_carbon_cache_port: 2003
This is the web port on the docker host that is proxied back into the
container (useful when there are a number of other web services running,
e.g. if deploying on an undercloud host):
docker_graphite_port: 8888
Same with the following. It's not likely there will be clash with this
port but just in case you are running an instance of grafana, this
allows mapping an alternate port to the docker container:
docker_grafana_port: 3000
This is the published docker image for carbon-cache. This is based on
the centos7 image using python-carbon rpms. This is published into
hub.docker.com. See: https://hub.docker.com/r/kambiz/carbon-cache/
carbon_cache_docker_image: kambiz/carbon-cache:0.9.15
This is the published docker image for graphite-web. This is based on
the centos7 image using httpd and graphite-web rpms. This is published
into hub.docker.com. See: https://hub.docker.com/r/kambiz/graphite-web/
graphite_web_docker_image: kambiz/graphite-web:0.9.15
This is the official grafana project docker image. The image will never
change since we are using 2.6.0, rather than "latest". See the images
here: https://hub.docker.com/r/grafana/grafana/
grafana_docker_image: grafana/grafana:2.6.0
Change-Id: Icf63d0b044cded56cbecc7d306847dedd84bec87
371 lines
15 KiB
Plaintext
371 lines
15 KiB
Plaintext
[cache]
|
|
# Configure carbon directories.
|
|
#
|
|
# OS environment variables can be used to tell carbon where graphite is
|
|
# installed, where to read configuration from and where to write data.
|
|
#
|
|
# GRAPHITE_ROOT - Root directory of the graphite installation.
|
|
# Defaults to ../
|
|
# GRAPHITE_CONF_DIR - Configuration directory (where this file lives).
|
|
# Defaults to $GRAPHITE_ROOT/conf/
|
|
# GRAPHITE_STORAGE_DIR - Storage directory for whipser/rrd/log/pid files.
|
|
# Defaults to $GRAPHITE_ROOT/storage/
|
|
#
|
|
# To change other directory paths, add settings to this file. The following
|
|
# configuration variables are available with these default values:
|
|
#
|
|
# STORAGE_DIR = $GRAPHITE_STORAGE_DIR
|
|
# LOCAL_DATA_DIR = STORAGE_DIR/whisper/
|
|
# WHITELISTS_DIR = STORAGE_DIR/lists/
|
|
# CONF_DIR = STORAGE_DIR/conf/
|
|
# LOG_DIR = STORAGE_DIR/log/
|
|
# PID_DIR = STORAGE_DIR/
|
|
#
|
|
# For FHS style directory structures, use:
|
|
#
|
|
# STORAGE_DIR = /var/lib/carbon/
|
|
# CONF_DIR = /etc/carbon/
|
|
# LOG_DIR = /var/log/carbon/
|
|
# PID_DIR = /var/run/
|
|
#
|
|
#LOCAL_DATA_DIR = /opt/graphite/storage/whisper/
|
|
|
|
STORAGE_DIR = /var/lib/carbon/
|
|
LOCAL_DATA_DIR = /var/lib/carbon/whisper/
|
|
WHITELISTS_DIR = /var/lib/carbon/lists/
|
|
CONF_DIR = /etc/carbon/
|
|
LOG_DIR = /var/log/carbon/
|
|
PID_DIR = /var/run/
|
|
|
|
# Enable daily log rotation. If disabled, a kill -HUP can be used after a manual rotate
|
|
|
|
ENABLE_LOGROTATION = True
|
|
|
|
# Specify the user to drop privileges to
|
|
# If this is blank carbon runs as the user that invokes it
|
|
# This user must have write access to the local data directory
|
|
USER = carbon
|
|
|
|
#
|
|
# NOTE: The above settings must be set under [relay] and [aggregator]
|
|
# to take effect for those daemons as well
|
|
|
|
# Limit the size of the cache to avoid swapping or becoming CPU bound.
|
|
# Sorts and serving cache queries gets more expensive as the cache grows.
|
|
# Use the value "inf" (infinity) for an unlimited cache size.
|
|
MAX_CACHE_SIZE = inf
|
|
|
|
# Limits the number of whisper update_many() calls per second, which effectively
|
|
# means the number of write requests sent to the disk. This is intended to
|
|
# prevent over-utilizing the disk and thus starving the rest of the system.
|
|
# When the rate of required updates exceeds this, then carbon's caching will
|
|
# take effect and increase the overall throughput accordingly.
|
|
MAX_UPDATES_PER_SECOND = 500
|
|
|
|
# If defined, this changes the MAX_UPDATES_PER_SECOND in Carbon when a
|
|
# stop/shutdown is initiated. This helps when MAX_UPDATES_PER_SECOND is
|
|
# relatively low and carbon has cached a lot of updates; it enables the carbon
|
|
# daemon to shutdown more quickly.
|
|
# MAX_UPDATES_PER_SECOND_ON_SHUTDOWN = 1000
|
|
|
|
# Softly limits the number of whisper files that get created each minute.
|
|
# Setting this value low (like at 50) is a good way to ensure your graphite
|
|
# system will not be adversely impacted when a bunch of new metrics are
|
|
# sent to it. The trade off is that it will take much longer for those metrics'
|
|
# database files to all get created and thus longer until the data becomes usable.
|
|
# Setting this value high (like "inf" for infinity) will cause graphite to create
|
|
# the files quickly but at the risk of slowing I/O down considerably for a while.
|
|
MAX_CREATES_PER_MINUTE = 2000
|
|
|
|
LINE_RECEIVER_INTERFACE = 0.0.0.0
|
|
LINE_RECEIVER_PORT = 2003
|
|
|
|
# Set this to True to enable the UDP listener. By default this is off
|
|
# because it is very common to run multiple carbon daemons and managing
|
|
# another (rarely used) port for every carbon instance is not fun.
|
|
ENABLE_UDP_LISTENER = False
|
|
UDP_RECEIVER_INTERFACE = 0.0.0.0
|
|
UDP_RECEIVER_PORT = 2003
|
|
|
|
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
|
|
PICKLE_RECEIVER_PORT = 2004
|
|
|
|
# Set to false to disable logging of successful connections
|
|
LOG_LISTENER_CONNECTIONS = True
|
|
|
|
# Per security concerns outlined in Bug #817247 the pickle receiver
|
|
# will use a more secure and slightly less efficient unpickler.
|
|
# Set this to True to revert to the old-fashioned insecure unpickler.
|
|
USE_INSECURE_UNPICKLER = False
|
|
|
|
CACHE_QUERY_INTERFACE = 0.0.0.0
|
|
CACHE_QUERY_PORT = 7002
|
|
|
|
# Set this to False to drop datapoints received after the cache
|
|
# reaches MAX_CACHE_SIZE. If this is True (the default) then sockets
|
|
# over which metrics are received will temporarily stop accepting
|
|
# data until the cache size falls below 95% MAX_CACHE_SIZE.
|
|
USE_FLOW_CONTROL = True
|
|
|
|
# By default, carbon-cache will log every whisper update and cache hit. This can be excessive and
|
|
# degrade performance if logging on the same volume as the whisper data is stored.
|
|
LOG_UPDATES = False
|
|
LOG_CACHE_HITS = False
|
|
LOG_CACHE_QUEUE_SORTS = True
|
|
|
|
# The thread that writes metrics to disk can use on of the following strategies
|
|
# determining the order in which metrics are removed from cache and flushed to
|
|
# disk. The default option preserves the same behavior as has been historically
|
|
# available in version 0.9.10.
|
|
#
|
|
# sorted - All metrics in the cache will be counted and an ordered list of
|
|
# them will be sorted according to the number of datapoints in the cache at the
|
|
# moment of the list's creation. Metrics will then be flushed from the cache to
|
|
# disk in that order.
|
|
#
|
|
# max - The writer thread will always pop and flush the metric from cache
|
|
# that has the most datapoints. This will give a strong flush preference to
|
|
# frequently updated metrics and will also reduce random file-io. Infrequently
|
|
# updated metrics may only ever be persisted to disk at daemon shutdown if
|
|
# there are a large number of metrics which receive very frequent updates OR if
|
|
# disk i/o is very slow.
|
|
#
|
|
# naive - Metrics will be flushed from the cache to disk in an unordered
|
|
# fashion. This strategy may be desirable in situations where the storage for
|
|
# whisper files is solid state, CPU resources are very limited or deference to
|
|
# the OS's i/o scheduler is expected to compensate for the random write
|
|
# pattern.
|
|
#
|
|
CACHE_WRITE_STRATEGY = sorted
|
|
|
|
# On some systems it is desirable for whisper to write synchronously.
|
|
# Set this option to True if you'd like to try this. Basically it will
|
|
# shift the onus of buffering writes from the kernel into carbon's cache.
|
|
WHISPER_AUTOFLUSH = False
|
|
|
|
# By default new Whisper files are created pre-allocated with the data region
|
|
# filled with zeros to prevent fragmentation and speed up contiguous reads and
|
|
# writes (which are common). Enabling this option will cause Whisper to create
|
|
# the file sparsely instead. Enabling this option may allow a large increase of
|
|
# MAX_CREATES_PER_MINUTE but may have longer term performance implications
|
|
# depending on the underlying storage configuration.
|
|
# WHISPER_SPARSE_CREATE = False
|
|
|
|
# Only beneficial on linux filesystems that support the fallocate system call.
|
|
# It maintains the benefits of contiguous reads/writes, but with a potentially
|
|
# much faster creation speed, by allowing the kernel to handle the block
|
|
# allocation and zero-ing. Enabling this option may allow a large increase of
|
|
# MAX_CREATES_PER_MINUTE. If enabled on an OS or filesystem that is unsupported
|
|
# this option will gracefully fallback to standard POSIX file access methods.
|
|
WHISPER_FALLOCATE_CREATE = True
|
|
|
|
# Enabling this option will cause Whisper to lock each Whisper file it writes
|
|
# to with an exclusive lock (LOCK_EX, see: man 2 flock). This is useful when
|
|
# multiple carbon-cache daemons are writing to the same files
|
|
# WHISPER_LOCK_WRITES = False
|
|
|
|
# Set this to True to enable whitelisting and blacklisting of metrics in
|
|
# CONF_DIR/whitelist and CONF_DIR/blacklist. If the whitelist is missing or
|
|
# empty, all metrics will pass through
|
|
# USE_WHITELIST = False
|
|
|
|
# By default, carbon itself will log statistics (such as a count,
|
|
# metricsReceived) with the top level prefix of 'carbon' at an interval of 60
|
|
# seconds. Set CARBON_METRIC_INTERVAL to 0 to disable instrumentation
|
|
# CARBON_METRIC_PREFIX = carbon
|
|
# CARBON_METRIC_INTERVAL = 60
|
|
|
|
# Enable AMQP if you want to receve metrics using an amqp broker
|
|
# ENABLE_AMQP = False
|
|
|
|
# Verbose means a line will be logged for every metric received
|
|
# useful for testing
|
|
# AMQP_VERBOSE = False
|
|
|
|
# AMQP_HOST = localhost
|
|
# AMQP_PORT = 5672
|
|
# AMQP_VHOST = /
|
|
# AMQP_USER = guest
|
|
# AMQP_PASSWORD = guest
|
|
# AMQP_EXCHANGE = graphite
|
|
# AMQP_METRIC_NAME_IN_BODY = False
|
|
|
|
# The manhole interface allows you to SSH into the carbon daemon
|
|
# and get a python interpreter. BE CAREFUL WITH THIS! If you do
|
|
# something like time.sleep() in the interpreter, the whole process
|
|
# will sleep! This is *extremely* helpful in debugging, assuming
|
|
# you are familiar with the code. If you are not, please don't
|
|
# mess with this, you are asking for trouble :)
|
|
#
|
|
# ENABLE_MANHOLE = False
|
|
# MANHOLE_INTERFACE = 127.0.0.1
|
|
# MANHOLE_PORT = 7222
|
|
# MANHOLE_USER = admin
|
|
# MANHOLE_PUBLIC_KEY = ssh-rsa AAAAB3NzaC1yc2EAAAABiwAaAIEAoxN0sv/e4eZCPpi3N3KYvyzRaBaMeS2RsOQ/cDuKv11dlNzVeiyc3RFmCv5Rjwn/lQ79y0zyHxw67qLyhQ/kDzINc4cY41ivuQXm2tPmgvexdrBv5nsfEpjs3gLZfJnyvlcVyWK/lId8WUvEWSWHTzsbtmXAF2raJMdgLTbQ8wE=
|
|
|
|
# Patterns for all of the metrics this machine will store. Read more at
|
|
# http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol#Bindings
|
|
#
|
|
# Example: store all sales, linux servers, and utilization metrics
|
|
# BIND_PATTERNS = sales.#, servers.linux.#, #.utilization
|
|
#
|
|
# Example: store everything
|
|
# BIND_PATTERNS = #
|
|
|
|
# To configure special settings for the carbon-cache instance 'b', uncomment this:
|
|
#[cache:b]
|
|
#LINE_RECEIVER_PORT = 2103
|
|
#PICKLE_RECEIVER_PORT = 2104
|
|
#CACHE_QUERY_PORT = 7102
|
|
# and any other settings you want to customize, defaults are inherited
|
|
# from [carbon] section.
|
|
# You can then specify the --instance=b option to manage this instance
|
|
|
|
|
|
|
|
[relay]
|
|
USER = carbon
|
|
LINE_RECEIVER_INTERFACE = 0.0.0.0
|
|
LINE_RECEIVER_PORT = 2013
|
|
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
|
|
PICKLE_RECEIVER_PORT = 2014
|
|
|
|
# Set to false to disable logging of successful connections
|
|
LOG_LISTENER_CONNECTIONS = True
|
|
|
|
# Carbon-relay has several options for metric routing controlled by RELAY_METHOD
|
|
#
|
|
# Use relay-rules.conf to route metrics to destinations based on pattern rules
|
|
#RELAY_METHOD = rules
|
|
#
|
|
# Use consistent-hashing for even distribution of metrics between destinations
|
|
#RELAY_METHOD = consistent-hashing
|
|
#
|
|
# Use consistent-hashing but take into account an aggregation-rules.conf shared
|
|
# by downstream carbon-aggregator daemons. This will ensure that all metrics
|
|
# that map to a given aggregation rule are sent to the same carbon-aggregator
|
|
# instance.
|
|
# Enable this for carbon-relays that send to a group of carbon-aggregators
|
|
#RELAY_METHOD = aggregated-consistent-hashing
|
|
RELAY_METHOD = rules
|
|
|
|
# If you use consistent-hashing you can add redundancy by replicating every
|
|
# datapoint to more than one machine.
|
|
REPLICATION_FACTOR = 1
|
|
|
|
# This is a list of carbon daemons we will send any relayed or
|
|
# generated metrics to. The default provided would send to a single
|
|
# carbon-cache instance on the default port. However if you
|
|
# use multiple carbon-cache instances then it would look like this:
|
|
#
|
|
# DESTINATIONS = 127.0.0.1:2004:a, 127.0.0.1:2104:b
|
|
#
|
|
# The general form is IP:PORT:INSTANCE where the :INSTANCE part is
|
|
# optional and refers to the "None" instance if omitted.
|
|
#
|
|
# Note that if the destinations are all carbon-caches then this should
|
|
# exactly match the webapp's CARBONLINK_HOSTS setting in terms of
|
|
# instances listed (order matters!).
|
|
#
|
|
# If using RELAY_METHOD = rules, all destinations used in relay-rules.conf
|
|
# must be defined in this list
|
|
DESTINATIONS = 127.0.0.1:2004
|
|
|
|
# This defines the maximum "message size" between carbon daemons.
|
|
# You shouldn't need to tune this unless you really know what you're doing.
|
|
MAX_DATAPOINTS_PER_MESSAGE = 2000
|
|
MAX_QUEUE_SIZE = 400000
|
|
|
|
# Set this to False to drop datapoints when any send queue (sending datapoints
|
|
# to a downstream carbon daemon) hits MAX_QUEUE_SIZE. If this is True (the
|
|
# default) then sockets over which metrics are received will temporarily stop accepting
|
|
# data until the send queues fall below 80% MAX_QUEUE_SIZE.
|
|
USE_FLOW_CONTROL = True
|
|
|
|
# Set this to True to enable whitelisting and blacklisting of metrics in
|
|
# CONF_DIR/whitelist and CONF_DIR/blacklist. If the whitelist is missing or
|
|
# empty, all metrics will pass through
|
|
# USE_WHITELIST = False
|
|
|
|
# By default, carbon itself will log statistics (such as a count,
|
|
# metricsReceived) with the top level prefix of 'carbon' at an interval of 60
|
|
# seconds. Set CARBON_METRIC_INTERVAL to 0 to disable instrumentation
|
|
# CARBON_METRIC_PREFIX = carbon
|
|
# CARBON_METRIC_INTERVAL = 60
|
|
|
|
|
|
[aggregator]
|
|
USER = carbon
|
|
LINE_RECEIVER_INTERFACE = 0.0.0.0
|
|
LINE_RECEIVER_PORT = 2023
|
|
|
|
PICKLE_RECEIVER_INTERFACE = 0.0.0.0
|
|
PICKLE_RECEIVER_PORT = 2024
|
|
|
|
# Set to false to disable logging of successful connections
|
|
LOG_LISTENER_CONNECTIONS = True
|
|
|
|
# If set true, metric received will be forwarded to DESTINATIONS in addition to
|
|
# the output of the aggregation rules. If set false the carbon-aggregator will
|
|
# only ever send the output of aggregation.
|
|
FORWARD_ALL = True
|
|
|
|
# This is a list of carbon daemons we will send any relayed or
|
|
# generated metrics to. The default provided would send to a single
|
|
# carbon-cache instance on the default port. However if you
|
|
# use multiple carbon-cache instances then it would look like this:
|
|
#
|
|
# DESTINATIONS = 127.0.0.1:2004:a, 127.0.0.1:2104:b
|
|
#
|
|
# The format is comma-delimited IP:PORT:INSTANCE where the :INSTANCE part is
|
|
# optional and refers to the "None" instance if omitted.
|
|
#
|
|
# Note that if the destinations are all carbon-caches then this should
|
|
# exactly match the webapp's CARBONLINK_HOSTS setting in terms of
|
|
# instances listed (order matters!).
|
|
DESTINATIONS = 127.0.0.1:2004
|
|
|
|
# If you want to add redundancy to your data by replicating every
|
|
# datapoint to more than one machine, increase this.
|
|
REPLICATION_FACTOR = 1
|
|
|
|
# This is the maximum number of datapoints that can be queued up
|
|
# for a single destination. Once this limit is hit, we will
|
|
# stop accepting new data if USE_FLOW_CONTROL is True, otherwise
|
|
# we will drop any subsequently received datapoints.
|
|
MAX_QUEUE_SIZE = 10000
|
|
|
|
# Set this to False to drop datapoints when any send queue (sending datapoints
|
|
# to a downstream carbon daemon) hits MAX_QUEUE_SIZE. If this is True (the
|
|
# default) then sockets over which metrics are received will temporarily stop accepting
|
|
# data until the send queues fall below 80% MAX_QUEUE_SIZE.
|
|
USE_FLOW_CONTROL = True
|
|
|
|
# This defines the maximum "message size" between carbon daemons.
|
|
# You shouldn't need to tune this unless you really know what you're doing.
|
|
MAX_DATAPOINTS_PER_MESSAGE = 500
|
|
|
|
# This defines how many datapoints the aggregator remembers for
|
|
# each metric. Aggregation only happens for datapoints that fall in
|
|
# the past MAX_AGGREGATION_INTERVALS * intervalSize seconds.
|
|
MAX_AGGREGATION_INTERVALS = 5
|
|
|
|
# By default (WRITE_BACK_FREQUENCY = 0), carbon-aggregator will write back
|
|
# aggregated data points once every rule.frequency seconds, on a per-rule basis.
|
|
# Set this (WRITE_BACK_FREQUENCY = N) to write back all aggregated data points
|
|
# every N seconds, independent of rule frequency. This is useful, for example,
|
|
# to be able to query partially aggregated metrics from carbon-cache without
|
|
# having to first wait rule.frequency seconds.
|
|
# WRITE_BACK_FREQUENCY = 0
|
|
|
|
# Set this to True to enable whitelisting and blacklisting of metrics in
|
|
# CONF_DIR/whitelist and CONF_DIR/blacklist. If the whitelist is missing or
|
|
# empty, all metrics will pass through
|
|
# USE_WHITELIST = False
|
|
|
|
# By default, carbon itself will log statistics (such as a count,
|
|
# metricsReceived) with the top level prefix of 'carbon' at an interval of 60
|
|
# seconds. Set CARBON_METRIC_INTERVAL to 0 to disable instrumentation
|
|
# CARBON_METRIC_PREFIX = carbon
|
|
# CARBON_METRIC_INTERVAL = 60
|