openstack-ansible-ops/elk_metrics_6x/templates/_include_log_stash_output.yml.j2
Kevin Carter bc2937d9c9
Use elasticsearch coordinator nodes as smart LBs
Elasticsearch can be used as a smart load balancer for all traffic
which will remove the requirement for a VIP and move the cluster to a
mesh topology. All of the Kibana nodes will now run elasticsearch as
cordonator.

* Kibana will now connect to elasticsearch on localhost.
* All of the beats have been setup to use use the new mesh topology.
* jvm memory management has been updated to reflect the additional
  services.

More on node assigments can be found here:
* https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-node.html#modules-node

* The readme has been updated to reflect these changes.

Change-Id: I769e0251072f5dbde56fcce7753236d37d5c3b19
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2018-06-13 23:37:48 -05:00

95 lines
3.7 KiB
Django/Jinja

output.logstash:
# Boolean flag to enable or disable the output module.
enabled: true
# The Logstash hosts
hosts: {{ logstash_data_hosts | shuffle(seed=inventory_hostname) | to_json }}
# Number of workers per Logstash host.
worker: 1
# Set gzip compression level.
compression_level: 3
# Optional maximum time to live for a connection to Logstash, after which the
# connection will be re-established. A value of `0s` (the default) will
# disable this feature.
#
# Not yet supported for async connections (i.e. with the "pipelining" option set)
#ttl: 30s
# Optional load balance the events between the Logstash hosts. Default is false.
loadbalance: true
# Number of batches to be sent asynchronously to logstash while processing
# new batches.
pipelining: 2
# If enabled only a subset of events in a batch of events is transferred per
# transaction. The number of events to be sent increases up to `bulk_max_size`
# if no error is encountered.
slow_start: true
{% set thread_pool_size = ansible_processor_cores * ((ansible_processor_threads_per_core > 0) | ternary(ansible_processor_threads_per_core, 1)) %}
# The maximum number of events to bulk in a single Logstash request. The
# default is the number of cores multiplied by the number of threads,
# the resultant is then multiplied again by 256 which results in a the defined
# bulk max size. If the Beat sends single events, the events are collected
# into batches. If the Beat publishes a large batch of events (larger than
# the value specified by bulk_max_size), the batch is split. Specifying a
# larger batch size can improve performance by lowering the overhead of
# sending events. However big batch sizes can also increase processing times,
# which might result in API errors, killed connections, timed-out publishing
# requests, and, ultimately, lower throughput. Setting bulk_max_size to values
# less than or equal to 0 disables the splitting of batches. When splitting
# is disabled, the queue decides on the number of events to be contained in a
# batch.
bulk_max_size: {{ thread_pool_size * 256 }}
{% if named_index is defined %}
# Optional index name. The default index name is set to {{ named_index }}
# in all lowercase.
index: '{{ named_index }}'
{% endif %}
# SOCKS5 proxy server URL
#proxy_url: socks5://user:password@socks5-server:2233
# Resolve names locally when using a proxy server. Defaults to false.
#proxy_use_local_resolver: false
# Enable SSL support. SSL is automatically enabled, if any SSL setting is set.
#ssl.enabled: true
# Configure SSL verification mode. If `none` is configured, all server hosts
# and certificates will be accepted. In this mode, SSL based connections are
# susceptible to man-in-the-middle attacks. Use only for testing. Default is
# `full`.
#ssl.verification_mode: full
# List of supported/valid TLS versions. By default all TLS versions 1.0 up to
# 1.2 are enabled.
#ssl.supported_protocols: [TLSv1.0, TLSv1.1, TLSv1.2]
# Optional SSL configuration options. SSL is off by default.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
# Optional passphrase for decrypting the Certificate Key.
#ssl.key_passphrase: ''
# Configure cipher suites to be used for SSL connections
#ssl.cipher_suites: []
# Configure curve types for ECDHE based cipher suites
#ssl.curve_types: []
# Configure what types of renegotiation are supported. Valid options are
# never, once, and freely. Default is never.
#ssl.renegotiation: never