opendev/system-config
Code Issues Proposed changes

Handle multiline Logstash events per file.

Browse Source 
* modules/openstack_project/files/logstash/log-pusher.py: Add a filename
field to logstash events that can be used to associate multiline events
in files to their appropriate parents in the same file.

* modules/openstack_project/templates/logstash/indexer.conf.erb: Add
stream_identities to the mutliline filters that use the source host and
file name to determine relationships between mutliline events.

Change-Id: Ia325c0e1257131ab1b721c4df8f70f6bea1d0b99
Reviewed-on: https://review.openstack.org/30953
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
This commit is contained in:
Clark Boylan 2013-05-29 14:43:44 -07:00 committed by Jenkins
parent 9ed2be3098
commit c9d484e797
2 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
modules/openstack_project/files/logstash/log-pusher.py
View File

@ -133,6 +133,7 @@ class LogRetriever(threading.Thread):
def _parse_fields(self, event):
fields = {}
fields["filename"] = self.filename
fields["build_name"] = event.get("name", "UNKNOWN")
fields["build_status"] = event["build"].get("status", "UNKNOWN")
fields["build_number"] = event["build"].get("number", "UNKNOWN")

2
modules/openstack_project/templates/logstash/indexer.conf.erb
View File

@ -32,6 +32,7 @@ filter {
negate => true
pattern => "^%{DATESTAMP} \|"
what => "previous"
stream_identity => "%{@source_host}.%{filename}"
}
multiline {
type => "jenkins"
@ -39,6 +40,7 @@ filter {
negate => true
pattern => "^%{DATESTAMP} "
what => "previous"
stream_identity => "%{@source_host}.%{filename}"
}
grok {
type => "jenkins"