opendev/base-jobs
Code Issues Proposed changes

Stop treating .gz files as special in log handling

Browse Source 
With our switch to swift hosted logs files ended in .gz are no longer
equivalent to the filenames without the .gz. Swift expects us to be
speific. For this reason normalize on using the actual file names as
stored and reported rather than removing the suffix for human
friendliness.

Note we keep the .gz less tag value in the tags list. This is because
e-r typically uses tags to identify files rather than pure paths or
filenames.

Change-Id: Ie9063f0ab35317357280690d9ad5e273025e3240
Related-Change: https://review.opendev.org/#/c/677236
This commit is contained in:
Clark Boylan 2019-08-23 13:35:59 -07:00
parent 8786199a2d
commit 2f355836ae
1 changed files with 7 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
roles/submit-log-processor-jobs/library/submit_log_processor_jobs.py
View File

@ -38,16 +38,7 @@ class FileMatcher(object):
class File(object):
def __init__(self, name, tags):
# Note that even if we upload a .gz we want to use the logical
# non compressed name for handling (it is easier on humans).
# The reason we can get away with this is that this name is used
# to construct the log_url below. The server serving that
# log_url treats foo.txt and foo.txt.gz as being the same content
# and serves both paths from the same backend content.
if name.endswith('.gz'):
self._name = name[:-3]
else:
self._name = name
self._name = name
self._tags = tags
@property
@ -128,8 +119,12 @@ class LogMatcher(object):
def makeEvent(self, file_object):
out_event = {}
out_event["fields"] = self.makeFields(file_object.name)
out_event["tags"] = [os.path.basename(file_object.name)] + \
file_object.tags
basename = os.path.basename(file_object.name)
out_event["tags"] = [basename] + file_object.tags
if basename.endswith(".gz"):
# Backward compat for e-r which relies on tag values
# without the .gx suffix
out_event["tags"].append(basename[:-3])
return out_event
def makeFields(self, filename):