opendev/system-config
Code Issues Proposed changes

Don't treat IDs as uniquely special in CRM114

Browse Source 
The openstack logs a full of various IDs and UUIDs but they are not
uniquely special when it comes to filtering them. Instead replace each
ID with a token making CRM114's life much easier.

Change-Id: Id9b430c0d31889b89e4e0c1790a2405d73f501b5
This commit is contained in:
Clark Boylan 2014-03-24 12:16:56 -07:00
parent 716ae11cfc
commit 99e530479a
1 changed files with 33 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
modules/log_processor/files/classify-log.crm
View File

@ -57,6 +57,39 @@ learn [:_nl:] <osb unique microgroom> (:*:prefix:/FAILURE.css)
match (:timestamp:) /^[-.0-9 |:]+/
alter (:timestamp:) //
}
{
# Don't treat UUIDs as uniquely special.
match (:uuidtoken:) /[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}/
alter (:uuidtoken:) /UUIDTOKEN/
{
match (:uuidtoken:) <fromnext> /[[:xdigit:]]{8}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{4}-[[:xdigit:]]{12}/
alter (:uuidtoken:) /UUIDTOKEN/
# Loop to replace all TOKENS in line
liaf
}
}
{
# Don't treat IDs as uniquely special.
match (:idtoken:) /[[:xdigit:]]{32,40}/
alter (:idtoken:) /IDTOKEN/
{
match (:idtoken:) <fromnext> /[[:xdigit:]]{32,40}/
alter (:idtoken:) /IDTOKEN/
# Loop to replace all TOKENS in line
liaf
}
}
{
# Don't treat IDs as uniquely special.
match (:numtoken:) /-[[:digit:]]{7,}/
alter (:numtoken:) /-NUMTOKEN/
{
match (:numtoken:) <fromnext> /-[[:digit:]]{7,}/
alter (:numtoken:) /-NUMTOKEN/
# Loop to replace all TOKENS in line
liaf
}
}
# Train on the line
learn <osb unique microgroom> (:*:prefix:/:*:target:.css)
# Classify the line to see if it looks more like a SUCCESS or FAILURE line