.. This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode ============= Aodh Notifier ============= launchpad blueprint: https://blueprints.launchpad.net/vitrage/+spec/aodh-notifier The Evaluator performs root cause analysis on the Vitrage Graph and may determine that an alarm should be created, deleted or otherwise updated. Other components are notified of such changes by the Vitrage Notifier service. Among others, Vitrage Notifier is responsible for handling Aodh Alarms. This blueprint describes the implementation of Vitrage Notifier for notifying Aodh on Vitrage alarms. :: +------------------+ +------------------+ +------------------+ | Aodh <--+ | | | | +------------------+ | Update | Vitrage | Raise | Vitrage | +--------| <----------| | +------------------+ | Alarm | Notifier | Alarm | Evaluator | | Other components <--+ | | | | +------------------+ +------------------+ +------------------+ Problem description =================== Vitrage should be capable of creating, deleting and otherwise updating alarms as requested by the Evaluator Engine. The notifier is responsible for ensuring these updates are executed. Specifically we will start here with Aodh alarms. Main challenges: * There is no way to define a 'custom alarm' in Aodh * Vitrage alarms are based on resources. There is a need to pass the resource information to Aodh * Several alarms of the same type can be triggered at the same time, each for a different resource. For example, in case there is an alarm on a host, Vitrage will raise a deduced alarm on every instance in this host. * How can someone ask for notifications on updates of Vitrage alarms? Proposed change =============== The Vitrage Notifier will be separate from the Evaluator, as the two will have different demands of scale and other performance considerations. The Vitrage Notifier will supply an API used by the Vitrage Evaluator, containing create/delete/update alarm. In Aodh, Vitrage alarms will be defined as event alarms, this seems like the most appropriate option. The resource id will be defined in the alarm query. Vitrage deduced alarms will look like this: +---------------------------+---------------------------------------------------------+ | Property | Value | +---------------------------+---------------------------------------------------------+ | alarm_actions | [] | +---------------------------+---------------------------------------------------------+ | alarm_id | 4a3cb988-a620-4bf3-87f7-077c751c408f | +---------------------------+---------------------------------------------------------+ | description | Instance is unreachable | +---------------------------+---------------------------------------------------------+ | enabled | True | +---------------------------+---------------------------------------------------------+ | event_type | vitrage.alarm.instance_unreachable | +---------------------------+---------------------------------------------------------+ | insufficient_data_actions | [] | +---------------------------+---------------------------------------------------------+ | name | vitrage_instance_unreachable_1 | +---------------------------+---------------------------------------------------------+ | ok_actions | [] | +---------------------------+---------------------------------------------------------+ | project_id | 5542b27142154f30b32dea6238aa81aa | +---------------------------+---------------------------------------------------------+ | query | [{u'field': u'resource_id', u'type': u'', u'value': | | | u'b0bf3635-d9e8-4624-9793-7aac82948c0a', u'op': u'eq'}] | +---------------------------+---------------------------------------------------------+ | repeat_actions | False | +---------------------------+---------------------------------------------------------+ | severity | moderate | +---------------------------+---------------------------------------------------------+ | state | alarm | +---------------------------+---------------------------------------------------------+ | type | event | +---------------------------+---------------------------------------------------------+ | user_id | 8ab65ef808b245e3ba234b7b3554cb94 | +---------------------------+---------------------------------------------------------+ In this example, Vitrage triggers a deduced alarm that an instance is unreachable due to a failure in the public switch (which was detected by Nagios). There will be several alarms with the same event_type and different instance ids in their query. There are two options how to trigger Vitrage alarms in Aodh, none is perfect. Alternative 1 ------------- Vitrage will create an event alarm in Aodh. Then, it will send a notification to the message bus. The notification will be converted to a Ceilometer event, which will trigger the Aodh alarm. The exact notification and event format are still TBD. The main problem with this solution is that the Aodh alarm will be created on-the-fly and triggered immediately, so it will be impossible for another project to register a web-hook on the alarm before it is triggered. It will be possbile to see Vitrage alarms in list-alarms, but not to be notified when they are first triggered. Alternative 2 ------------- Vitrage will create an event alarm in Aodh, with 'alarm' state. The event itself will never be sent, so the alarm state will remain 'alarm'. The problem with this solution is that Aodh will not send a notification about the alarm being triggered. But since in Alternative 1 it is also impossible to register on the alarm, there is no real difference between the two options. Data model impact ----------------- None REST API impact --------------- None Versioning impact ----------------- None Other end user impact --------------------- None Deployer impact --------------- For Alternative 1 - there is a need to define the notification->event configuration For Alternative 2 - None Developer impact ---------------- None Horizon impact -------------- None Implementation ============== Assignee(s) ----------- Primary assignee: idan-hefetz Work Items ---------- None Dependencies ============ None Testing ======= This blueprint requires unit tests and Tempest tests. Documentation Impact ==================== For Alternative 1 - there is a need to document the notification->event configuration For Alternative 2 - None References ========== Vitrage wiki page: https://wiki.openstack.org/wiki/Vitrage Vitrage use cases: https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-use-cases.rst