05b0c53133
This commit adds a spec for implementing a unified Infra message bus, the "firehose." Change-Id: I6295aec6025600a9ef02de5249a3a5c834dcba8c Story: #2000643
166 lines
5.4 KiB
ReStructuredText
166 lines
5.4 KiB
ReStructuredText
::
|
|
This work is licensed under a Creative Commons Attribution 3.0
|
|
Unported License.
|
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
|
|
|
==================================================
|
|
Firehose: A unified message bus for Infra services
|
|
==================================================
|
|
|
|
https://storyboard.openstack.org/#!/story/2000643
|
|
|
|
Using MQTT start running a unified message bus that community infrastructure
|
|
services can emit events on. This will enable efficient external consumption of
|
|
infrastructure events for experimentation and rapid innovation of tooling
|
|
within the community.
|
|
|
|
Problem Description
|
|
===================
|
|
|
|
There are a lot of services running in openstack-infra doing lots of different
|
|
things all the time. Following events generated by these services can be
|
|
tricky, and we don't have a consistent solution to making arbitrary state
|
|
change details externally visible (so we fall back to relying on those with
|
|
access to the machines letting us know what operations are being performed).
|
|
What event data does exist is inconsistently presented in a variety of places
|
|
over different protocols and requires significant effort to correlate
|
|
cross-service.
|
|
|
|
|
|
Proposed Change
|
|
===============
|
|
|
|
The goal of this spec is to create a unified message bus that Infra services
|
|
can publish event streams to. This will enable external public consumption of
|
|
Infra events in a consistent manner.
|
|
|
|
To accomplish this we will deploy a new server and run the mosquitto_ MQTT_
|
|
broker on it. MQTT protocol was designed in the late '90s to be lightweight
|
|
pub-sub protocol that would work with connections to remote locations that
|
|
needed a "small code footprint" and where there was limited network bandwith.
|
|
More recently this has made it quite popular in the IoT space. However, it is
|
|
also a good fit for the distributed cloud model that a lot of Infra services
|
|
use. We will also deploy mosquitto using websockets_ this should also provide
|
|
a benefit to international community members where things like the Gerrit event
|
|
stream are problematic to access over Gerrit's SSH port.
|
|
|
|
.. _mosquitto: http://mosquitto.org/
|
|
.. _MQTT: http://mqtt.org/
|
|
.. _websockets: http://jpmens.net/2014/07/03/the-mosquitto-mqtt-broker-gets-websockets-support/
|
|
|
|
That will provide the base infrastructure for publishing and subscribing to
|
|
events but nothing will be using MQTT after that. To start the event reporting
|
|
we'll leverage germqtt_ (and make it an Infra project). This will be the
|
|
initial usage for the unified message bus and make the Gerrit event stream
|
|
available over MQTT. While the scope of this spec ends with getting Gerrit into
|
|
MQTT, once we have an example we can more easily add more sources reporting
|
|
directly into MQTT.
|
|
|
|
.. _germqtt: https://github.com/mtreinish/germqtt
|
|
|
|
Alternatives
|
|
------------
|
|
|
|
Use fedmsg
|
|
^^^^^^^^^^
|
|
The fedora infrastructure team had a similar need/desire to have a unified
|
|
message bus and they started the fedmsg_ project to address this need.
|
|
The primary issue with fedmsg is that it depends on AMQP or ZMQ both of which
|
|
we have had previous issues with. The choice to go with MQTT was partially a
|
|
reaction to that.
|
|
|
|
.. _fedmsg: http://www.fedmsg.com/en/latest/
|
|
|
|
|
|
Use a different mqtt broker
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
There are several open source MQTT brokers besides Mosquitto out there
|
|
including ones like RabbitMQ_, VerneMQ_, and mosca_.
|
|
|
|
.. _RabbitMQ: http://www.rabbitmq.com/blog/2012/09/12/mqtt-adapter/
|
|
.. _VerneMQ: https://github.com/erlio/vernemq
|
|
.. _mosca: https://github.com/mcollina/mosca
|
|
|
|
Mosquitto was chosen because it's a lightweight C program which will need less
|
|
resources to run it. It also should be simpler to deploy since we don't have to
|
|
worry about deploying a JavaVM, nodejs, or erlang (which seems to be quite
|
|
popular for writing message brokers).
|
|
|
|
|
|
Implementation
|
|
==============
|
|
|
|
Assignee(s)
|
|
-----------
|
|
|
|
Primary assignee:
|
|
|
|
* mtreinish
|
|
* fungi
|
|
|
|
|
|
Gerrit Topic
|
|
------------
|
|
|
|
Use Gerrit topic "firehose" for all patches related to this spec.
|
|
|
|
.. code-block:: bash
|
|
|
|
git-review -t firehose
|
|
|
|
Work Items
|
|
----------
|
|
|
|
#. Write puppet-mosquitto module
|
|
#. Deploy firehose01 server with puppet-mosquitto module and a
|
|
firehose.openstack.org CNAME to it
|
|
#. Add germqtt to openstack-infra
|
|
#. Write puppet-germqtt
|
|
#. Deploy puppet-germqtt on firehose01
|
|
|
|
Repositories
|
|
------------
|
|
|
|
2 New repos will be created in the initial implemention, puppet-mosquitto and
|
|
puppet-germqtt.
|
|
|
|
Servers
|
|
-------
|
|
|
|
1 new server will need to be created for running mosquitto, firehose01. Since
|
|
mosquitto and MQTT is fairly lightweight we probably can get away with a
|
|
smaller server (2GB flavor should be plenty).
|
|
|
|
DNS Entries
|
|
-----------
|
|
|
|
Forward/reverse DNS entries will be added for firehose01.openstack.org, as well
|
|
as a CNAME of firehose.openstack.org.
|
|
|
|
Documentation
|
|
-------------
|
|
|
|
Initial service documentation will be added to the
|
|
openstack-infra/system-config repo as usual. An availability announcement will
|
|
be sent to the openstack-dev mailing list.
|
|
|
|
Security
|
|
--------
|
|
|
|
Sensitive data should not be published over the message bus. It is public and
|
|
anonymously accessible.
|
|
|
|
Testing
|
|
-------
|
|
|
|
The new Puppet modules will run our usual Puppet test and style jobs, and the
|
|
germqtt repo will have typical Python unit testing and style checks as well.
|
|
|
|
Dependencies
|
|
============
|
|
|
|
- This will be the first use of germqtt in infra (which will be added as an
|
|
infra project)
|
|
- 2 new puppet modules will be created, 1 to deploy mosquitto and the other to
|
|
run germqtt
|