valet/doc/ostro_listener.md

OpenStack Event Listener for Ostro

This script listens for specific messages needed by Ostro to maintain up-to-date cloud state. It can persist digested versions of the messages to Music, which Ostro then picks up.

Note: This version of the listener does not use oslo.messaging. It listens directly to RabbitMQ. Future revisions are expected to use oslo.messaging in order to keep the means of transport abstract.

Prerequisites

Prior to installation:

• Ubuntu 14.04 LTS
• Python 2.7.6 with pip
• Access to an OpenStack Kilo cloud (RabbitMQ in particular)
• Access to Music and Ostro

Throughout this document, the following installation-specific items are required. Have values for these prepared and ready before continuing. Suggestions for values are provided in this document where applicable.

Name	Description	Example
$USER	User id	user1234
$VENV	Python virtual environment path (if any)	/etc/ostro-listener/venv
$OSTRO_LISTENER_PATH	Local git repository's ostro_listener directory	/home/user1234/git/allegro/ostro_listener
$CONFIG_FILE	Event Listener configuration file	/etc/ostro-listener/ostro-listener.conf
$RABBITMQ_HOST	RabbitMQ hostname or IP address	localhost
$RABBITMQ_USERNAME	RabbitMQ username	guest
$RABBITMQ_PASSWORD_FILE	Full path to RabbitMQ password file	/etc/ostro-listener/passwd
$MUSIC_URL	Music API endpoints and port in URL format	http://127.0.0.1:8080/
$MUSIC_KEYSPACE	Music keyspace	valet
$MUSIC_REPLICATION_FACTOR	Music replication factor	1

Root or sufficient sudo privileges are required for some steps.

A Note About Python Virtual Environments

It is recommended to consider using a python virtuals environment (venv). A venv helps avoid instabilities and conflicts within the default python environment.

Installation

Install ostro-listener on any node with line-of-sight to the RabbitMQ endpoint from which messages are to be monitored.

ostro-listener is located in ostro_listener.

$ cd $OSTRO_LISTENER_PATH
$ sudo pip install .

Command Line Usage

usage: ostro-listener [-h] [-c OSTRO_LISTENER_CONFIG] [-x EXCHANGE]
                      [-t {topic,fanout}] [-a] [-H HOST] [-p PORT]
                      [-u USERNAME] [-P PASSWDFILE] [-o {yaml,json,dict}] [-s]
                      [-m MUSIC] [-k KEYSPACE] [-r REPLICATION_FACTOR]

Ostro-specific OpenStack Event Listener

optional arguments:
  -h, --help            show this help message and exit
  -c OSTRO_LISTENER_CONFIG, --conf_file OSTRO_LISTENER_CONFIG
                        Defaults to env[OSTRO_LISTENER_CONFIG]
  -x EXCHANGE, --exchange EXCHANGE
                        rabbit exchange to listen to
  -t {topic,fanout}, --exchange_type {topic,fanout}
                        type of exchange (default="topic")
  -a, --auto_delete     autodelete exchange (default=False)
  -H HOST, --host HOST  compute node on which rabbitmq is running
  -p PORT, --port PORT  port on which rabbitmq is running
  -u USERNAME, --username USERNAME
                        rabbitmq username (default="guest")
  -P PASSWDFILE, --passwdfile PASSWDFILE
                        file containing host rabbitmq passwords
  -o {yaml,json,dict}, --output_format {yaml,json,dict}
                        output format (default="dict")
  -s, --store           store messages in music (default=False)
  -m MUSIC, --music MUSIC
                        music endpoint
  -k KEYSPACE, --keyspace KEYSPACE
                        music keyspace
  -r REPLICATION_FACTOR, --replication_factor REPLICATION_FACTOR
                        music replication factor

Example Invocation

Split across lines for readability:

# ostro-listener -x nova -t topic -s
                 -H $RABBITMQ_HOST
                 -u $RABBITMQ_USERNAME
                 -P $RABBITMQ_PASSWORD_FILE
                 -m $MUSIC_URL
                 -k $MUSIC_KEYSPACE
                 -r $MUSIC_REPLICATION_FACTOR

Note: This script has kept its original flexibility in that it may also be used to listen to other exchanges/topics.

Always use the nova exchange (-x nova) and topic exchange type (-t topic) when using in conjunction with Ostro and Music.

IMPORTANT: Always use topic exchanges for "listening on the wire" to OpenStack message traffic. Failure to do so could risk other RabbitMQ users (e.g., OpenStack services) missing important messages, and then you will be sad.

Password File

A sample password file can be found in $OSTRO_LISTENER_PATH/etc/ostro_listener/passwd.txt.

Copy this file to another location before editing, for example /etc/ostro-listener/passwd.

The password file must be protected. In particular, it must not be readable by group/other users. It is often set to root ownership.

Within the file, separate hosts/IPs and passwords with a single space. For example:

127.0.0.1 password
localhost password
myhost password
myhost.at.att.com password

Hosts/IPs will match based on the value of $RABBITMQ_HOST.

Using a Configuration File

A sample configuration file can be found in $OSTRO_LISTENER_PATH/etc/ostro_listener/ostro-listener.conf.txt:

[DEFAULT]
exchange = nova
exchange_type = topic
auto_delete = false
host = localhost
port = 5672
username = guest
passwdfile = /etc/ostro-listener/passwd
output_format = dict
store = true
music = http://127.0.0.1:8080/
keyspace = music
replication_factor = 1

Copy this file to another location before editing.

Configuration files may be referenced in one of two ways, through the --config-file option:

# ostro-listener --config-file $CONFIG_FILE

... or via an environment variable:

# export OSTRO_EVENT_LISTENER_CONFIG=$CONFIG_FILE
# ostro-listener

Running as an Ubuntu Service

A sample Ubuntu init.d script can be found in $OSTRO_LISTENER_PATH/etc/ostro_listener/ostro-listener.initd.txt.

To use, first copy this script to /etc/init.d/ostro-listener:

$ sudo cp $OSTRO_LISTENER_PATH/etc/ostro_listener/ostro-listener.initd.txt /etc/init.d/ostro-listener
$ sudo chmod 755 /etc/init.d/ostro-listener

If ostro-listener was installed in a Python virtual environment, edit /etc/init.d/ostro-listener, uncomment the export VENV line, and adjust as needed. For example, if the virtual environment is installed in /etc/ostro-listener/venv, assign VENV as follows:

export VENV=/etc/ostro-listener/venv

Create /var/log and /var/run directories for use by the service:

$ sudo mkdir /var/log/ostro-listener
$ sudo mkdir /var/run/ostro-listener
$ sudo chmod 750 /var/log/ostro-listener
$ sudo chmod 755 /var/run/ostro-listener

Set the run level defaults, then enable ostro-listener as a service:

$ sudo update-rc.d ostro-listener defaults
$ sudo update-rc.d ostro-listener enable

To start the ostro-listener service:

$ sudo service ostro-listener start

While running, a process ID file will be found in /var/run/ostro-listener/ostro-listener.pid, and a log file will be found in /var/log/ostro-listener/ostro-listener.log. The log is appended to upon subsequent starts. Log rotation is left to the discretion of the server administrator.

To stop the ostro-listener service:

$ sudo service ostro-listener stop

The process ID file will be removed from /var/run/ostro-listener upon stopping.

Uninstallation

Activate a virtual environment (venv) first if necessary.

Disable ostro-listener as a service, then uninstall:

$ sudo update-rc.d ostro-listener disable
$ sudo pip uninstall ostro-listener

Remove previously made configuration file changes, files, and other settings as needed.