OpenStack Event Listener for Ostro ================================== This script listens for specific messages needed by Ostro to maintain up-to-date cloud state. It can persist digested versions of the messages to Music, which Ostro then picks up. *Note: This version of the listener does not use oslo.messaging. It listens directly to RabbitMQ. Future revisions are expected to use oslo.messaging in order to keep the means of transport abstract.* Prerequisites ------------- Prior to installation: - Ubuntu 14.04 LTS - Python 2.7.6 with pip - Access to an OpenStack Kilo cloud (RabbitMQ in particular) - Access to Music and Ostro Throughout this document, the following installation-specific items are required. Have values for these prepared and ready before continuing. Suggestions for values are provided in this document where applicable. | Name | Description | Example | |------|-------------|---------| | `$USER` | User id | `user1234` | | `$VENV` | Python virtual environment path (if any) | `/etc/ostro-listener/venv` | | `$OSTRO_LISTENER_PATH` | Local git repository's `ostro_listener` directory | `/home/user1234/git/allegro/ostro_listener` | | `$CONFIG_FILE` | Event Listener configuration file | `/etc/ostro-listener/ostro-listener.conf` | | `$RABBITMQ_HOST` | RabbitMQ hostname or IP address | `localhost` | | `$RABBITMQ_USERNAME` | RabbitMQ username | `guest` | | `$RABBITMQ_PASSWORD_FILE` | Full path to RabbitMQ password file | `/etc/ostro-listener/passwd` | | `$MUSIC_URL` | Music API endpoints and port in URL format | `http://127.0.0.1:8080/` | | `$MUSIC_KEYSPACE` | Music keyspace | `valet` | | `$MUSIC_REPLICATION_FACTOR` | Music replication factor | `1` | Root or sufficient sudo privileges are required for some steps. ### A Note About Python Virtual Environments It is recommended to consider using a python [virtuals environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/) (venv). A venv helps avoid instabilities and conflicts within the default python environment. Installation ------------ Install ostro-listener on any node with line-of-sight to the RabbitMQ endpoint from which messages are to be monitored. ostro-listener is located in `ostro_listener`. ~~~~ {.bash} $ cd $OSTRO_LISTENER_PATH $ sudo pip install . ~~~~ Command Line Usage ------------------ usage: ostro-listener [-h] [-c OSTRO_LISTENER_CONFIG] [-x EXCHANGE] [-t {topic,fanout}] [-a] [-H HOST] [-p PORT] [-u USERNAME] [-P PASSWDFILE] [-o {yaml,json,dict}] [-s] [-m MUSIC] [-k KEYSPACE] [-r REPLICATION_FACTOR] Ostro-specific OpenStack Event Listener optional arguments: -h, --help show this help message and exit -c OSTRO_LISTENER_CONFIG, --conf_file OSTRO_LISTENER_CONFIG Defaults to env[OSTRO_LISTENER_CONFIG] -x EXCHANGE, --exchange EXCHANGE rabbit exchange to listen to -t {topic,fanout}, --exchange_type {topic,fanout} type of exchange (default="topic") -a, --auto_delete autodelete exchange (default=False) -H HOST, --host HOST compute node on which rabbitmq is running -p PORT, --port PORT port on which rabbitmq is running -u USERNAME, --username USERNAME rabbitmq username (default="guest") -P PASSWDFILE, --passwdfile PASSWDFILE file containing host rabbitmq passwords -o {yaml,json,dict}, --output_format {yaml,json,dict} output format (default="dict") -s, --store store messages in music (default=False) -m MUSIC, --music MUSIC music endpoint -k KEYSPACE, --keyspace KEYSPACE music keyspace -r REPLICATION_FACTOR, --replication_factor REPLICATION_FACTOR music replication factor Example Invocation ------------------ Split across lines for readability: # ostro-listener -x nova -t topic -s -H $RABBITMQ_HOST -u $RABBITMQ_USERNAME -P $RABBITMQ_PASSWORD_FILE -m $MUSIC_URL -k $MUSIC_KEYSPACE -r $MUSIC_REPLICATION_FACTOR *Note: This script has kept its original flexibility in that it may also be used to listen to other exchanges/topics.* Always use the nova exchange (`-x nova`) and topic exchange type (`-t topic`) when using in conjunction with Ostro and Music. **IMPORTANT**: Always use topic exchanges for "listening on the wire" to OpenStack message traffic. Failure to do so could risk other RabbitMQ users (e.g., OpenStack services) missing important messages, and then you will be sad. Password File ------------- A sample password file can be found in `$OSTRO_LISTENER_PATH/etc/ostro_listener/passwd.txt`. Copy this file to another location before editing, for example `/etc/ostro-listener/passwd`. The password file must be protected. In particular, it must not be readable by group/other users. It is often set to root ownership. Within the file, separate hosts/IPs and passwords with a single space. For example: 127.0.0.1 password localhost password myhost password myhost.at.att.com password Hosts/IPs will match based on the value of `$RABBITMQ_HOST`. Using a Configuration File -------------------------- A sample configuration file can be found in `$OSTRO_LISTENER_PATH/etc/ostro_listener/ostro-listener.conf.txt`: ~~~~ {.ini} [DEFAULT] exchange = nova exchange_type = topic auto_delete = false host = localhost port = 5672 username = guest passwdfile = /etc/ostro-listener/passwd output_format = dict store = true music = http://127.0.0.1:8080/ keyspace = music replication_factor = 1 ~~~~ Copy this file to another location before editing. Configuration files may be referenced in one of two ways, through the `--config-file` option: ~~~~ {.bash} # ostro-listener --config-file $CONFIG_FILE ~~~~ ... or via an environment variable: ~~~~ {.bash} # export OSTRO_EVENT_LISTENER_CONFIG=$CONFIG_FILE # ostro-listener ~~~~ Running as an Ubuntu Service ---------------------------- A sample Ubuntu init.d script can be found in `$OSTRO_LISTENER_PATH/etc/ostro_listener/ostro-listener.initd.txt`. To use, first copy this script to `/etc/init.d/ostro-listener`: ~~~~ {.bash} $ sudo cp $OSTRO_LISTENER_PATH/etc/ostro_listener/ostro-listener.initd.txt /etc/init.d/ostro-listener $ sudo chmod 755 /etc/init.d/ostro-listener ~~~~ If ostro-listener was installed in a Python virtual environment, edit `/etc/init.d/ostro-listener`, uncomment the `export VENV` line, and adjust as needed. For example, if the virtual environment is installed in `/etc/ostro-listener/venv`, assign `VENV` as follows: ~~~~ {.bash} export VENV=/etc/ostro-listener/venv ~~~~ Create `/var/log` and `/var/run` directories for use by the service: ~~~~ {.bash} $ sudo mkdir /var/log/ostro-listener $ sudo mkdir /var/run/ostro-listener $ sudo chmod 750 /var/log/ostro-listener $ sudo chmod 755 /var/run/ostro-listener ~~~~ Set the run level defaults, then enable ostro-listener as a service: ~~~~ {.bash} $ sudo update-rc.d ostro-listener defaults $ sudo update-rc.d ostro-listener enable ~~~~ To start the ostro-listener service: ~~~~ {.bash} $ sudo service ostro-listener start ~~~~ While running, a process ID file will be found in `/var/run/ostro-listener/ostro-listener.pid`, and a log file will be found in `/var/log/ostro-listener/ostro-listener.log`. The log is appended to upon subsequent starts. Log rotation is left to the discretion of the server administrator. To stop the ostro-listener service: ~~~~ {.bash} $ sudo service ostro-listener stop ~~~~ The process ID file will be removed from `/var/run/ostro-listener` upon stopping. Uninstallation -------------- Activate a virtual environment (venv) first if necessary. Disable ostro-listener as a service, then uninstall: ~~~~ {.bash} $ sudo update-rc.d ostro-listener disable $ sudo pip uninstall ostro-listener ~~~~ Remove previously made configuration file changes, files, and other settings as needed.