setup and arch diagram

docs/index.rst

@@ -9,10 +9,10 @@ Welcome to StackTach's documentation!
 Contents:
 
 .. toctree::
-   :maxdepth: 2
+   :maxdepth: 3
 
-    intro
-    setup
+   intro
+   setup
 
 
 Indices and tables

docs/intro.rst

@@ -2,11 +2,11 @@
 An Introduction to StackTach
 ============================
 
-StackTach was initially created as a browser-based debugging tool 
-for OpenStack Nova. Since that time, StackTach has evolved into a 
+StackTach was initially created as a browser-based debugging tool
+for OpenStack Nova. Since that time, StackTach has evolved into a
 tool that can do debugging, performance monitoring and perform
 audit, validation and reconcilation of Nova and Glance usage in a
-manner suitable for billing. 
+manner suitable for billing.
 
 
 How it works
@@ -15,7 +15,7 @@ How it works
 Nearly all OpenStack components are capable of generating
 *notifications* when significant events occur. Notifications
 are messages placed on the OpenStack queue (generally RabbitMQ)
-for consumption by downstream systems.  
+for consumption by downstream systems.
 
 The OpenStack wiki has info on the `notification format`_.
 
@@ -25,12 +25,12 @@ StackTach has a *worker* that is configured to read these notifications
 and store them in a database (ideally a database separate from the
 OpenStack production database). From there, StackTach reviews the stream
 of notifications to glean usage information and assemble it in an
-easy-to-query fashion. 
+easy-to-query fashion.
 
 Users can inquire on instances, requests, servers, etc using the
-browser interface or command line tool (`Stacky`_). 
+browser interface or command line tool (`Stacky`_).
 
 .. _Stacky: https://github.com/rackerlabs/stacky
 
-
+.. image:: images/diagram.gif
 

docs/setup.rst

@@ -0,0 +1,90 @@
+
+Installing StackTach
+####################
+
+The "Hurry Up" Install Guide
+****************************
+#. Create a database for StackTach to use. By default, StackTach assumes MySql, but you can modify the settings.py file to others.
+#. Install django and the other required libraries listed in ``./etc/pip-requires.txt`` (please let us know if any are missing)
+#. Clone this repo
+#. Copy and configure the config files in ``./etc`` (see below for details)
+#. Create the necessary database tables (python manage.py syncdb) You don't need an administrator account since there are no user profiles used.
+#. Configure OpenStack to publish Notifications back into RabbitMQ (see below)
+#. Restart the OpenStack services.
+#. Run the Worker to start consuming messages. (see below)
+#. Run the web server (``python manage.py runserver``)
+#. Point your browser to ``http://127.0.0.1:8000`` (the default server location)
+#. Click on stuff, see what happens. You can't hurt anything, it's all read-only.
+
+Of course, this is only suitable for playing around. If you want to get serious about deploying StackTach you should set up a proper webserver and database on standalone servers. There is a lot of data that gets collected by StackTach (depending on your deployment size) ... be warned. Keep an eye on DB size.
+
+The Config Files
+****************
+There are two config files for StackTach. The first one tells us where the second one is. A sample of these two files is in ``./etc/sample_*``. Create a local copy of these files and populate them with the appropriate config values as described below.
+
+The ``sample_stacktach_config.sh`` shell script defines the necessary environment variables StackTach needs. Most of these are just information about the database (assuming MySql) but some are a little different. Copy this file and modify it for your environment. ``source`` this
+``stacktach_config.sh`` shell script to set up the necessary environment variables.
+
+``STACKTACH_INSTALL_DIR`` should point to where StackTach is running out of. In most cases this will be your repo directory, but it could be elsewhere if your going for a proper deployment.
+The StackTach worker needs to know which RabbitMQ servers to listen to. This information is stored in the deployment file. ``STACKTACH_DEPLOYMENTS_FILE`` should point to this json file. To learn more about the deployments file, see further down.
+
+Finally, ``DJANGO_SETTINGS_MODULE`` tells Django where to get its configuration from. This should point to the ``setting.py`` file. You shouldn't have to do much with the ``settings.py`` file and most of what it needs is in these environment variables.
+
+The ``sample_stacktach_worker_config.json`` file tells StackTach where each of the RabbitMQ servers are that it needs to get events from. In most cases you'll only have one entry in this file, but for large multi-cell deployments, this file can get pretty large. It's also handy for setting up one StackTach for each developer environment.
+
+The file is in json format and the main configuration is under the ``deployments`` key, which should contain a list of deployment dictionaries.
+
+A blank worker config file would look like this: ::
+
+    {"deployments": [] }
+
+But that's not much fun. A deployment entry would look like this: ::
+
+    {"deployments": [
+         {
+             "name": "east_coast.prod.cell1",
+             "durable_queue": false,
+             "rabbit_host": "10.0.1.1",
+             "rabbit_port": 5672,
+             "rabbit_userid": "rabbit",
+             "rabbit_password": "rabbit",
+             "rabbit_virtual_host": "/"
+         }
+    ]}
+
+where, *name* is whatever you want to call your deployment, and *rabbit_\** are the connectivity details for your rabbit server. It should be the same information in your `nova.conf` file that OpenStack is using. Note, json has no concept of comments, so using ``#``, ``//`` or ``/* */`` as a comment won't work.
+
+By default, Nova uses ephemeral queues. If you are using durable queues, be sure to change the necessary flag here.
+
+You can add as many deployments as you like.
+
+Starting the Worker
+===================
+
+Note: the worker now uses librabbitmq, be sure to install that first.
+
+``./worker/start_workers.py`` will spawn a worker.py process for each deployment defined. Each worker will consume from a single Rabbit queue.
+
+
+Configuring Nova to Generate Notifications
+==========================================
+
+In the OpenStack service you wish to have generate notifications, add the
+following to its ``.conf`` file: ::
+
+    --notification_driver=nova.openstack.common.notifier.rabbit_notifier
+    --notification_topics=monitor
+
+**Note:** *This will likely change once the various project switch to ``oslo.messaging``
+which uses endpoints to define the notification drivers.*
+
+This will tell OpenStack to publish notifications to a Rabbit exchange starting with
+``monitor.*`` ... this may result in ``monitor.info``, ``monitor.error``, etc.
+
+You'll need to restart Nova once these changes are made.
+
+Next Steps
+==========
+
+Once you have this working well, you should download and install ``Stacky`` and play with the command line tool.
+