Introduce documentation related to new worker

related to blueprint stevedorize-orchestration
Closes-Bug: 1538536

Change-Id: I097a68f5acd6f88fa58b44898e65de21523f3132

doc/source/index.rst

@@ -18,7 +18,7 @@ Contents:
    architecture
    resource
    resource_repository
-   orchestration
+   orchestration/index
    transports
    examples
    deployment_plan

doc/source/orchestration/configuration.rst

@@ -0,0 +1,114 @@
+.. _orchestartion_configuration:
+
+Configuration
+=============
+
+Orchestration is configured using two different methods.
+
+1. :ref:`orchestration_config`
+2. :ref:`orchestration_entrypoints`
+
+.. _orchestration_config:
+
+Config options
+--------------
+
+system_log_address
+^^^^^^^^^^^^^^^^^^
+Passed to executor which will run system log worker
+
+tasks_address
+^^^^^^^^^^^^^
+Passed to executor which will run tasks worker
+
+scheduler_address
+^^^^^^^^^^^^^^^^^
+Passed to executor which will run scheduler worker
+
+executor
+^^^^^^^^
+Driver name should be registered in entrypoints, see :ref:`namespace_executors`
+
+tasks_driver
+^^^^^^^^^^^^
+Driver name should be registered in appropriate entrypoints (see :ref:`namespace_workers`)
+
+scheduler_driver
+^^^^^^^^^^^^^^^^
+Driver name should be registered in appropriate entrypoints (see :ref:`namespace_workers`)
+
+system_log_driver
+^^^^^^^^^^^^^^^^^
+Driver name should be registered in appropriate entrypoints (see :ref:`namespace_workers`)
+
+runner
+^^^^^^
+Driver name should be registered in entrypoints (see :ref:`namespace_runner`)
+
+.. _orchestration_entrypoints:
+
+Entrypoints
+-----------
+
+.. _namespace_executors:
+
+Executor namespace
+^^^^^^^^^^^^^^^^^^
+.. note::
+    solar.orchestration.executors
+
+One specified in configuration will be used.
+
+.. _namespace_extensions:
+
+Extensions namespace
+^^^^^^^^^^^^^^^^^^^^
+.. note::
+    solar.orchestration.extensions
+
+Using driver namespaces for each worker - loads all workers.
+
+.. _namespace_workers:
+
+Worker driver namespaces
+^^^^^^^^^^^^^^^^^^^^^^^^
+.. note::
+    | solar.orchestration.drivers.tasks
+    | solar.orchestration.drivers.scheduler
+    | solar.orchestration.drivers.system_log
+
+Only one driver can be selected from each namespace, see driver options
+in config.
+
+.. _namespace_constructor:
+
+Constructor namespace
+^^^^^^^^^^^^^^^^^^^^^
+.. note::
+    solar.orchestration.constructors
+
+Loads callables from this namespace and executes hooks connected
+to those namespaces.
+
+.. _namespace_contructor_hooks:
+
+Constructor hooks namespaces
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+.. note::
+    | solar.orchestration.hooks.tasks.construct
+    | solar.orchestration.hooks.system_log.construct
+    | solar.orchestration.hooks.scheduler.construct
+
+All callables in each hook will be loaded and executed before spawning
+executor with instance of worker. Currently all subscriptions are done
+in this hooks.
+
+.. _namespace_runner:
+
+Runner namespace
+^^^^^^^^^^^^^^^^
+.. note::
+    solar.orchestration.runners
+
+Runner should be selected in solar config. Runner will be executed
+as a last step in solar-worker main function.

doc/source/orchestration/daemon.rst

@@ -0,0 +1,23 @@
+.. _orchestration_daemon:
+
+Daemonizing solar-worker
+========================
+
+.. _orchestration_daemon_upstart:
+
+Upstart
+-------
+
+To daemonize solar-worker on debian or ubuntu `upstart script`_ should be used,
+in script and pre-script stanzas - */etc/default/solar-worker* will be sourced, and following variables used::
+
+    SOLAR_UID=solar
+    SOLAR_GID=solar
+    SOLAR_PIDFILE=/var/opt/solar/solar-worker.pid
+
+.. warning::
+    SOLAR_UID and SOLAR_GID should be present in the system.
+
+.. _upstart script: https://github.com/openstack/solar/blob/master/utils/solar-worker.conf
+
+

doc/source/orchestration/entities.rst

@@ -0,0 +1,58 @@
+.. _orchestration_entities:
+
+Entities
+========
+
+Worker
+------
+
+Worker encapsulates logic related to certain area in solar.
+Current we have next workers:
+
+Scheduler
+  Correctly initates execution plans and updates state of tasks.
+Tasks
+  Execute tasks scheduled by Scheduler worker
+System log
+  Updates system log e.g. commits and moves log item from staged log
+  to history, or in case of error updates log item as erred
+
+Executors
+---------
+
+Each executor module should provide:
+
+Executor
+  Executor responsible for processing events and handle them via given
+  worker. Concurrency policies is up to the executor implementation.
+Client
+  Client communicates with executor
+
+In current version of Solar we are using executor based on Push/Pull
+zeromq sockets, and gevent pool for concurrent processing of events.
+
+Subscriptions
+-------------
+
+Each public method of worker is subscribable, in current version
+4 events are available to subscribers.
+
+on_sucess
+  Called in the case of successful execution, provides context, result
+  and event arguments
+on_error
+  Called in the case of error, prorives context, error type, event
+  arguments
+before
+  Called before method execution, provides only context
+after
+  Called after method executuon, provides only context
+
+To subscribe use::
+
+    worker.method.on_sucess(callable)
+
+Additionally each worker provides *for_all* descriptor which allows
+to subscribe to all public methods::
+
+    worker.for_all.before(callable)

doc/source/orchestration/index.rst

@@ -0,0 +1,14 @@
+.. _orchestration:
+
+Orchestration
+=============
+
+Contents:
+
+.. toctree::
+   :maxdepth: 1
+
+   entities
+   configuration
+   daemon
+   workflow

doc/source/orchestration/workflow.rst

@@ -1,4 +1,4 @@
-.. _orchestration:
+.. _orchestration_workflow:
 
 Deployment operations
 =====================