diff --git a/README.rst b/README.rst index 71f6b69..9ef30ae 100644 --- a/README.rst +++ b/README.rst @@ -1,40 +1,11 @@ -OpenStack-Ansible Sahara -######################## -:tags: openstack, sahara, cloud, ansible -:category: \*nix +======================== +OpenStack-Ansible sahara +======================== -This Ansible role installs and configures OpenStack sahara and sahara -engine. +This Ansible role installs and configures OpenStack sahara. -This role will install the following Upstart services: - * sahara-api - * sahara-engine - -Required Variables -================== - -.. code-block:: yaml - - sahara_galera_address - sahara_container_mysql_password - sahara_service_password - sahara_rabbitmq_password - -Example Playbook -================ - -.. code-block:: yaml - - - name: Install sahara server - hosts: sahara_all - user: root - roles: - - { role: "os_sahara", tags: [ "os-sahara" ] } - vars: - external_lb_vip_address: 172.16.24.1 - internal_lb_vip_address: 192.168.0.1 - sahara_galera_address: "{{ internal_lb_vip_address }}" - sahara_container_mysql_password: "SuperSecretePassword1" - sahara_service_password: "SuperSecretePassword2" - sahara_rabbitmq_password: "SuperSecretePassword3" +Documentation for the project can be found at: + http://docs.openstack.org/developer/openstack-ansible-os_sahara/ +The project home is at: + http://launchpad.net/openstack-ansible diff --git a/doc/source/configure-sahara.rst b/doc/source/configure-sahara.rst new file mode 100644 index 0000000..4062834 --- /dev/null +++ b/doc/source/configure-sahara.rst @@ -0,0 +1,181 @@ +=========================================================== +Configuring the Data Processing (sahara) service (optional) +=========================================================== + +.. note:: + + This feature is experimental at this time and it has not been fully + production tested yet. + +Sahara provide users with a simple means to provision data processing +frameworks (such as Hadoop, Spark and Storm) on OpenStack. + +Sahara is configured using the ``/etc/openstack_deploy/conf.d/sahara.yml`` +file and the ``/etc/openstack_deploy/user_variables.yml`` file. + +Configuring target hosts +~~~~~~~~~~~~~~~~~~~~~~~~ + +Modify ``/etc/openstack_deploy/conf.d/sahara.yml`` by adding a list +containing the infrastructure target hosts in the sahara-infra_hosts +section: + +In ``sahara.yml``: + + .. code-block:: yaml + + sahara-infra_hosts: + infra01: + ip: INFRA01_IP_ADDRESS + infra02: + ip: INFRA02_IP_ADDRESS + infra03: + ip: INFRA03_IP_ADDRESS + +Replace ``*_IP_ADDRESS`` with the IP address of the br-mgmt container +management bridge on each target host. + +This hosts will be used to deploy the containers where sahara will be +installed. + +Configuring the cluster network +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Sahara is configured to use the neutron implementation of OpenStack +Networking. + +Floating IP management +---------------------- + +By default sahara is configured to use fixed IP addresses for access. This +is controlled by the ``sahara_use_floating_ips`` variable. By changing +``sahara_use_floating_ips`` to ``True`` the user may specify a floating IP +address pool for each node group directly. + + In ``user_variables.yml``: + + .. code-block:: yaml + + sahara_use_floating_ips: False + +.. warning:: + When using floating IP addresses for management **every** instance in + the cluster must have a floating IP address, otherwise sahara will not + be able to utilize that cluster. + +When using fixed IP addresses (``sahara_use_floating_ips=False``) the user +will be able to choose the fixed IP network for all instances in a cluster. +It is important to ensure that all instances running sahara have access to +the fixed IP networks. + + +Object Storage access using proxy users +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By default sahara is configured to use proxy users and delegated trusts +for Object Storage access. In that way, users are not required to enter +credentials for their data sources and job binaries referenced in Object +Storage. To disable this funcionality change the following variable to +``False``. + + In ``user_variables.yml``: + + .. code-block:: yaml + + sahara_use_domain_for_proxy_users: True + +Also, is it possible to change wich roles the trust users will receive +in the proxy domain, by default it receives the ``_member_`` role. + + In ``user_variables.yml``: + + .. code-block:: yaml + + sahara_proxy_user_role_names: _member_ + +.. warning:: + In the context of the proxy user, any roles that are required for + Object Storage access by the project owning the object store must + be delegated to the proxy user for authentication to be successful. + + +Configuring cluster instances NTP +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By default sahara will enable the NTP service on all cluster instances if +the NTP package is included in the image. The default NTP server will be +``pool.ntp.org`` this can be overridden using the +``sahara_default_ntp_server`` variable. + + In ``user_variables.yml``: + + .. code-block:: yaml + + sahara_default_ntp_server: "pool.ntp.org" + + +Configuring plugins +~~~~~~~~~~~~~~~~~~~ + +The following plugins are loaded by default: + + .. code-block:: yaml + + sahara_plugin_base: + - vanilla + - spark + - cdh + - ambari + +To add/remove plugins, just change the ``sahara_plugin_base`` variable +accordingly, in the ``user_variables.yml`` file. + + +Configuring notifications +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Sahara can be configured to send notifications to the OpenStack Telemetry +module. By default, the variable is set to true if there are any Ceilometer +hosts in the environment. To change this, the following variable must be +set: + + In ``user_variables.yml``: + + .. code-block:: yaml + + sahara_ceilometer_enabled: True + + +Dashboard +~~~~~~~~~ + +To enable the Data Processing panel on horizon, the following variable +should be set: + + In ``user_variables.yml``: + + .. code-block:: yaml + + horizon_enable_sahara_ui: True + + +Setting up Sahara +~~~~~~~~~~~~~~~~~ + +Run the setup-hosts playbook, to create the sahara containers, and the +repo-build playbook to update the repository with the sahara packages. + + .. code-block:: console + + # cd /opt/openstack-ansible/playbooks + # openstack-ansible setup-hosts.yml + # openstack-ansible repo-build.yml + +Run the sahara and horizon playbooks to install sahara and enable the +Data Processing panel in horizon: + + .. code-block:: console + + # cd /opt/openstack-ansible/playbooks + # openstack-ansible os-sahara-install.yml + # openstack-ansible os-horizon-install.yml diff --git a/doc/source/index.rst b/doc/source/index.rst index a6210d3..fe42d62 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -1 +1,43 @@ -.. include:: ../../README.rst +============================= +OpenStack-Ansible sahara role +============================= + +.. toctree:: + :maxdepth: 2 + + configure-sahara.rst + +This role installs the following Upstart services: + + * sahara-api + * sahara-engine + +Default variables +~~~~~~~~~~~~~~~~~ + +.. literalinclude:: ../../defaults/main.yml + :language: yaml + :start-after: under the License. + +Required variables +~~~~~~~~~~~~~~~~~~ + +.. code-block:: yaml + + sahara_galera_address + sahara_container_mysql_password + sahara_service_password + sahara_rabbitmq_password + +Example playbook +~~~~~~~~~~~~~~~~ + +.. literalinclude:: ../../examples/playbook.yml + :language: yaml + +Tags +~~~~ + +This role supports two tags: ``sahara-install`` and ``sahara-config``. +The ``sahara-install`` tag can be used to install and upgrade. The +``sahara-config`` tag can be used to manage configuration. diff --git a/examples/playbook.yml b/examples/playbook.yml new file mode 100644 index 0000000..a23a85f --- /dev/null +++ b/examples/playbook.yml @@ -0,0 +1,12 @@ +- name: Install sahara server + hosts: sahara_all + user: root + roles: + - { role: "os_sahara", tags: [ "os-sahara" ] } + vars: + external_lb_vip_address: 172.16.24.1 + internal_lb_vip_address: 192.168.0.1 + sahara_galera_address: "{{ internal_lb_vip_address }}" + sahara_container_mysql_password: "SuperSecretePassword1" + sahara_service_password: "SuperSecretePassword2" + sahara_rabbitmq_password: "SuperSecretePassword3"