replace 'funcionality' with 'functionality' replace 'wich' with 'which' Change-Id: Id79df09a6c85688b785eaaec0b2af17b0e29c55b
5.0 KiB
Configuring the Data Processing (sahara) service (optional)
Note
This feature is experimental at this time and it has not been fully production tested yet.
Sahara provide users with a simple means to provision data processing frameworks (such as Hadoop, Spark and Storm) on OpenStack.
Sahara is configured using the
/etc/openstack_deploy/conf.d/sahara.yml
file and the
/etc/openstack_deploy/user_variables.yml
file.
Configuring target hosts
Modify /etc/openstack_deploy/conf.d/sahara.yml
by adding
a list containing the infrastructure target hosts in the
sahara-infra_hosts section:
In sahara.yml
:
sahara-infra_hosts: infra01: ip: INFRA01_IP_ADDRESS infra02: ip: INFRA02_IP_ADDRESS infra03: ip: INFRA03_IP_ADDRESS
Replace *_IP_ADDRESS
with the IP address of the br-mgmt
container management bridge on each target host.
This hosts will be used to deploy the containers where sahara will be installed.
Configuring the cluster network
Sahara is configured to use the neutron implementation of OpenStack Networking.
Floating IP management
By default sahara is configured to use fixed IP addresses for access.
This is controlled by the sahara_use_floating_ips
variable.
By changing sahara_use_floating_ips
to True
the user may specify a floating IP address pool for each node group
directly.
In
user_variables.yml
:sahara_use_floating_ips: False
Warning
When using floating IP addresses for management every instance in the cluster must have a floating IP address, otherwise sahara will not be able to utilize that cluster.
When using fixed IP addresses
(sahara_use_floating_ips=False
) the user will be able to
choose the fixed IP network for all instances in a cluster. It is
important to ensure that all instances running sahara have access to the
fixed IP networks.
Object Storage access using proxy users
By default sahara is configured to use proxy users and delegated
trusts for Object Storage access. In that way, users are not required to
enter credentials for their data sources and job binaries referenced in
Object Storage. To disable this functionality change the following
variable to False
.
In
user_variables.yml
:sahara_use_domain_for_proxy_users: True
Also, is it possible to change which roles the trust users will
receive in the proxy domain, by default it receives the
_member_
role.
In
user_variables.yml
:sahara_proxy_user_role_names: _member_
Warning
In the context of the proxy user, any roles that are required for Object Storage access by the project owning the object store must be delegated to the proxy user for authentication to be successful.
Configuring cluster instances NTP
By default sahara will enable the NTP service on all cluster
instances if the NTP package is included in the image. The default NTP
server will be pool.ntp.org
this can be overridden using
the sahara_default_ntp_server
variable.
In
user_variables.yml
:sahara_default_ntp_server: "pool.ntp.org"
Configuring plugins
The following plugins are loaded by default:
sahara_plugin_base: - vanilla - spark - cdh - ambari
To add/remove plugins, just change the
sahara_plugin_base
variable accordingly, in the
user_variables.yml
file.
Configuring notifications
Sahara can be configured to send notifications to the OpenStack Telemetry module. By default, the variable is set to true if there are any Ceilometer hosts in the environment. To change this, the following variable must be set:
In
user_variables.yml
:sahara_ceilometer_enabled: True
Dashboard
To enable the Data Processing panel on horizon, the following variable should be set:
In
user_variables.yml
:horizon_enable_sahara_ui: True
Setting up Sahara
Run the setup-hosts playbook, to create the sahara containers, and the repo-build playbook to update the repository with the sahara packages.
# cd /opt/openstack-ansible/playbooks # openstack-ansible setup-hosts.yml # openstack-ansible repo-build.yml
Run the sahara and horizon playbooks to install sahara and enable the Data Processing panel in horizon:
# cd /opt/openstack-ansible/playbooks # openstack-ansible os-sahara-install.yml # openstack-ansible os-horizon-install.yml