Clean up docs and bootstrap script

This change updates the bootstrap script to be cleaner and easier to
use. This change updates the docs so they should now be a lot easier to
follow.

Change-Id: Ie2e5722dcaa229bdbec0e05bf994795f4cb10129
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>

elk_metrics_6x/ansible-role-requirements.yml

@@ -0,0 +1,9 @@
+---
+- name: systemd_service
+  scm: git
+  src: https://git.openstack.org/openstack/ansible-role-systemd_service
+  version: master
+- name: config_template
+  scm: git
+  src: https://git.openstack.org/openstack/ansible-config_template
+  version: master

elk_metrics_6x/bootstrap-embedded-ansible.sh

@@ -13,34 +13,59 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-if [[ ! -f "/opt/ansible25/bin/ansible" ]]; then
+export ANSIBLE_EMBED_HOME="${HOME}/ansible25"
+
+if [[ ! -e "${ANSIBLE_EMBED_HOME}/bin/ansible" ]]; then
   apt-get update
-  apt-get -y install python3-virtualenv || apt-get -y install python-virtualenv
+  apt-get -y install python3-virtualenv python-virtualenv
-  virtualenv --python=/usr/bin/python3 /opt/ansible25 || virtualenv --python=/usr/bin/python2 /opt/ansible25
+  if [[ -f "/usr/bin/python2" ]]; then
-  /opt/ansible25/bin/pip install --upgrade ansible==2.5.2.0 --isolated
+    virtualenv --python="/usr/bin/python2" "${ANSIBLE_EMBED_HOME}"
+  elif [[ -f "/usr/bin/python3" ]]; then
+    virtualenv --python="/usr/bin/python3" "${ANSIBLE_EMBED_HOME}"
+  else
+    virtualenv "${ANSIBLE_EMBED_HOME}"
+  fi
+  eval "${ANSIBLE_EMBED_HOME}/bin/pip install --upgrade --force pip"
+  eval "${ANSIBLE_EMBED_HOME}/bin/pip install --upgrade ansible==2.5.2.0 --isolated"
+  echo "Ansible can be found here: ${ANSIBLE_EMBED_HOME}/bin"
 fi
 
-if [[ ! -d "/opt/ansible25/repositories/ansible-config_template" ]]; then
+if [[ ! -d "${ANSIBLE_EMBED_HOME}/repositories/ansible-config_template" ]]; then
-  mkdir -p /opt/ansible25/repositories
+  mkdir -p "${ANSIBLE_EMBED_HOME}/repositories"
-  git clone https://github.com/openstack/ansible-config_template /opt/ansible25/repositories/ansible-config_template
+  git clone https://git.openstack.org/openstack/ansible-config_template "${ANSIBLE_EMBED_HOME}/repositories/ansible-config_template"
 fi
 
-if [[ ! -d "/opt/ansible25/repositories/roles/systemd_service" ]]; then
+if [[ ! -d "${ANSIBLE_EMBED_HOME}/repositories/roles/systemd_service" ]]; then
-  mkdir -p /opt/ansible25/repositories
+  mkdir -p "${ANSIBLE_EMBED_HOME}/repositories"
-  git clone https://github.com/openstack/ansible-role-systemd_service /opt/ansible25/repositories/roles/systemd_service
+  git clone https://git.openstack.org/openstack/ansible-role-systemd_service "${ANSIBLE_EMBED_HOME}/repositories/roles/systemd_service"
 fi
 
 if [[ -f "/etc/openstack_deploy/openstack_inventory.json" ]]; then
-  mkdir -p /opt/ansible25/inventory
+  if [[ ! -f "${ANSIBLE_EMBED_HOME}/inventory/openstack_inventory.sh" ]]; then
-  cat > /opt/ansible25/inventory/openstack_inventory.sh <<EOF
+    mkdir -p "${ANSIBLE_EMBED_HOME}/inventory"
+    cat > "${ANSIBLE_EMBED_HOME}/inventory/openstack_inventory.sh" <<EOF
 #!/usr/bin/env bash
 cat /etc/openstack_deploy/openstack_inventory.json
 EOF
-  chmod +x /opt/ansible25/inventory/openstack_inventory.sh
+    chmod +x "${ANSIBLE_EMBED_HOME}/inventory/openstack_inventory.sh"
   fi
 
-export ANSIBLE_ROLES_PATH="/opt/ansible25/repositories/roles"
+  export USER_VARS="$(for i in $(ls -1 /etc/openstack_deploy/user_*secret*.yml); do echo -n "-e@$i "; done)"
-export ANSIBLE_ACTION_PLUGINS="/opt/ansible25/repositories/ansible-config_template/action"
+  echo "env USER_VARS set"
-export ANSIBLE_INVENTORY="${ANSIBLE_INVENTORY:-/opt/ansible25/inventory/openstack_inventory.sh}"
+  echo "Extra users variables can be expanded by including the option \$USER_VARS on a playbook run."
 
-export ANSIBLE_USER_VARS="$(for i in $(ls /etc/openstack_deploy/user_*secret*.yml); do echo -n "-e@$i "; done)"
+  export ANSIBLE_INVENTORY="${ANSIBLE_EMBED_HOME}/inventory/openstack_inventory.sh"
+  echo "env ANSIBLE_INVENTORY set"
+fi
+
+export ANSIBLE_HOST_KEY_CHECKING="False"
+echo "env ANSIBLE_HOST_KEY_CHECKING set"
+
+export ANSIBLE_ROLES_PATH="${ANSIBLE_EMBED_HOME}/repositories/roles"
+echo "env ANSIBLE_ACTION_PLUGINS set"
+
+export ANSIBLE_ACTION_PLUGINS="${ANSIBLE_EMBED_HOME}/repositories/ansible-config_template/action"
+echo "env ANSIBLE_ROLES_PATH set"
+
+source ${ANSIBLE_EMBED_HOME}/bin/activate
+echo "Embedded Ansible has been activated. Run 'deactivate' to leave the embedded environment".

elk_metrics_6x/readme.rst

@@ -11,16 +11,6 @@ with topbeat to gather metrics from hosts metrics to the ELK cluster.
 
 **These playbooks require Ansible 2.5+.**
 
-Before running these playbooks the ``systemd_service`` role is required and is
-used in community roles. If these playbooks are being run in an
-OpenStack-Ansible installation the required role will be resolved for you. If
-the Installation is outside of OpenStack-Ansible, clone the role or add it to an
-ansible role requirements file.
-
-.. code-block:: bash
-
-    git clone https://github.com/openstack/ansible-role-systemd_service /etc/ansible/roles/systemd_service
-
 
 OpenStack-Ansible Integration
 -----------------------------
@@ -119,30 +109,40 @@ Create the containers
    openstack-ansible lxc-containers-create.yml -e 'container_group=elastic-logstash:kibana'
 
 
-Deployment | legacy environment
+Deploying | Installing with embedded Ansible
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-If these playbooks are to be run in an environment that does not have access to
+If this is being executed on a system that already has Ansible installed but is
-modern Ansible source the script ``bootstrap-embeded-ansible.sh`` before running
+incompatible with these playbooks the script `bootstrap-embedded-ansible.sh` can
-the playbooks. This script will install Ansible **2.5.2** in a virtual
+be sourced to grab an embedded version of Ansible prior to executing the
-environment within ``/opt``. This will provide for everything needed to run
+playbooks.
-these playbooks in an OpenStack-Ansible cloud without having to upgrade the
-Ansible version from within the legacy environment. When it comes time to
-execute these playbooks substite the ``openstack-ansible`` command with the
-full path to ``ansible-playbook`` within the embeded ansible virtual
-environment making sure to include the available user provided variables.
 
-Example commands to deploy all of these playbooks using the embeded ansible.
+``` bash
-
+source bootstrap-embedded-ansible.sh
-.. code-block:: bash
+```
-
-    cd /opt/openstack-ansible-ops/elk_metrics_6x
-    source bootstrap-embeded-ansible.sh
-    /opt/ansible25/bin/ansible-playbook ${ANSIBLE_USER_VARS} site.yml
 
 
-Deploying | modern environment
+Deploying | Manually resolving the dependencies
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This playbook has external role dependencies. If Ansible is not installed with
+the `bootstrap-ansible.sh` script these dependencies can be resolved with the
+``ansible-galaxy`` command and the ``ansible-role-requirements.yml`` file.
+
+* Example galaxy execution
+
+``` bash
+ansible-galaxy install -r ansible-role-requirements.yml
+```
+
+Once the dependencies are set make sure to set the action plugin path to the
+location of the config_template action directory. This can be done using the
+environment variable `ANSIBLE_ACTION_PLUGINS` or through the use of an
+`ansible.cfg` file.
+
+
+Deploying | The environment
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Install master/data elasticsearch nodes on the elastic-logstash containers,
 deploy logstash, deploy kibana, and then deploy all of the service beats.
@@ -150,7 +150,15 @@ deploy logstash, deploy kibana, and then deploy all of the service beats.
 .. code-block:: bash
 
     cd /opt/openstack-ansible-ops/elk_metrics_6x
-    openstack-ansible site.yml
+    ansible-playbook site.yml
+
+
+* The `openstack-ansible` command can be used if the version of ansible on the
+  system is greater than **2.5**.
+
+
+The individual playbooks found within this repository can be independently run
+at anytime.
 
 
 Optional | add Grafana visualizations