Support storage of introspection data in Nginx

As an operator I want to be able to persist raw and processed
introspection data so that I am able to view it at a later date.

As an operator I want to be able to persist raw and processed
introspection data so that I am able to reprocess the data after the
initial inspection process has completed.

In the absence of swift, we can use the bifrost nginx web server -
masquerading as an object store - to store raw and processed
introspection data for nodes.

This allows introspection data to be retrieved and reprocessed after the
initial inspection has completed. This can be useful when the processing
pipeline or introspection rules are changed.

Change-Id: Ia2bd16080594e854054f380d4f7670eaea98e82b
Closes-Bug: #1685879

playbooks/roles/bifrost-ironic-install/README.md

@@ -228,6 +228,14 @@ inspector_processing_hooks: String value containing a comma-separated list,
                             non-default list of comma-separated processing
                             hooks for inspector.
 
+inspector_store_data_in_nginx: Boolean value, default true. If true, this
+                               enables data gathered during introspection to be
+                               stored in the local Nginx web server. In this
+                               mode, Nginx masquerades as an unauthenticated
+                               'Swift' object storage service. Nginx is
+                               configured to only allow the required operations
+                               on the 'ironic-inspector' object container.
+
 ### Virtual Environment Install
 
 Bifrost can install ironic into a python virtual environment using the

playbooks/roles/bifrost-ironic-install/defaults/main.yml

@@ -165,6 +165,14 @@ inspector_keep_ports: "present"
 # list of processing hooks for inspector.
 #inspector_processing_hooks:
 
+# Whether to store introspection data using the local Nginx web server as an
+# object storage service.
+inspector_store_data_in_nginx: true
+
+# When inspector_store_data_in_nginx is true, this is the URL of the Nginx
+# 'Swift' API endpoint.
+inspector_store_data_url: "http://localhost:{{ file_url_port }}"
+
 # Inspector defaults
 inspector:
   discovery:

playbooks/roles/bifrost-ironic-install/tasks/bootstrap.yml

@@ -294,6 +294,15 @@
      testing | bool == true
 - name: "Deploy nginx configuration file for serving HTTP requests"
   template: src=nginx.conf.j2 dest=/etc/nginx/nginx.conf
+- name: "Ensure inspector object storage directory exists"
+  file:
+    path: "{{ http_boot_folder }}/ironic-inspector"
+    state: directory
+    owner: "{{ nginx_user }}"
+    group: "{{ nginx_user }}"
+  when:
+    - enable_inspector | bool
+    - inspector_store_data_in_nginx | bool
 - name: "Download Ironic Python Agent kernel & image"
   include: download_ipa_image.yml
   when: create_ipa_image | bool == false and download_ipa | bool == true
@@ -329,6 +338,15 @@
         setype: httpd_sys_content_t
         state: present
 
+    - name: "Add proper context on inspector data store"
+      sefcontext:
+        target: "{{ http_boot_folder }}/ironic-inspector(/.*)?"
+        setype: httpd_sys_rw_content_t
+        state: present
+      when:
+        - enable_inspector | bool
+        - inspector_store_data_in_nginx | bool
+
     - name: Copy ironic policy file to temporary directory
       copy:
         src: ironic_policy.te

playbooks/roles/bifrost-ironic-install/templates/ironic-inspector.conf.j2

@@ -63,9 +63,19 @@ always_store_ramdisk_logs = {{ inspector_store_ramdisk_logs | default('true') |
 {% if inspector_processing_hooks is defined %}
 processing_hooks = {{ inspector_processing_hooks }}
 {% endif %}
+{% if inspector_store_data_in_nginx | bool %}
+store_data = swift
+{% endif %}
 {% if inspector.discovery.enabled == true %}
 node_not_found_hook = enroll
 
 [discovery]
 enroll_node_driver = {{ inspector.discovery.default_node_driver }}
 {% endif %}
+
+{% if inspector_store_data_in_nginx | bool %}
+[swift]
+# Use the local nginx web server as a Swift-list object storage service.
+auth_type = none
+endpoint = {{ inspector_store_data_url }}
+{% endif %}

playbooks/roles/bifrost-ironic-install/templates/nginx.conf.j2

@@ -46,6 +46,14 @@ http {
 		location {{ http_boot_folder }}/ {
 			alias {{ http_boot_folder }}/;
 		}
+{% if inspector_store_data_in_nginx | bool %}
+                location /ironic-inspector {
+                        return 200 "";
+                }
+                location /ironic-inspector/ {
+                        dav_methods  PUT DELETE;
+                }
+{% endif %}
 	}
         include /etc/nginx/conf.d/bifrost*.conf;
 }

playbooks/roles/bifrost-test-inspection/README.md

@@ -0,0 +1,53 @@
+bifrost-test-inspection
+=======================
+
+Tests nodes that have been inspected by ironic inspector.
+
+Requirements
+------------
+
+None at this time.  See Dependencies.
+
+Role Variables
+--------------
+
+None at this time.  See Dependencies.
+
+Dependencies
+------------
+
+This role is intended to be executed as part of bifrost, after the
+ironic-inspect-node role, as part of the test sequence.
+
+Example Playbook
+----------------
+
+    hosts: baremetal
+    name: "Tests inspection of baremetal nodes"
+    connection: local
+    become: no
+    gather_facts: no
+    roles:
+      - role: bifrost-test-inspection
+
+License
+-------
+
+Copyright (c) 2018 StackHPC Ltd.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Author Information
+------------------
+
+Ironic Developers

playbooks/roles/bifrost-test-inspection/defaults/main.yml

@@ -0,0 +1,12 @@
+---
+# defaults file for bifrost-test-inspection
+
+file_url_port: "8080"
+
+# Whether to store introspection data using the local Nginx web server as an
+# object storage service.
+inspector_store_data_in_nginx: true
+
+# When inspector_store_data_in_nginx is true, this is the URL of the Nginx
+# 'Swift' API endpoint.
+inspector_store_data_url: "http://localhost:{{ file_url_port }}"

playbooks/roles/bifrost-test-inspection/handlers/main.yml

@@ -0,0 +1,2 @@
+---
+# handlers file for bifrost-test-inspection

playbooks/roles/bifrost-test-inspection/meta/main.yml

@@ -0,0 +1,116 @@
+---
+galaxy_info:
+  author: Ironic Developers
+  description: Tests inspection of nodes created by Bifrost.
+  company: OpenStack
+  license: Apache
+  min_ansible_version: 1.9
+  #
+  # Below are all platforms currently available. Just uncomment
+  # the ones that apply to your role. If you don't see your
+  # platform on this list, let us know and we'll get it added!
+  #
+  platforms:
+  #- name: EL
+  #  versions:
+  #  - all
+  #  - 5
+  #  - 6
+  #  - 7
+  #- name: GenericUNIX
+  #  versions:
+  #  - all
+  #  - any
+  #- name: Fedora
+  #  versions:
+  #  - all
+  #  - 16
+  #  - 17
+  #  - 18
+  #  - 19
+  #  - 20
+  #- name: SmartOS
+  #  versions:
+  #  - all
+  #  - any
+  #- name: opensuse
+  #  versions:
+  #  - all
+  #  - 12.1
+  #  - 12.2
+  #  - 12.3
+  #  - 13.1
+  #  - 13.2
+  #- name: Amazon
+  #  versions:
+  #  - all
+  #  - 2013.03
+  #  - 2013.09
+  #- name: GenericBSD
+  #  versions:
+  #  - all
+  #  - any
+  #- name: FreeBSD
+  #  versions:
+  #  - all
+  #  - 8.0
+  #  - 8.1
+  #  - 8.2
+  #  - 8.3
+  #  - 8.4
+  #  - 9.0
+  #  - 9.1
+  #  - 9.1
+  #  - 9.2
+  #- name: Ubuntu
+  #  versions:
+  #  - all
+  #  - lucid
+  #  - maverick
+  #  - natty
+  #  - oneiric
+  #  - precise
+  #  - quantal
+  #  - raring
+  #  - saucy
+    - trusty
+  #- name: SLES
+  #  versions:
+  #  - all
+  #  - 10SP3
+  #  - 10SP4
+  #  - 11
+  #  - 11SP1
+  #  - 11SP2
+  #  - 11SP3
+  #- name: GenericLinux
+  #  versions:
+  #  - all
+  #  - any
+  #- name: Debian
+  #  versions:
+  #  - all
+  #  - etch
+  #  - lenny
+  #  - squeeze
+  #  - wheezy
+  #
+  # Below are all categories currently available. Just as with
+  # the platforms above, uncomment those that apply to your role.
+  #
+  categories:
+  - cloud
+  - cloud:openstack
+  #- cloud:gce
+  #- cloud:rax
+  #- clustering
+  #- database
+  #- database:nosql
+  #- database:sql
+  #- development
+  #- monitoring
+  #- networking
+  #- packaging
+  #- system
+  #- web
+dependencies: []

playbooks/roles/bifrost-test-inspection/tasks/main.yml

@@ -0,0 +1,40 @@
+# Copyright (c) 2018 StackHPC Ltd.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+---
+# TODO(mgoddard): Ideally we would grab inspection data from ironic inspector
+# rather than going direct to the web server. That would require either
+# installing python-openstackclient, or creating an ansible module that uses
+# python-ironic-inspector-client.
+- block:
+    - name: Check node hardware inspection data
+      uri:
+        url: "{{ inspector_store_data_url ~ '/ironic-inspector/inspector_data-' ~ uuid }}"
+        method: GET
+        return_content: True
+      register: inspection_data
+
+    # TODO(mgoddard): More validation of data format and contents.
+    - name: Validate the inspection data format
+      assert:
+        that:
+          - "'inventory' in data"
+          - "'memory' in inventory"
+          - "'cpu' in inventory"
+          - "'bmc_address' in inventory"
+          - "'interfaces' in inventory"
+          - "'disks' in inventory"
+      vars:
+        data: "{{ inspection_data.content | from_json }}"
+        inventory: "{{ data.inventory }}"
+  when: inspector_store_data_in_nginx | bool

playbooks/roles/bifrost-test-inspection/vars/main.yml

@@ -0,0 +1,2 @@
+---
+# vars file for bifrost-test-inspection

playbooks/test-bifrost.yaml

@@ -99,6 +99,9 @@
   roles:
     - role: ironic-enroll-dynamic
     - { role: ironic-inspect-node, when: inspect_nodes | default('false') | bool == true }
+    - role: bifrost-test-inspection
+      when: inspect_nodes | default('false') | bool == true
+
 
 - hosts: baremetal
   name: "Create configuration drive files and deploy machines"

releasenotes/notes/store-introspection-data-bc4f2fef2f5bb543.yaml

@@ -0,0 +1,9 @@
+---
+features:
+  - |
+    Stores introspection data in nginx.
+
+    In the absence of swift, we can now use the bifrost nginx web server -
+    masquerading as an object store - to store raw and processed introspection
+    data for nodes.  This is configured via the boolean variable
+    ``inspector_store_data_in_nginx`` and is enabled by default.