Add docs for baremetal jobs

Baremetal jobs are now running on internal Software Factory
instance of Zuul. This doc update gives an overview of how it
works and where it lives.

I also provides an example on how one would replicate the config
to run baremetal jobs on a similar environment.

Change-Id: I8698379de18207b4bfbcf7ce59ea88d6087b1cb3

doc/source/ci/baremetal_jobs.rst

@@ -0,0 +1,278 @@
+Baremetal jobs
+==============
+
+This section gives an overview and some details on the baremetal CI jobs. The
+baremetal deployment is intended as a multinode real world "production-like"
+environment for TripleO - see :doc:`../install/environments/baremetal`
+for more information on setting up a baremetal environment.
+
+The baremetal jobs, previously running in the RDO Phase 2 of the promotion
+pipeline from Jenkins servers, now are triggered from an internal Software
+Factory instance of Zuul. These promotion jobs testing containers built on
+tripleo-ci-testing hashes run on real baremetal hardware, report to dlrn and
+can be included in the TripleO promotion criteria.
+
+The goal is to give developers feedback on real deployments and allow us to
+have better coverage on issues seen in production environments. It also
+allows an aproximation of OVB jobs running in RDO cloud in order to get an
+"apples-to-apples" comparison to eliminate infra issues.
+
+
+Where
+-----
+
+The hardware is maintained internally and cannot be accessed by upstream
+Zuul or RDO Cloud. The internal Software Factory instance provides a version
+of infra upstream tools as Zuul, Gerrit and Nodepool for running the defined
+baremetal jobs. Refer to `Software_Factory_documentation_` for more details.
+
+The jobs will use hardware_environments/<env name>/instackenv.json file and the
+hardware_environments/<env name>/network_configs/single_nic_vlans settings file.
+These configurations are explored in more detail below.
+
+.. _Software_Factory_documentation: https://softwarefactory-project.io/docs/index.html
+
+
+How
+---
+
+The baremetal job workflow is described as follows:
+
+  1. The baremetal jobs are triggered in the periodic pipeline and initially
+     run on a Nodepool node that can be called as "executor", where the job
+     starts executing its playbooks and roles.
+
+  2. The job ssh's to the baremetal machine which will host the undercloud vm
+     and creates a new vm on which the undercloud will be installed and set
+     up.
+
+  3. Finally the undercloud VM deploys the overcloud on real baremetal nodes
+     defined in the instackenv.json configuration over pxe boot.
+
+This workflow for baremetal jobs is illustrated in the following figure:
+
+.. image:: ./_images/baremetal-jobs.svg
+
+
+Parenting from upstream and RDO repos
+-------------------------------------
+
+Jobs that run from internal Zuul can parent off, and use resources (jobs,
+roles etc.) from, upstream (review.openstack.org) and RDO
+(review.rdoproject.org) repos. As such, duplication can be kept to a minimum
+and jobs that run internally on baremetal hardware can maintain parity with
+OVB jobs run in RDO Cloud.
+
+For example, a base TripleO CI job in Zuul ::
+
+   - job:
+      name: tripleo-ci-base-baremetal
+      abstract: true
+      description: |
+         Base abstract job for Baremetal TripleO
+      parent: tripleo-ci-base
+      nodeset: tripleo-baremetal-centos-7-primary
+      attempts: 1
+      required-projects:
+         - rdo-jobs
+      roles:
+         - zuul: rdo-jobs
+      pre-run:
+         - playbooks/configure-mirrors.yaml
+         - playbooks/copy-env-vars-baremetal.yaml
+      vars:
+         # must be overridden
+         undercloud: <undercloud>
+         environment_infra: baremetal
+         environment_type: baremetal
+         playbooks:
+         - baremetal-prep-virthost.yml
+         - baremetal-full-undercloud.yml
+         - baremetal-full-overcloud-prep.yml
+         - baremetal-full-overcloud.yml
+         - baremetal-full-overcloud-validate.yml
+         tags:
+         - all
+
+Now adding the dlrn reporting ::
+
+   - job:
+      name: tripleo-ci-base-baremetal-dlrn
+      parent: tripleo-ci-base-baremetal
+      abstract: true
+      description: |
+         Base abstract job to do DLRN reporting
+      required-projects:
+         - config
+      roles:
+         - zuul: config
+      pre-run:
+         - playbooks/dlrn/pre-dlrn.yaml
+      post-run:
+         - playbooks/dlrn/post-dlrn.yaml
+      secrets:
+         - dlrnapi
+
+Example of a specfic hardware job in Zuul:
+
+Note that multiple jobs cannot be run on the hardware concurrently.
+The base job is modified to include semaphore
+https://zuul-ci.org/docs/zuul/user/config.html#semaphore to run
+each only one at a time ::
+
+   - job:
+      name: tripleo-ci-base-baremetal-dlrn-my_env
+      abstract: true
+      parent: tripleo-ci-base-baremetal-dlrn
+      vars:
+         baremetal_env_vars: >-
+           {{ local_working_dir }}/hardware_environments/my_env/<truncated_path>/env_settings.yml
+         undercloud: <my_env-undercloud-baremetal-host-address>
+      semaphore:
+         name: my_env
+
+   - job:
+      name: periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master
+      parent: tripleo-ci-base-baremetal-dlrn-my_env
+      vars:
+         nodes: 3ctlr_1comp
+         featureset: '001'
+         release: master
+
+
+Hardware Settings
+-----------------
+
+An example of hardware settings for baremetal environment 'my_env' is shown
+below:
+
+hardware_environments / my_env / network_configs / single_nic_vlans /
+ env_settings.yml ::
+
+   environment_type: my_env
+
+   # undercloud.conf settings
+   undercloud_network_cidr: 10.10.10.0/26
+   undercloud_local_ip: 10.10.10.1/26
+   undercloud_network_gateway: 10.10.10.100
+   undercloud_undercloud_public_vip: 10.10.10.2
+   undercloud_undercloud_admin_vip: 10.10.10.3
+   undercloud_local_interface: eth1
+   undercloud_masquerade_network: 10.10.10.0/26
+   undercloud_dhcp_start: 10.10.10.5
+   undercloud_dhcp_end: 10.10.10.24
+   undercloud_inspection_iprange: 10.10.10.25,10.10.10.39
+   undercloud_undercloud_nameservers: 10.10.10.200
+   network_isolation_ipv4_cidr: 10.10.10.64/26
+   undercloud_external_network_cidr: 10.10.10.64/26
+
+   # undercloud vm settings
+   virthost_provisioning_interface: eno2
+   virthost_provisioning_ip: 10.10.10.4
+   virthost_provisioning_netmask: 255.255.255.192
+   virthost_provisioning_hwaddr: FF:FF:FF:FF:FF:FF
+   virthost_ext_provision_interface: eno1
+
+   undercloud_memory: 28672
+   undercloud_disk: 80
+   undercloud_vcpu: 8
+
+   undercloud_instackenv_template: >-
+     {{ local_working_dir }}/hardware_environments/my_env/instackenv.json
+
+   undercloud_type: virtual
+   step_introspect: true
+   introspect: true
+
+   # network-environment.yaml settings
+   network_environment_args:
+   InternalApiNetCidr: 172.21.33.0/24
+   StorageNetCidr: 172.21.36.0/24
+   StorageMgmtNetCidr: 172.21.35.0/24
+   TenantNetCidr: 172.16.0.0/24
+   ExternalNetCidr: 10.10.10.64/26
+   BondInterfaceOvsOptions: "mode=4 lacp_rate=fast"
+   InternalApiAllocationPools: [{'start': '172.21.33.10', 'end': '172.21.33.200'}]
+   StorageAllocationPools: [{'start': '172.21.36.10', 'end': '172.21.36.200'}]
+   StorageMgmtAllocationPools: [{'start': '172.21.35.10', 'end': '172.21.35.200'}]
+   TenantAllocationPools: [{'start': '172.16.0.10', 'end': '172.16.0.200'}]
+   # Leave room for floating IPs starting at .128
+   ExternalAllocationPools: [{'start': '10.10.10.101', 'end': '10.10.10.120'}]
+   ExternalInterfaceDefaultRoute: 10.10.10.130
+   InternalApiNetworkVlanID: 1272
+   StorageNetworkVlanID: 1273
+   StorageMgmtNetworkVlanID: 1274
+   ExternalNetworkVlanID: 113
+   TenantNetworkVlanID: 1275
+   NeutronExternalNetworkBridge: "''"
+   PublicVirtualFixedIPs: [{"ip_address": "10.10.10.90"}]
+   ControlPlaneSubnetCidr: "26"
+   ControlPlaneDefaultRoute: 10.10.10.1
+   EC2MetadataIp: 10.10.10.1
+   DnsServers: ["8.8.8.8", "8.8.4.4"]
+   NtpServer: ["216.239.35.12","time.google.com","0.north-america.pool.ntp.org"]
+
+   step_root_device_size: false
+   step_install_upstream_ipxe: false
+   hw_env: my_env
+   enable_vbmc: false
+
+hardware_environments / my_env / instackenv.json ::
+
+   {
+   "nodes": [
+      {
+         "pm_password": "<passwd>",
+         "pm_type": "ipmi",
+         "mac": [
+         "FF:FF:FF:FF:FF:FF"
+         ],
+         "cpu": "12",
+         "memory": "32768",
+         "disk": "558",
+         "arch": "x86_64",
+         "pm_user": "Administrator",
+         "pm_addr": "10.1.1.11"
+      },
+      {
+         "pm_password": "<passwd>",
+         "pm_type": "ipmi",
+         "mac": [
+         "FF:FF:FF:FF:FF:FF"
+         ],
+         "cpu": "12",
+         "memory": "32768",
+         "disk": "558",
+         "arch": "x86_64",
+         "pm_user": "Administrator",
+         "pm_addr": "10.1.1.12"
+      },
+      {
+         "pm_password": "<passwd>",
+         "pm_type": "ipmi",
+         "mac": [
+         "FF:FF:FF:FF:FF:FF"
+         ],
+         "cpu": "12",
+         "memory": "32768",
+         "disk": "558",
+         "arch": "x86_64",
+         "pm_user": "Administrator",
+         "pm_addr": "10.1.1.13"
+      },
+      {
+         "pm_password": "<passwd>",
+         "pm_type": "ipmi",
+         "mac": [
+         "FF:FF:FF:FF:FF:FF"
+         ],
+         "cpu": "12",
+         "memory": "32768",
+         "disk": "558",
+         "arch": "x86_64",
+         "pm_user": "Administrator",
+         "pm_addr": "10.1.1.14"
+      }
+   ]
+   }
+

doc/source/ci/index.rst

@@ -9,5 +9,6 @@ TripleO CI Guide
   reproduce-ci
   check_gates
   standalone_scenario_jobs
+  baremetal_jobs
   dlrn-promoter-overview
   stages-overview