opendev/system-config
Code Issues Proposed changes
system-config/zuul.d/system-config-run.yaml
Ian Wienand 0c90c128d7
Reference bastion through prod_bastion group 
In thinking harder about the bootstrap process, it struck me that the
"bastion" group we have is two separate ideas that become a bit
confusing because they share a name.

We have the testing and production paths that need to find a single
bridge node so they can run their nested Ansible.  We've recently
merged changes to the setup playbooks to not hard-code the bridge node
and they now use groups["bastion"][0] to find the bastion host -- but
this group is actually orthogonal to the group of the same name
defined in inventory/service/groups.yaml.

The testing and production paths are running on the executor, and, as
mentioned, need to know the bridge node to log into.  For the testing
path this is happening via the group created in the job definition
from zuul.d/system-config-run.yaml.  For the production jobs, this
group is populated via the add-bastion-host role which dynamically
adds the bridge host and group.

Only the *nested* Ansible running on the bastion host reads
s-c:inventory/service/groups.yaml.  None of the nested-ansible
playbooks need to target only the currently active bastion host.  For
example, we can define as many bridge nodes as we like in the
inventory and run service-bridge.yaml against them.  It won't matter
because the production jobs know the host that is the currently active
bridge as described above.

So, instead of using the same group name in two contexts, rename the
testing/production group "prod_bastion".  groups["prod_bastion"][0]
will be the host that the testing/production jobs use as the bastion
host -- references are updated in this change (i.e. the two places
this group is defined -- the group name in the system-config-run jobs,
and add-bastion-host for production).

We then can return the "bastion" group match to bridge*.opendev.org in
inventory/service/groups.yaml.

This fixes a bootstrapping problem -- if you launch, say,
bridge03.opendev.org the launch node script will now apply the
base.yaml playbook against it, and correctly apply all variables from
the "bastion" group which now matches this new host.  This is what we
want to ensure, e.g. the zuul user and keys are correctly populated.

The other thing we can do here is change the testing path
"prod_bastion" hostname to "bridge99.opendev.org".  By doing this we
ensure we're not hard-coding for the production bridge host in any way
(since if both testing and production are called bridge01.opendev.org
we can hide problems).  This is a big advantage when we want to rotate
the production bridge host, as we can be certain there's no hidden
dependencies.

Change-Id: I137ab824b9a09ccb067b8d5f0bb2896192291883
2022-11-04 09:18:35 +11:00

1180 lines
37 KiB
YAML
Raw Blame History

- job:
name: system-config-run
timeout: 3600
description: |
Run the "base" playbook for system-config hosts.
This is a parent job designed to be inherited.
abstract: true
pre-run: playbooks/zuul/run-base-pre.yaml
run: playbooks/zuul/run-base.yaml
post-run: playbooks/zuul/run-base-post.yaml
vars:
install_ansible_ara_enable: true
zuul_copy_output: "{{ copy_output | combine(host_copy_output | default({})) }}"
stage_dir: "{{ ansible_user_dir }}/zuul-output"
copy_output:
'/var/log/syslog': logs_txt
'/var/log/messages': logs_txt
'/var/log/exim4': logs
'/var/log/docker': logs
'/var/log/containers': logs
'/var/log/dstat-csv.log': logs
'/etc/iptables/rules.v4': logs_txt
'/etc/iptables/rules.v6': logs_txt
host-vars:
bridge01.opendev.org:
host_copy_output:
'{{ zuul.project.src_dir }}/junit.xml': logs
'{{ zuul.project.src_dir }}/test-results.html': logs
'{{ zuul.project.src_dir }}/inventory/base/gate-hosts.yaml': logs
'/var/log/screenshots': logs
'/var/log/ansible': logs
# Note: the following two jobs implement the variant-based multiple
# inheritance trick. Both of these variants will always apply,
# therefore both parents will appear in the inheritance hierarchy).
- job:
name: system-config-run-containers
parent: system-config-run
# Note: see above re multiple-inheritance.
- job:
name: system-config-run-containers
parent: opendev-buildset-registry-consumer
description: |
Run the "base" playbook for system-config hosts which use
containers.
This is a parent job designed to be inherited. Use this job if
the service in question is container-based. It expects a
buildset registry and pulls images from the intermediate
registry.
- job:
name: system-config-run-base
parent: system-config-run
description: |
Run the "base" playbook on each of the node types
currently in use.
nodeset:
nodes:
- &bridge_node_x86 { name: bridge99.opendev.org, label: ubuntu-jammy }
- name: xenial
label: ubuntu-xenial
- name: bionic
label: ubuntu-bionic
- name: focal
label: ubuntu-focal
- name: jammy
label: ubuntu-jammy
groups:
# Each job should define this group -- to avoid hard-coding
# the bastion hostname in the job setup, playbooks/tasks refer
# to it only by this group. This should only have one entry
# -- in a couple of places the jobs use the actual hostname
# and assume element [0] here is that hostname.
#
# Note that this shouldn't be confused with the group in
# inventory/service/groups.yaml -- this group contains the
# host that Zuul, running on the executor, will setup as the
# bridge node. This node will then run a nested Ansible to
# test the production playbooks -- *that* Ansible has a
# "bastion" group too
- &bastion_group { name: prod_bastion, nodes: [ bridge99.opendev.org ] }
files:
- tox.ini
- playbooks/
- roles/
- testinfra/
- job:
name: system-config-run-base-ansible-devel
parent: system-config-run-base
description: |
Run the base playbook with the latest ansible.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: xenial
label: ubuntu-xenial
- name: bionic
label: ubuntu-bionic
- name: focal
label: ubuntu-focal
- name: jammy
label: ubuntu-jammy
groups:
- <<: *bastion_group
required-projects:
- name: github.com/ansible/ansible
override-checkout: devel
- name: github.com/pytest-dev/pytest-testinfra
- name: openstack/openstacksdk
- name: github.com/ansible-collections/ansible.posix
override-checkout: main
- name: github.com/ansible-collections/ansible.netcommon
override-checkout: main
- name: github.com/ansible-collections/community.crypto
override-checkout: main
- name: github.com/ansible-collections/community.general
override-checkout: main
- name: github.com/ansible-community/ara
vars:
bridge_ansible_name: '{{ ansible_user_dir}}/src/github.com/ansible/ansible'
bridge_ansible_version: null
bridge_openstacksdk_name: '{{ ansible_user_dir }}/src/opendev.org/openstack/openstacksdk'
bridge_openstacksdk_version: null
bridge_ara_name: '{{ ansible_user_dir}}/src/github.com/ansible-community/ara[server]'
bridge_ara_version: null
# Although we don't have an arm64 based bridge; Zuul can't currently
# allocate a mixed x86/arm64 situation across clouds. Thus it helps
# to keep this clean so we can run the other tests.
- job:
name: system-config-run-base-arm64
parent: system-config-run
description: |
Run the "base" playbook on ARM64.
nodeset:
nodes:
- &bridge_node_arm64 { name: bridge99.opendev.org, label: ubuntu-jammy-arm64 }
- name: bionic
label: ubuntu-bionic-arm64
- name: focal
label: ubuntu-focal-arm64
- name: jammy
label: ubuntu-jammy-arm64
groups:
- <<: *bastion_group
files:
- playbooks/
- roles/
- testinfra/
- job:
name: system-config-run-eavesdrop
parent: system-config-run-containers
description: |
Run the playbook for an eavesdrop server.
required-projects:
- opendev/system-config
- openstack/project-config
requires:
- accessbot-container-image
- gerritbot-container-image
- statusbot-container-image
- ircbot-container-image
- matrix-eavesdrop-container-image
- ptgbot-container-image
nodeset:
nodes:
- <<: *bridge_node_x86
- name: eavesdrop01.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-eavesdrop.yaml
host-vars:
eavesdrop01.opendev.org:
host_copy_output:
'/var/lib/limnoria': logs
'/var/log/apache2': logs
'/var/log/acme.sh': logs
'/etc/apache2': logs
'/var/log/statusbot': logs
'/etc/statusbot': logs
files:
- playbooks/service-eavesdrop.yaml
- playbooks/run-accessbot.yaml
- inventory/service/group_vars/eavesdrop.yaml
- playbooks/roles/install-docker
- playbooks/roles/accessbot
- playbooks/roles/limnoria
- playbooks/roles/logrotate
- playbooks/roles/matrix-eavesdrop
- playbooks/roles/matrix-gerritbot
- playbooks/roles/statusbot
- playbooks/roles/ptgbot
- playbooks/zuul/templates/group_vars/eavesdrop.yaml.j2
- docker/accessbot/
- docker/ircbot
- docker/matrix-eavesdrop
- testinfra/test_eavesdrop.py
- job:
name: system-config-run-letsencrypt
parent: system-config-run
description: |
Run the playbook for letsencrypt key acquisition
nodeset:
nodes:
- <<: *bridge_node_x86
- name: adns-letsencrypt.opendev.org
label: ubuntu-bionic
- name: letsencrypt01.opendev.org
label: ubuntu-bionic
- name: letsencrypt02.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/service-nameserver.yaml
- playbooks/letsencrypt.yaml
# Make sure this test runs acme.sh
letsencrypt_self_generate_tokens: False
host-vars:
bridge99.opendev.org:
host_copy_output:
'/var/lib/certcheck': logs
letsencrypt01.opendev.org:
host_copy_output:
'/var/log/acme.sh': logs
letsencrypt02.opendev.org:
host_copy_output:
'/var/log/acme.sh': logs
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/group_vars/letsencrypt.yaml
# This is split because we ant to avoid
# ...create-certs/handlers/main.yaml matching since every
# letsencrypt user has its handler in there. re2 matching
# doesn't provide us a way to say "everything but this file"
- playbooks/roles/letsencrypt-acme-sh-install
- playbooks/roles/letsencrypt-config-certcheck
- playbooks/roles/letsencrypt-create-certs/defaults
- playbooks/roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- playbooks/roles/letsencrypt-create-certs/handlers/touch_file.yaml
- playbooks/roles/letsencrypt-create-certs/tasks
- playbooks/roles/letsencrypt-install-txt-record
- playbooks/roles/letsencrypt-request-certs
- job:
name: system-config-run-lists
parent: system-config-run
description: |
Run the playbook for a list server.
post-run: playbooks/zuul/run-lists-post.yaml
nodeset:
nodes:
- <<: *bridge_node_x86
- name: lists.openstack.org
label: ubuntu-focal
- name: lists.katacontainers.io
label: ubuntu-focal
groups:
- <<: *bastion_group
required-projects:
- opendev/system-config
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/host_vars/lists.openstack.org.yaml
- inventory/service/host_vars/lists.katacontainers.io.yaml
- inventory/service/group_vars/mailman.yaml
- playbooks/roles/base/exim
- playbooks/roles/mailman
- playbooks/service-lists.yaml
- playbooks/test-lists.yaml
- playbooks/zuul/templates/host_vars/lists.openstack.org.yaml.j2
- playbooks/zuul/templates/host_vars/lists.katacontainers.io.yaml.j2
- testinfra/test_lists_k_i.py
- testinfra/test_lists_o_o.py
- playbooks/zuul/run-lists-post.yaml
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-lists.yaml
# Run this twice to check idempotency
- playbooks/service-lists.yaml
run_test_playbook: playbooks/test-lists.yaml
host-vars:
lists.katacontainers.io:
host_copy_output:
'/var/log/acme.sh': logs
'/var/log/apache2': logs
'/var/log/mailman': logs
lists.openstack.org:
host_copy_output:
'/etc/aliases.domain': logs_txt
'/var/log/acme.sh': logs
'/var/log/apache2': logs
'/var/log/mailman': logs
- job:
name: system-config-run-nodepool
parent: system-config-run
description: |
Run the playbook for nodepool.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: zk04.opendev.org
label: ubuntu-focal
- name: nl01.opendev.org
label: ubuntu-focal
- name: nb01.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
required-projects:
- opendev/system-config
- openstack/project-config
host-vars:
nl01.opendev.org:
host_copy_output:
'/etc/nodepool/nodepool.yaml': logs
'/var/log/nodepool/launcher-debug.log': logs
nb01.opendev.org:
host_copy_output:
'/etc/nodepool/nodepool.yaml': logs
'/var/log/nodepool/builder-debug.log': logs
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-zookeeper.yaml
- playbooks/service-nodepool.yaml
# Test our ad hoc restart playbook works
- playbooks/nodepool_restart.yaml
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/group_vars/nodepool.yaml
- inventory/service/group_vars/nodepool-builder.yaml
- inventory/service/group_vars/nodepool-launcher.yaml
- playbooks/roles/configure-openstacksdk/
- playbooks/roles/nodepool
- playbooks/templates/clouds/
- playbooks/nodepool_restart.yaml
- testinfra/test_nodepool.py
- job:
name: system-config-run-dns
parent: system-config-run
description: |
Run the playbook for dns.
required-projects:
- opendev/zone-opendev.org
- opendev/zone-zuul-ci.org
nodeset:
nodes:
- <<: *bridge_node_x86
- name: adns1.opendev.org
label: ubuntu-bionic
- name: ns1.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/service-nameserver.yaml
host-vars:
adns1.opendev.org:
host_copy_output:
'/etc/bind/named.conf': logs
'/var/lib/bind/zones': logs
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/group_vars/adns.yaml
- inventory/service/group_vars/dns.yaml
- inventory/service/host_vars/(ad)?ns\d+.opendev.org.yaml
- playbooks/zuul/templates/group_vars/adns.yaml.j2
- playbooks/zuul/templates/group_vars/ns.yaml.j2
- playbooks/roles/master-nameserver/
- playbooks/roles/nameserver/
- testinfra/test_adns.py
- testinfra/test_ns.py
- job:
name: system-config-run-borg-backup
parent: system-config-run
description: |
Run the playbook for borg backup configuration
nodeset:
nodes:
- <<: *bridge_node_x86
- name: borg-backup01.region.provider.opendev.org
label: ubuntu-focal
- name: borg-backup-focal.opendev.org
label: ubuntu-focal
- name: borg-backup-bionic.opendev.org
label: ubuntu-bionic
- name: borg-backup-jammy.opendev.org
label: ubuntu-jammy
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/service-borg-backup.yaml
run_test_playbook: playbooks/test-borg-backup.yaml
files:
- playbooks/service-borg-backup.yaml
- playbooks/test-borg-bcakup.yaml
- playbooks/bootstrap-bridge.yaml
- playbooks/roles/install-borg
- playbooks/roles/borg-backup
- playbooks/roles/create-venv
- playbooks/zuul/templates/host_vars/borg-backup
- testinfra/test_borg_backups.py
host-vars:
borg-backup01.region.provider.opendev.org:
host_copy_output:
'/var/log/prune-borg-backups.log': logs
'/var/log/verify-borg-backups.log': logs
borg-backup-bionic.opendev.org:
host_copy_output:
'/var/log/borg-backup-borg-backup01.region.provider.opendev.org.log': logs
borg-backup-focal.opendev.org:
host_copy_output:
'/var/log/borg-backup-borg-backup01.region.provider.opendev.org.log': logs
borg-backup-jammy.opendev.org:
host_copy_output:
'/var/log/borg-backup-borg-backup01.region.provider.opendev.org.log': logs
- job:
name: system-config-run-mirror-base
parent: system-config-run
abstract: true
description: |
Run the playbook for a mirror node
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-mirror.yaml
host-vars:
mirror01.openafs.provider.opendev.org:
host_copy_output:
'/var/log/apache2/': logs
'/var/log/acme.sh': logs
'/etc/apache2/sites-available/mirror.conf': logs
mirror02.openafs.provider.opendev.org:
host_copy_output:
'/var/log/apache2/': logs
'/var/log/acme.sh': logs
'/etc/apache2/sites-available/mirror.conf': logs
files:
- playbooks/bootstrap-bridge.yaml
- roles/
- playbooks/roles/base/
- inventory/service/group_vars/mirror.yaml
- playbooks/roles/mirror/
- playbooks/roles/letsencrypt
- playbooks/letsencrypt.yaml
- playbooks/service-mirror.yaml
- playbooks/zuul/templates/group_vars/mirror.yaml.j2
- testinfra/test_mirror.py
- job:
name: system-config-run-mirror-x86
parent: system-config-run-mirror-base
nodeset:
nodes:
- <<: *bridge_node_x86
- name: mirror01.openafs.provider.opendev.org
label: ubuntu-bionic
- name: mirror02.openafs.provider.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
- job:
name: system-config-run-mirror-arm64
parent: system-config-run-mirror-base
nodeset:
nodes:
- <<: *bridge_node_arm64
- name: mirror01.openafs.provider.opendev.org
label: ubuntu-focal-arm64
groups:
- <<: *bastion_group
- job:
name: system-config-run-mirror-update
parent: system-config-run
description: |
Run the playbook for a mirror update node
nodeset:
nodes:
- <<: *bridge_node_x86
- name: mirror-update99.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/service-mirror-update.yaml
files:
- playbooks/bootstrap-bridge.yaml
- roles/
- playbooks/roles/mirror-update/
- playbooks/roles/reprepro/
- playbooks/roles/afs-release/
- playbooks/service-mirror-update.yaml
- playbooks/zuul/templates/host_vars/mirror-update99.opendev.org.yaml.j2
- testinfra/test_mirror-update.py
- job:
name: system-config-run-docker-registry
parent: system-config-run
description: |
Run the playbook for the docker registry.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: insecure-ci-registry01.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-registry.yaml
host-vars:
insecure-ci-registry01.opendev.org:
host_copy_output:
'/var/registry/auth': logs
'/var/registry/certs': logs
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/group_vars/registry.yaml
- inventory/service/host_vars/insecure-ci-registry\d+.opendev.org.yaml
- playbooks/zuul/templates/group_vars/registry.yaml.j2
- playbooks/roles/letsencrypt-create-certs/handlers/restart_zuul_registry.yaml
- playbooks/roles/registry/
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
- testinfra/test_registry.py
- job:
name: system-config-run-codesearch
parent: system-config-run-containers
description: |
Run the playbook for the codesearch server.
requires: codesearch-container-image
required-projects:
- opendev/system-config
nodeset:
nodes:
- <<: *bridge_node_x86
- name: codesearch01.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-codesearch.yaml
run_test_playbook: playbooks/test-codesearch.yaml
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-codesearch.yaml
- playbooks/roles/codesearch/
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/run-selenium/
- testinfra/util.py
- docker/hound/
- testinfra/test_codesearch.py
- job:
name: system-config-run-etherpad
parent: system-config-run-containers
description: |
Run the playbook for the etherpad servers.
requires: etherpad-container-image
nodeset:
nodes:
- <<: *bridge_node_x86
- name: etherpad01.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-etherpad.yaml
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-etherpad.yaml
- playbooks/roles/etherpad/
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- docker/etherpad/
- testinfra/test_etherpad.py
- job:
name: system-config-run-gitea
parent: system-config-run-containers
description: |
Run the playbook for the gitea servers.
timeout: 4800
nodeset:
nodes:
- <<: *bridge_node_x86
label: ubuntu-bionic
- name: gitea-lb02.opendev.org
label: ubuntu-jammy
- name: gitea99.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
required-projects:
- openstack/project-config
- opendev/system-config
requires:
- gitea-container-image
- haproxy-statsd-container-image
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-gitea-lb.yaml
- playbooks/service-gitea.yaml
- playbooks/manage-projects.yaml
- playbooks/test-update-zuul-description.yaml
# Run twice to ensure that we noop properly when
# all projects are created in gitea. We also update
# zuul's description to ensure that descriptions are
# updated. This uses a test specific playbook to set
# the always_update flag.
- playbooks/test-manage-projects.yaml
run_test_playbook: playbooks/test-gitea.yaml
host-vars:
gitea99.opendev.org:
host_copy_output:
'/var/gitea/conf': logs
'/var/gitea/certs': logs
'/var/gitea/logs': logs
'/var/log/apache2': logs
gitea-lb02.opendev.org:
host_copy_output:
'/var/haproxy/etc': logs
'/var/log/haproxy.log': logs
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-gitea-lb.yaml
- playbooks/service-gitea.yaml
- playbooks/manage-projects.yaml
- playbooks/test-gitea.yaml
- playbooks/rename_repos.yaml
- inventory/service/group_vars/gitea.yaml
- inventory/service/group_vars/gitea-lb.yaml
- inventory/service/host_vars/gitea
- playbooks/zuul/templates/group_vars/gitea.yaml.j2
- playbooks/zuul/templates/group_vars/gitea-lb.yaml.j2
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
- playbooks/roles/letsencrypt
- playbooks/roles/gerrit/
- playbooks/roles/gitea.*
- playbooks/roles/haproxy/
- playbooks/roles/letsencrypt-create-certs/handlers/restart_gitea.yaml
- testinfra/test_gitea.py
- testinfra/test_gitea_lb.py
# From gitea_files -- If we rebuild the image, we want to run
# this job as well.
- docker/gitea/
# From haproxy-statsd_files -- If we rebuild the image, we want
# to run this job as well.
- docker/haproxy-statsd/
- playbooks/roles/run-selenium/
- testinfra/util.py
- job:
name: system-config-run-grafana
parent: system-config-run-containers
description: |
Run the playbook for the etherpad servers.
requires: grafyaml-container-image
required-projects:
- opendev/system-config
- openstack/project-config
nodeset:
nodes:
- <<: *bridge_node_x86
- name: grafana01.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-grafana.yaml
run_test_playbook: playbooks/test-grafana.yaml
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-grafana.yaml
- playbooks/roles/grafana/
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/run-selenium/
- tesinfra/util.py
- testinfra/test_grafana.py
- job:
name: system-config-run-graphite
parent: system-config-run
description: |
Run the playbook for the graphite servers.
required-projects:
- opendev/system-config
nodeset:
nodes:
- <<: *bridge_node_x86
- name: graphite02.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-graphite.yaml
host-vars:
graphite02.opendev.org:
host_copy_output:
'/var/log/graphite': logs
'/etc/graphite-docker': logs
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-graphite.yaml
- playbooks/roles/graphite
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- testinfra/test_graphite.py
- job:
name: system-config-run-keycloak
parent: system-config-run
description: |
Run the playbook for the keycloak servers.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: keycloak01.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-keycloak.yaml
files:
- inventory/service/host_vars/keycloak01.opendev.org.yaml
- playbooks/install-ansible.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-keycloak.yaml
- playbooks/roles/keycloak/
- playbooks/roles/install-docker/
- playbooks/roles/iptables/
- playbooks/zuul/templates/group_vars/keycloak.yaml.j2
- testinfra/test_keycloak.py
- job:
name: system-config-run-meetpad
parent: system-config-run
description: |
Run the playbook for jitsi-meet.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: meetpad01.opendev.org
label: ubuntu-bionic
- name: jvb01.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-meetpad.yaml
host-vars:
meetpad01.opendev.org:
host_copy_output:
'/var/jitsi-meet': logs
jvb01.opendev.org:
host_copy_output:
'/var/jitsi-meet': logs
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/group_vars/meetpad.yaml
- inventory/service/host_vars/meetpad\d+.opendev.org.yaml
- playbooks/roles/letsencrypt-create-certs/handlers/restart_jitsi_meet.yaml
- playbooks/roles/jitsi-meet/
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
- playbooks/zuul/templates/group_vars/meetpad.yaml.j2
- testinfra/test_meetpad.py
- job:
name: system-config-run-paste
parent: system-config-run-containers
description: |
Run the playbook for the paste server.
required-projects:
- opendev/system-config
requires:
- lodgeit-container-image
nodeset:
nodes:
- <<: *bridge_node_x86
- name: paste99.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-paste.yaml
run_test_playbook: playbooks/test-paste.yaml
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-paste.yaml
- playbooks/roles/lodgeit
- playbooks/roles/install-docker/
- playbooks/roles/pip3/
- playbooks/roles/run-selenium/
- testinfra/util.py
- playbooks/test-paste.yaml
- testinfra/test_paste.py
- job:
name: system-config-run-tracing
parent: system-config-run
description: |
Run the playbook for the jaeger servers.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: tracing99.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-tracing.yaml
files:
- inventory/service/group_vars/tracing.yaml
- playbooks/install-ansible.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-tracing.yaml
- playbooks/roles/jaeger/
- playbooks/roles/install-docker/
- playbooks/roles/iptables/
- testinfra/test_tracing.py
- job:
name: system-config-run-zookeeper
parent: system-config-run
description: |
Run the playbook for the zookeeper cluster.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: zk04.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/service-zookeeper.yaml
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/group_vars/zookeeper.yaml
- ^inventory/service/host_vars/zk\d+\..*
- playbooks/roles/zookeeper/
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
- testinfra/test_zookeeper.py
# From zookeeper-statsd_files -- If we rebuild the image, we want
# to run this job as well.
- docker/zookeeper-statsd/
- job:
name: system-config-run-zuul-preview
parent: system-config-run
description: |
Run the playbook for the zuul-preview service.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: zp01.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/service-zuul-preview.yaml
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/roles/zuul-preview/
- playbooks/roles/pip3/
- playbooks/roles/install-docker/
- testinfra/test_zuul_preview.py
- job:
name: system-config-run-zuul
parent: system-config-run
description: |
Run the playbook for the main Zuul cluster.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: zk04.opendev.org
label: ubuntu-focal
- name: zm01.opendev.org
label: ubuntu-focal
- name: ze01.opendev.org
label: ubuntu-focal
- name: zuul02.opendev.org
label: ubuntu-focal
- name: zuul-lb01.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
required-projects:
- openstack/project-config
- opendev/system-config
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-zookeeper.yaml
- playbooks/service-zuul.yaml
- playbooks/service-zuul-lb.yaml
# Test our ad hoc restart playbook works
- playbooks/zuul_restart.yaml
host-vars:
zm01.opendev.org:
host_copy_output:
'/etc/hosts': logs
'/etc/zuul/zuul.conf': logs
'/var/log/zuul/merger-debug.log': logs
ze01.opendev.org:
host_copy_output:
'/etc/hosts': logs
'/etc/zuul/zuul.conf': logs
'/var/log/zuul/executor-debug.log': logs
zuul02.opendev.org:
host_copy_output:
'/etc/hosts': logs
'/etc/zuul/zuul.conf': logs
'/var/log/zuul/debug.log': logs
bridge99.opendev.org:
host_copy_output:
'/etc/hosts': logs
zuul-lb01.opendev.org:
host_copy_output:
'/var/haproxy/etc': logs
'/var/log/haproxy.log': logs
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/service-zookeeper.yaml
- playbooks/service-zuul.yaml
- playbooks/service-zuul-lb.yaml
- inventory/service/group_vars/zuul
- inventory/service/group_vars/zuul-lb.yaml
- inventory/service/group_vars/zookeeper.yaml
- inventory/service/host_vars/zk\d+
- inventory/service/host_vars/zuul02.opendev.org
- playbooks/roles/zookeeper/
- playbooks/roles/install-apt-repo/
- playbooks/roles/zuul.*
- playbooks/zuul/templates/group_vars/zuul.*
- playbooks/zuul/templates/group_vars/zookeeper.yaml
- playbooks/zuul/templates/group_vars/zuul-lb.yaml.j2
- playbooks/zuul/templates/host_vars/zk\d+
- playbooks/zuul/templates/host_vars/zuul02.opendev.org
- playbooks/zuul_restart.yaml
- testinfra/test_zuul_executor.py
- testinfra/test_zuul_scheduler.py
- testinfra/test_zuul_merger.py
- testinfra/util.py
- job:
name: system-config-run-review-base
parent: system-config-run-containers
description: |
Base job for testing gerrit
nodeset:
nodes:
- <<: *bridge_node_x86
- name: review99.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
required-projects:
- openstack/project-config
- opendev/system-config
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-review.yaml
run_test_playbook: playbooks/zuul/bootstrap-and-test-review.yaml
host-vars:
review99.opendev.org:
host_copy_output:
'/home/gerrit2/review_site/etc': logs
'/home/gerrit2/review_site/logs': logs
'/var/log/apache2/': logs
'/var/log/acme.sh': logs
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/letsencrypt.yaml
- playbooks/service-review.*.yaml
- playbooks/rename_repos.yaml
- inventory/service/group_vars/review.yaml
- playbooks/roles/pip3/
- playbooks/roles/run-selenium/
- testinfra/util.py
- playbooks/roles/install-docker/
- playbooks/roles/letsencrypt
- playbooks/roles/gerrit/
- playbooks/zuul/gerrit/
- playbooks/zuul/templates/host_vars/review99.opendev.org.yaml.j2
- roles/bazelisk-build/
- testinfra/test_gerrit.py
- docker/gerrit/
- playbooks/zuul/bootstrap-and-test-review.yaml
- playbooks/zuul/bootstrap-test-review.yaml
- playbooks/zuul/test-review.yaml
- playbooks/zuul/upgrade-review.yaml
- job:
name: system-config-run-review-3.5
parent: system-config-run-review-base
description: |
Run the playbook for gerrit 3.5 (in a container).
requires: gerrit-3.5-container-image
vars:
zuul_test_gerrit_version: '3.5'
- job:
name: system-config-run-review-3.6
parent: system-config-run-review-base
description: |
Run the playbook for gerrit 3.6 (in a container).
requires: gerrit-3.6-container-image
vars:
zuul_test_gerrit_version: '3.6'
- job:
name: system-config-upgrade-review
parent: system-config-run-review-base
description: |
Test we can upgrade a gerrit 3.5 to 3.6
requires:
- gerrit-3.5-container-image
- gerrit-3.6-container-image
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/zuul/upgrade-review.yaml
run_test_playbook: playbooks/zuul/test-review.yaml
zuul_test_gerrit_version: '3.5'
- job:
name: system-config-run-static
parent: system-config-run
description: |
Run the playbook for a static node.
nodeset:
nodes:
- <<: *bridge_node_x86
- name: static99.opendev.org
label: ubuntu-bionic
groups:
- <<: *bastion_group
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-static.yaml
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/roles/static/
- playbooks/roles/letsencrypt
- playbooks/letsencrypt.yaml
- playbooks/service-static.yaml
- testinfra/test_static.py
host-vars:
static99.opendev.org:
host_copy_output:
'/var/log/acme.sh/': logs
'/etc/apache2/': logs
'/var/log/apache2/': logs
- job:
name: system-config-run-refstack
parent: system-config-run
description: |
Run the playbook for refstack server.
requires:
- refstack-container-image
nodeset:
nodes:
- <<: *bridge_node_x86
- name: refstack01.openstack.org
label: ubuntu-focal
groups:
- <<: *bastion_group
host-vars:
refstack01.openstack.org:
host_copy_output:
'/var/log/apache2/': logs
'/var/lib/refstack/': logs
'/var/refstack/': logs
vars:
run_playbooks:
- playbooks/letsencrypt.yaml
- playbooks/service-refstack.yaml
container_command: docker
files:
- playbooks/bootstrap-bridge.yaml
- inventory/service/group-vars/refstack.yaml
- playbooks/zuul/templates/group_vars/refstack.yaml.j2
- playbooks/roles/refstack/
- playbooks/roles/letsencrypt-create-certs/handlers/restart_apache.yaml
- testinfra/test_refstack.py
# If we rebuild the image, we want to run this job as well.
- docker/refstack/.*
- job:
name: system-config-run-kerberos
parent: system-config-run
description: |
Run the playbook for kerberos servers
nodeset:
nodes:
- <<: *bridge_node_x86
- name: kdc-primary.opendev.org
label: ubuntu-focal
- name: kdc-replica.opendev.org
label: ubuntu-focal
groups:
- <<: *bastion_group
host-vars:
kdc-primary.opendev.org:
host_copy_output:
'/etc/krb5kdc/': logs
'/var/krb5kdc/': logs
kdc-replica.opendev.org:
host_copy_output:
'/etc/krb5kdc/': logs
'/var/krb5kdc/': logs
vars:
run_playbooks:
- playbooks/service-kerberos.yaml
# Run twice to double-check idempotence
- playbooks/service-kerberos.yaml
run_test_playbook: playbooks/test-kerberos.yaml
files:
- playbooks/bootstrap-bridge.yaml
- playbooks/roles/kerberos-kdc/
View Git Blame Copy Permalink