Add collection to deploy magnum cluster-api with vexxhost driver
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/908179 Change-Id: I121f5e97453354bb5c0227b296462805e269a7f5
This commit is contained in:
parent
9b50eb1088
commit
e983053d98
3
mcapi_vexxhost/README.md
Normal file
3
mcapi_vexxhost/README.md
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
# Ansible Collection - osa_ops.mcapi_vexxhost
|
||||||
|
|
||||||
|
Documentation for the collection.
|
325
mcapi_vexxhost/doc/source/README.rst
Normal file
325
mcapi_vexxhost/doc/source/README.rst
Normal file
@ -0,0 +1,325 @@
|
|||||||
|
Install vexxhost magnum-cluster-api driver
|
||||||
|
##########################################
|
||||||
|
|
||||||
|
About this repository
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
This repository includes playbooks and roles to deploy the Vexxhost
|
||||||
|
magnum-cluster-api driver for the OpenStack Magnum service.
|
||||||
|
|
||||||
|
The playbooks create a complete deployment including the control plane
|
||||||
|
k8s cluster which should result in a ready-to-go experience for operators.
|
||||||
|
|
||||||
|
The following architectural features are present:
|
||||||
|
|
||||||
|
* The control plane k8s cluster is an integral part of the openstack-ansible
|
||||||
|
deployment, and forms part of the foundational components alongside mariadb
|
||||||
|
and rabbitmq.
|
||||||
|
* The control plane k8s cluster is deployed on the infra hosts and integrated
|
||||||
|
with the haproxy loadbalancer and OpenStack internal API endpoint, and not
|
||||||
|
exposed outside of the deployment
|
||||||
|
* SSL is supported between all components and configuration is
|
||||||
|
possible to support different certificate authorities on the internal
|
||||||
|
and external loadbalancer endpoints.
|
||||||
|
* Control plane traffic can stay entirely within the management network
|
||||||
|
if required
|
||||||
|
* The magnum-cluster-api-proxy service is deployed to allow communication
|
||||||
|
between the control plane and workload clusters when a floating IP is not
|
||||||
|
attached to the workload cluster.
|
||||||
|
|
||||||
|
* It is possible to do a completely offline install for airgapped environments
|
||||||
|
|
||||||
|
The magnum-cluster-api driver for magnum can be found here https://github.com/vexxhost/magnum-cluster-api
|
||||||
|
|
||||||
|
Documentation for the Vexxhost magnum-cluster-api driver is here https://vexxhost.github.io/magnum-cluster-api/
|
||||||
|
|
||||||
|
The ansible collection used to deploy the controlplane k8s cluster is here https://github.com/vexxhost/ansible-collection-kubernetes
|
||||||
|
|
||||||
|
The ansible collection used to deploy the container runtime for the controlplane k8s cluster is here https://github.com/vexxhost/ansible-collection-containers
|
||||||
|
|
||||||
|
**These playbooks require Openstack-Ansible Antelope or later.**
|
||||||
|
|
||||||
|
Highlevel overview of the Magnum infrastructure these playbooks will
|
||||||
|
build and operate against.
|
||||||
|
|
||||||
|
.. image:: mcapi-architecture.png
|
||||||
|
:scale: 100 %
|
||||||
|
:alt: OSA Magnum Cluster API Architecture
|
||||||
|
:align: center
|
||||||
|
|
||||||
|
Pre-requisites
|
||||||
|
--------------
|
||||||
|
|
||||||
|
* An existing openstack-ansible deployment
|
||||||
|
* Control plane using LXC containers, bare metal deployment is not tested
|
||||||
|
* Core openstack services plus Octavia
|
||||||
|
|
||||||
|
OpenStack-Ansible Integration
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
The playbooks are distributed as an ansible collection, and integrate with
|
||||||
|
Openstack-Ansible by adding the collection to the deployment host by
|
||||||
|
adding the following to `/etc/openstack_deploy/user-collection-requirements.yml`
|
||||||
|
under the collections key.
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
collections:
|
||||||
|
- name: vexxhost.kubernetes
|
||||||
|
source: https://github.com/vexxhost/ansible-collection-kubernetes
|
||||||
|
type: git
|
||||||
|
version: main
|
||||||
|
- name: osa_ops.mcapi_vexxhost
|
||||||
|
type: git
|
||||||
|
version: master
|
||||||
|
source: https://opendev.org/openstack/openstack-ansible-ops#/mcapi_vexxhost
|
||||||
|
|
||||||
|
|
||||||
|
The collections can then be installed with the following command:
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
cd /opt/openstack-ansible
|
||||||
|
openstack-ansible scripts/get-ansible-collection-requirements.yml
|
||||||
|
|
||||||
|
OpenStack-Ansible configuration for magnum-cluster-api driver
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Specify the deployment of the control plane k8s cluster in
|
||||||
|
`/etc/openstack_deploy/env.d/k8s.yml`
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
---
|
||||||
|
component_skel:
|
||||||
|
k8s_capi:
|
||||||
|
belongs_to:
|
||||||
|
- k8s_all
|
||||||
|
|
||||||
|
container_skel:
|
||||||
|
k8s_container:
|
||||||
|
belongs_to:
|
||||||
|
- cluster-api_containers
|
||||||
|
contains:
|
||||||
|
- k8s_capi
|
||||||
|
|
||||||
|
physical_skel:
|
||||||
|
cluster-api_containers:
|
||||||
|
belongs_to:
|
||||||
|
- all_containers
|
||||||
|
cluster-api_hosts:
|
||||||
|
belongs_to:
|
||||||
|
- hosts
|
||||||
|
|
||||||
|
Define the physical hosts that will host the controlplane k8s
|
||||||
|
cluster, this example is for an all-in-one deployment and should
|
||||||
|
be adjusted to match a real deployment with multiple hosts if
|
||||||
|
high availability is required.
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
cluster-api_hosts:
|
||||||
|
aio1:
|
||||||
|
ip: 172.29.236.100
|
||||||
|
|
||||||
|
Integrate the control plane k8s cluster with the haproxy loadbalancer
|
||||||
|
in `/etc/openstack-deploy/group_vars/k8s_all/haproxy_service.yml`
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
---
|
||||||
|
haproxy_k8s_service:
|
||||||
|
haproxy_service_name: k8s
|
||||||
|
haproxy_backend_nodes: "{{ groups['k8s_all'] | default([]) }}"
|
||||||
|
haproxy_ssl: false
|
||||||
|
haproxy_ssl_all_vips: false
|
||||||
|
haproxy_port: 6443
|
||||||
|
haproxy_balance_type: tcp
|
||||||
|
haproxy_balance_alg: leastconn
|
||||||
|
haproxy_interval: '15000'
|
||||||
|
haproxy_backend_port: 16443
|
||||||
|
haproxy_backend_rise: 2
|
||||||
|
haproxy_backend_fall: 2
|
||||||
|
haproxy_timeout_server: '15m'
|
||||||
|
haproxy_timeout_client: '5m'
|
||||||
|
haproxy_backend_options:
|
||||||
|
- tcplog
|
||||||
|
- ssl-hello-chk
|
||||||
|
- log-health-checks
|
||||||
|
- httpchk GET /healthz
|
||||||
|
haproxy_backend_httpcheck_options:
|
||||||
|
- 'expect status 200'
|
||||||
|
haproxy_backend_server_options:
|
||||||
|
- check-ssl
|
||||||
|
- verify none
|
||||||
|
haproxy_accept_both_protocols: "{{ k8s_accept_both_protocols | default(openstack_service_accept_both_protocols) }}"
|
||||||
|
haproxy_service_enabled: "{{ groups['k8s_all'] is defined and groups['k8s_all'] | length > 0 }}"
|
||||||
|
|
||||||
|
k8s_haproxy_services:
|
||||||
|
- "{{ haproxy_k8s_service | combine(haproxy_k8s_service_overrides | default({})) }}"
|
||||||
|
|
||||||
|
Configure the LXC container that will host the control plane k8s cluster to
|
||||||
|
be suitable for running nested containers in `/etc/openstack-deploy/group_vars/k8s_all/main.yml`
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
---
|
||||||
|
lxc_container_config_list:
|
||||||
|
- "lxc.apparmor.profile=unconfined"
|
||||||
|
|
||||||
|
lxc_container_mount_auto:
|
||||||
|
- "proc:rw"
|
||||||
|
- "sys:rw"
|
||||||
|
|
||||||
|
Set up config-overrides for the magnum service in `/etc/openstack-deploy/user_variables_magnum.yml`.
|
||||||
|
Adjust the images and flavors here as necessary, these are just for demonstration. Upload as many
|
||||||
|
images as you need for the different workload cluster kubernetes versions.
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
#list the images to upload to glance here, or set to an empty list
|
||||||
|
#to handle image uploading by some other means
|
||||||
|
magnum_glance_images:
|
||||||
|
- disk_format: qcow2
|
||||||
|
distro: ubuntu
|
||||||
|
file: https://object-storage.public.mtl1.vexxhost.net/swift/v1/a91f106f55e64246babde7402c21b87a/magnum-capi/ubuntu-2204-kube-v1.23.17.qcow2
|
||||||
|
image_format: bare
|
||||||
|
name: ubuntu-2204-kube-v1.23.17
|
||||||
|
public: true
|
||||||
|
|
||||||
|
#the cluster templates cannot be created during the magnum installation
|
||||||
|
#as the control plane k8s credentials must be in place first
|
||||||
|
magnum_cluster_templates: []
|
||||||
|
|
||||||
|
#any flavors specified in the cluster template must already exist
|
||||||
|
#the magnum playbook can create flavors, or set to an empty list
|
||||||
|
#to handle flavor creation by some other means
|
||||||
|
magnum_flavors:
|
||||||
|
- cloud: default
|
||||||
|
disk: 40
|
||||||
|
name: m1.medium
|
||||||
|
ram: 4096
|
||||||
|
vcpus: 2
|
||||||
|
|
||||||
|
Set up config-overrides for the control plane k8s cluster in /etc/openstack-deploy/user_variables_k8s.yml`
|
||||||
|
Attention must be given to the SSL configuration. Users and workload clusters will
|
||||||
|
interact with the external endpoint and must trust the SSL certificate. The magnum
|
||||||
|
service and cluster-api can be configured to interact with either the external or
|
||||||
|
internal endpoint and must trust the SSL certificiate. Depending on the environment,
|
||||||
|
these may be derived from different certificate authorities.
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
# connect ansible group, host and network addresses into control plane k8s deployment
|
||||||
|
kubernetes_control_plane_group: k8s_all
|
||||||
|
kubelet_hostname: "{{ ansible_facts['hostname'] }}"
|
||||||
|
kubelet_node_ip: "{{ management_address }}"
|
||||||
|
kubernetes_hostname: "{{ internal_lb_vip_address }}"
|
||||||
|
kubernetes_non_init_namespace: true
|
||||||
|
|
||||||
|
# install the vexxhost magnum-cluster-api plugin into the magnum venv
|
||||||
|
magnum_user_pip_packages:
|
||||||
|
- git+https://github.com/vexxhost/magnum-cluster-api@main#egg=magnum-cluster-api
|
||||||
|
|
||||||
|
# make the required settings in magnum.conf
|
||||||
|
magnum_config_overrides:
|
||||||
|
drivers:
|
||||||
|
# ensure that the external VIP CA is trusted by the workload cluster
|
||||||
|
openstack_ca_file: '/usr/local/share/ca-certificates/ExampleCorpRoot.crt'
|
||||||
|
capi_client:
|
||||||
|
# ensure that the internal VIP CA is trusted by the CAPI driver
|
||||||
|
ca_file: '/usr/local/share/ca-certificates/ExampleCorpRoot.crt'
|
||||||
|
endpoint: 'internalURL'
|
||||||
|
cluster_template:
|
||||||
|
# the only permitted workload network driver is calico
|
||||||
|
kubernetes_allowed_network_drivers: 'calico'
|
||||||
|
kubernetes_default_network_driver: 'calico'
|
||||||
|
certificates:
|
||||||
|
# store certificates in the magnum database instead of barbican
|
||||||
|
cert_manager_type: x509keypair
|
||||||
|
|
||||||
|
# Pick a range of addresses for the control plane k8s cluster cilium
|
||||||
|
# network that do not collide with anything else in the deployment
|
||||||
|
cilium_ipv4_cidr: 172.29.200.0/22
|
||||||
|
|
||||||
|
# Set this manually, or kube-proxy will try to do this - not possible
|
||||||
|
# in a non-init namespace and will fail in LXC
|
||||||
|
openstack_host_nf_conntrack_max: 1572864
|
||||||
|
|
||||||
|
# OSA containers do not run ssh so cannot use the ansible synchronize module
|
||||||
|
upload_helm_chart_method: copy
|
||||||
|
|
||||||
|
Run the deployment
|
||||||
|
------------------
|
||||||
|
|
||||||
|
For a new deployment
|
||||||
|
^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Run the OSA playbooks/setup.yml playbooks as usual, following the normal
|
||||||
|
deployment guide.
|
||||||
|
|
||||||
|
Run the magnum-cluster-api deployment
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
openstack-ansible osa_ops.mcapi_vexxhost.k8s_install
|
||||||
|
|
||||||
|
For an existing deployment
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Create the k8s control plane containers
|
||||||
|
|
||||||
|
openstack-ansible playbooks/lxc-containers-create.yml --limit k8s_all
|
||||||
|
|
||||||
|
Run the magnum-cluster-api deployment
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
openstack-ansible osa_ops.mcapi_vexxhost.k8s_install
|
||||||
|
|
||||||
|
Use Magnum to create a workload cluster
|
||||||
|
---------------------------------------
|
||||||
|
|
||||||
|
Magnum cluster-api should now be ready to use
|
||||||
|
|
||||||
|
Upload Images
|
||||||
|
|
||||||
|
Create a cluster template
|
||||||
|
|
||||||
|
Optional Components
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
Deploy the workload clusters with a local registry
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
TODO - describe how to do this
|
||||||
|
|
||||||
|
Deploy the control plane cluster from a local registry
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
TODO - describe how to do this
|
||||||
|
|
||||||
|
Use of magnum-cluster-api-proxy
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
TODO - describe what this is for
|
||||||
|
|
||||||
|
Troubleshooting
|
||||||
|
---------------
|
||||||
|
|
||||||
|
Local testing
|
||||||
|
-------------
|
||||||
|
|
||||||
|
An OpenStack-Ansible all-in-one configured with Magnum and Octavia is
|
||||||
|
capable of running a functioning magnum-cluster-api deployment.
|
||||||
|
|
||||||
|
Sufficient memory should be available beyond the minimum 8G usually required
|
||||||
|
for an all-in-one. A multinode workload cluster may require nova to boot several
|
||||||
|
Ubuntu images in addition to an Octavia loadbalancer instance. 64G would
|
||||||
|
be an appropriate amount of system RAM.
|
||||||
|
|
||||||
|
There also must be sufficient disk space in `/var/lib/nova/instances` to
|
||||||
|
support the required number of instances - the normal minimum of 60G
|
||||||
|
required for an all-in-one deployment will be insufficient, 500G would
|
||||||
|
be plenty.
|
||||||
|
|
BIN
mcapi_vexxhost/doc/source/mcapi-architecture.png
Normal file
BIN
mcapi_vexxhost/doc/source/mcapi-architecture.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 208 KiB |
62
mcapi_vexxhost/galaxy.yml
Normal file
62
mcapi_vexxhost/galaxy.yml
Normal file
@ -0,0 +1,62 @@
|
|||||||
|
### REQUIRED
|
||||||
|
# The namespace of the collection. This can be a company/brand/organization or product namespace under which all
|
||||||
|
# content lives. May only contain alphanumeric lowercase characters and underscores. Namespaces cannot start with
|
||||||
|
# underscores or numbers and cannot contain consecutive underscores
|
||||||
|
namespace: osa_ops
|
||||||
|
|
||||||
|
# The name of the collection. Has the same character restrictions as 'namespace'
|
||||||
|
name: mcapi_vexxhost
|
||||||
|
|
||||||
|
# The version of the collection. Must be compatible with semantic versioning
|
||||||
|
version: 1.0.0
|
||||||
|
|
||||||
|
# The path to the Markdown (.md) readme file. This path is relative to the root of the collection
|
||||||
|
readme: README.md
|
||||||
|
|
||||||
|
# A list of the collection's content authors. Can be just the name or in the format 'Full Name <email> (url)
|
||||||
|
# @nicks:irc/im.site#channel'
|
||||||
|
authors:
|
||||||
|
- your name <example@domain.com>
|
||||||
|
|
||||||
|
|
||||||
|
### OPTIONAL but strongly recommended
|
||||||
|
# A short summary description of the collection
|
||||||
|
description: your collection description
|
||||||
|
|
||||||
|
# Either a single license or a list of licenses for content inside of a collection. Ansible Galaxy currently only
|
||||||
|
# accepts L(SPDX,https://spdx.org/licenses/) licenses. This key is mutually exclusive with 'license_file'
|
||||||
|
license:
|
||||||
|
- Apache-2.0
|
||||||
|
|
||||||
|
# The path to the license file for the collection. This path is relative to the root of the collection. This key is
|
||||||
|
# mutually exclusive with 'license'
|
||||||
|
license_file: ''
|
||||||
|
|
||||||
|
# A list of tags you want to associate with the collection for indexing/searching. A tag name has the same character
|
||||||
|
# requirements as 'namespace' and 'name'
|
||||||
|
tags: []
|
||||||
|
|
||||||
|
# Collections that this collection requires to be installed for it to be usable. The key of the dict is the
|
||||||
|
# collection label 'namespace.name'. The value is a version range
|
||||||
|
# L(specifiers,https://python-semanticversion.readthedocs.io/en/latest/#requirement-specification). Multiple version
|
||||||
|
# range specifiers can be set and are separated by ','
|
||||||
|
dependencies: {}
|
||||||
|
|
||||||
|
# The URL of the originating SCM repository
|
||||||
|
repository: http://example.com/repository
|
||||||
|
|
||||||
|
# The URL to any online docs
|
||||||
|
documentation: http://docs.example.com
|
||||||
|
|
||||||
|
# The URL to the homepage of the collection/project
|
||||||
|
homepage: http://example.com
|
||||||
|
|
||||||
|
# The URL to the collection issue tracker
|
||||||
|
issues: http://example.com/issue/tracker
|
||||||
|
|
||||||
|
# A list of file glob-like patterns used to filter any files or directories that should not be included in the build
|
||||||
|
# artifact. A pattern is matched from the relative path of the file or directory of the collection directory. This
|
||||||
|
# uses 'fnmatch' to match the files or directories. Some directories and files like 'galaxy.yml', '*.pyc', '*.retry',
|
||||||
|
# and '.git' are always filtered
|
||||||
|
build_ignore: []
|
||||||
|
|
195
mcapi_vexxhost/playbooks/k8s_install.yml
Normal file
195
mcapi_vexxhost/playbooks/k8s_install.yml
Normal file
@ -0,0 +1,195 @@
|
|||||||
|
---
|
||||||
|
# Copyright 2023, BBC R&D
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
- name: Gather k8s facts
|
||||||
|
hosts: k8s_all
|
||||||
|
gather_facts: false
|
||||||
|
tags:
|
||||||
|
- always
|
||||||
|
tasks:
|
||||||
|
- name: Gather minimal facts for k8s
|
||||||
|
setup:
|
||||||
|
gather_subset:
|
||||||
|
- "!all"
|
||||||
|
- min
|
||||||
|
when: osa_gather_facts | default(True)
|
||||||
|
|
||||||
|
- name: Create and configure k8s container
|
||||||
|
hosts: k8s_all
|
||||||
|
serial: "{{ k8s_serial | default('20%') }}"
|
||||||
|
gather_facts: true
|
||||||
|
user: root
|
||||||
|
pre_tasks:
|
||||||
|
- import_role:
|
||||||
|
name: openstack.osa.lxc_container_setup
|
||||||
|
vars:
|
||||||
|
list_of_bind_mounts:
|
||||||
|
- bind_dir_path: '/usr/lib/modules'
|
||||||
|
mount_path: '/usr/lib/modules'
|
||||||
|
- bind_dir_path: '/usr/src'
|
||||||
|
mount_path: '/usr/src'
|
||||||
|
- bind_dir_path: '/dev/kmsg'
|
||||||
|
mount_path: '/dev/kmsg'
|
||||||
|
create: file
|
||||||
|
extra_container_config:
|
||||||
|
- 'security.privileged=true'
|
||||||
|
- 'security.nested=true'
|
||||||
|
- 'raw.lxc="lxc.apparmor.profile=unconfined"'
|
||||||
|
- 'lxc.cap.drop='
|
||||||
|
- 'lxc.cgroup.devices.allow=a'
|
||||||
|
- 'lxc.cgroup2.devices.allow=a'
|
||||||
|
|
||||||
|
when: not is_metal
|
||||||
|
|
||||||
|
- include_tasks: common-tasks/unbound-clients.yml
|
||||||
|
when:
|
||||||
|
- hostvars['localhost']['resolvconf_enabled'] | bool
|
||||||
|
|
||||||
|
- name: ensure kernel headers are installed on host
|
||||||
|
package:
|
||||||
|
name: "linux-headers-{{ ansible_facts['kernel'] }}"
|
||||||
|
state: present
|
||||||
|
delegate_to: "{{ physical_host }}"
|
||||||
|
when: not is_metal
|
||||||
|
|
||||||
|
roles:
|
||||||
|
- role: "openstack.osa.system_crontab_coordination"
|
||||||
|
- role: "systemd_service"
|
||||||
|
systemd_services:
|
||||||
|
- service_name: bpf-mount
|
||||||
|
execstarts: /usr/bin/bash -c '/usr/bin/mount bpffs -t bpf /sys/fs/bpf && /usr/bin/mount --make-shared /sys/fs/bpf'
|
||||||
|
- service_name: cilium-cgroup2-mount
|
||||||
|
execstarts: /usr/bin/bash -c 'mkdir -p /run/cilium/cgroupv2 && /usr/bin/mount -t cgroup2 none /run/cilium/cgroupv2 && /usr/bin/mount --make-shared /run/cilium/cgroupv2'
|
||||||
|
|
||||||
|
# environment: "{{ deployment_environment_variables | default({}) }}"
|
||||||
|
tags:
|
||||||
|
- k8s-container
|
||||||
|
- k8s
|
||||||
|
|
||||||
|
- name: Configure haproxy services
|
||||||
|
import_playbook: openstack.osa.haproxy_service_config.yml
|
||||||
|
vars:
|
||||||
|
service_group: k8s_all
|
||||||
|
service_variable: "k8s_haproxy_services"
|
||||||
|
when: groups[service_group] | length > 0
|
||||||
|
tags:
|
||||||
|
- haproxy-service-config
|
||||||
|
|
||||||
|
- name: Install kubernetes
|
||||||
|
hosts: k8s_all
|
||||||
|
gather_facts: true
|
||||||
|
serial: "{{ k8s_serial | default('20%') }}"
|
||||||
|
user: root
|
||||||
|
roles:
|
||||||
|
- role: "vexxhost.containers.containerd"
|
||||||
|
- role: "vexxhost.kubernetes.kubernetes"
|
||||||
|
- role: "vexxhost.kubernetes.helm"
|
||||||
|
- role: "vexxhost.kubernetes.cilium"
|
||||||
|
environment: "{{ deployment_environment_variables | default({}) }}"
|
||||||
|
tags:
|
||||||
|
- k8s
|
||||||
|
- k8s-install
|
||||||
|
|
||||||
|
- name: Install cluster_api
|
||||||
|
hosts: k8s_all
|
||||||
|
gather_facts: true
|
||||||
|
user: root
|
||||||
|
roles:
|
||||||
|
- role: "vexxhost.kubernetes.cert_manager"
|
||||||
|
- role: "vexxhost.kubernetes.cluster_api"
|
||||||
|
# environment: "{{ deployment_environment_variables | default({}) }}"
|
||||||
|
tags:
|
||||||
|
- cluster-api
|
||||||
|
|
||||||
|
- name: Set up helm and k8s credentials in magnum hosts
|
||||||
|
hosts: magnum_all
|
||||||
|
gather_facts: true
|
||||||
|
user: root
|
||||||
|
vars:
|
||||||
|
k8s_admin_conf_src: "/etc/kubernetes/admin.conf"
|
||||||
|
k8s_admin_conf_dest: "/var/lib/magnum/.kube/config"
|
||||||
|
tasks:
|
||||||
|
- name: Collect admin config from k8s cluster
|
||||||
|
slurp:
|
||||||
|
src: "{{ k8s_admin_conf_src }}"
|
||||||
|
register: k8s_admin_conf_slurp
|
||||||
|
delegate_to: "{{ groups['k8s_all'][0] }}"
|
||||||
|
run_once: true
|
||||||
|
|
||||||
|
- name: Ensure target directory exists
|
||||||
|
file:
|
||||||
|
state: directory
|
||||||
|
path: "{{ k8s_admin_conf_dest | dirname }}"
|
||||||
|
owner: magnum
|
||||||
|
group: magnum
|
||||||
|
|
||||||
|
- name: Write k8s admin config to magnum home dir
|
||||||
|
copy:
|
||||||
|
content: "{{ k8s_admin_conf_slurp.content | b64decode }}"
|
||||||
|
dest: "{{ k8s_admin_conf_dest }}"
|
||||||
|
owner: magnum
|
||||||
|
group: magnum
|
||||||
|
mode: '0600'
|
||||||
|
|
||||||
|
- name: Install helm
|
||||||
|
include_role:
|
||||||
|
name: "vexxhost.kubernetes.helm"
|
||||||
|
|
||||||
|
# environment: "{{ deployment_environment_variables | default({}) }}"
|
||||||
|
tags:
|
||||||
|
- magnum_k8s_conf
|
||||||
|
|
||||||
|
# deploy the proxy service to communicate directly between magnum coe
|
||||||
|
# clusters and the capi control plane without going via a public floating
|
||||||
|
# IP
|
||||||
|
# For OVS/Linuxbridge this can be installed to compute nodes, but for
|
||||||
|
# OVN it must be installed to all computes. Override the target hosts
|
||||||
|
# for this play as required
|
||||||
|
- name: Install magnum-cluster-api-proxy
|
||||||
|
hosts: "{{ magnum_cluster_api_proxy_hosts | default('network_hosts') }}"
|
||||||
|
vars:
|
||||||
|
_venv_tag: "{{ venv_tag | default('untagged') }}"
|
||||||
|
_bin: "/openstack/venvs/magnum-cluster-api-proxy-{{ _venv_tag }}/bin"
|
||||||
|
magnum_cluster_api_proxy_system_group_name: 'capi_proxy'
|
||||||
|
magnum_cluster_api_proxy_system_user_name: 'capi_proxy'
|
||||||
|
magnum_cluster_api_proxy_system_user_comment: 'Magnum Cluster API Proxy System User'
|
||||||
|
magnum_cluster_api_proxy_system_user_home: '/var/lib/{{ magnum_cluster_api_proxy_system_user_name }}'
|
||||||
|
magnum_cluster_api_proxy_system_user_shell: '/bin/false'
|
||||||
|
magnum_cluster_api_proxy_etc_directory: '/etc/capi_proxy'
|
||||||
|
|
||||||
|
k8s_admin_conf_src: "/etc/kubernetes/admin.conf"
|
||||||
|
k8s_admin_conf_dest: "{{ magnum_cluster_api_proxy_system_user_home }}/.kube/config"
|
||||||
|
environment: "{{ deployment_environment_variables | default({}) }}"
|
||||||
|
tasks:
|
||||||
|
|
||||||
|
- name: Gather facts
|
||||||
|
setup:
|
||||||
|
gather_subset:
|
||||||
|
- "!all"
|
||||||
|
- min
|
||||||
|
when: osa_gather_facts | default(True)
|
||||||
|
tags:
|
||||||
|
- always
|
||||||
|
|
||||||
|
- name: Setup installation variables
|
||||||
|
import_role:
|
||||||
|
name: openstack.osa.install_defaults
|
||||||
|
|
||||||
|
- name: Install proxy service
|
||||||
|
include_role:
|
||||||
|
name: osa_ops.mcapi_vexxhost.proxy
|
||||||
|
|
||||||
|
tags:
|
||||||
|
- magnum-cluster-api-proxy
|
101
mcapi_vexxhost/roles/proxy/tasks/main.yml
Normal file
101
mcapi_vexxhost/roles/proxy/tasks/main.yml
Normal file
@ -0,0 +1,101 @@
|
|||||||
|
---
|
||||||
|
# Copyright 2023, BBC R&D
|
||||||
|
#
|
||||||
|
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
# you may not use this file except in compliance with the License.
|
||||||
|
# You may obtain a copy of the License at
|
||||||
|
#
|
||||||
|
# http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
#
|
||||||
|
# Unless required by applicable law or agreed to in writing, software
|
||||||
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
# See the License for the specific language governing permissions and
|
||||||
|
# limitations under the License.
|
||||||
|
|
||||||
|
# create virtualenv
|
||||||
|
- name: Install the python venv
|
||||||
|
import_role:
|
||||||
|
name: "python_venv_build"
|
||||||
|
vars:
|
||||||
|
_upper_constraints_url: "{{ requirements_git_url | default('https://releases.openstack.org/constraints/upper/' ~ requirements_git_install_branch | default('master')) }}"
|
||||||
|
_git_constraints:
|
||||||
|
- "--constraint {{ _upper_constraints_url }}"
|
||||||
|
venv_python_executable: "{{ openstack_venv_python_executable | default('python3') }}"
|
||||||
|
venv_build_constraints: "{{ _git_constraints }}"
|
||||||
|
venv_install_destination_path: "{{ _bin | dirname }}"
|
||||||
|
#venv_install_distro_package_list:
|
||||||
|
# - haproxy # this will be there for free on the host in an AIO
|
||||||
|
venv_pip_install_args: "{{ pip_install_options | default('') }}"
|
||||||
|
venv_pip_packages:
|
||||||
|
- git+https://github.com/vexxhost/magnum-cluster-api@main#egg=magnum-cluster-api
|
||||||
|
|
||||||
|
# create user and group
|
||||||
|
- name: Create the magnum_cluster_api_proxy system group
|
||||||
|
group:
|
||||||
|
name: "{{ magnum_cluster_api_proxy_system_group_name }}"
|
||||||
|
state: "present"
|
||||||
|
system: "yes"
|
||||||
|
|
||||||
|
- name: Create the magnum_cluster_api_proxy system user
|
||||||
|
user:
|
||||||
|
name: "{{ magnum_cluster_api_proxy_system_user_name }}"
|
||||||
|
group: "{{ magnum_cluster_api_proxy_system_group_name }}"
|
||||||
|
comment: "{{ magnum_cluster_api_proxy_system_user_comment }}"
|
||||||
|
shell: "{{ magnum_cluster_api_proxy_system_user_shell }}"
|
||||||
|
system: "yes"
|
||||||
|
createhome: "yes"
|
||||||
|
home: "{{ magnum_cluster_api_proxy_system_user_home }}"
|
||||||
|
|
||||||
|
- name: Create magnum_cluster_api_proxy directories
|
||||||
|
file:
|
||||||
|
path: "{{ item.path }}"
|
||||||
|
state: "directory"
|
||||||
|
owner: "{{ item.owner | default(magnum_cluster_api_proxy_system_user_name) }}"
|
||||||
|
group: "{{ item.group | default(magnum_cluster_api_proxy_system_group_name) }}"
|
||||||
|
mode: "{{ item.mode | default('0750') }}"
|
||||||
|
with_items:
|
||||||
|
- path: "{{ magnum_cluster_api_proxy_etc_directory }}"
|
||||||
|
- path: "{{ magnum_cluster_api_proxy_system_user_home }}"
|
||||||
|
- path: "{{ magnum_cluster_api_proxy_system_user_home }}/.kube"
|
||||||
|
|
||||||
|
- name: Collect admin config from k8s cluster
|
||||||
|
slurp:
|
||||||
|
src: "{{ k8s_admin_conf_src }}"
|
||||||
|
register: k8s_admin_conf_slurp
|
||||||
|
delegate_to: "{{ groups['k8s_all'][0] }}"
|
||||||
|
run_once: true
|
||||||
|
|
||||||
|
- name: Write k8s admin config to capi_proxy home dir
|
||||||
|
copy:
|
||||||
|
content: "{{ k8s_admin_conf_slurp.content | b64decode }}"
|
||||||
|
dest: "{{ k8s_admin_conf_dest }}"
|
||||||
|
owner: "{{ magnum_cluster_api_proxy_system_user_name }}"
|
||||||
|
group: "{{ magnum_cluster_api_proxy_system_group_name }}"
|
||||||
|
mode: '0600'
|
||||||
|
|
||||||
|
- name: Write capi_proxy sudoers config
|
||||||
|
template:
|
||||||
|
src: capi_sudoers.j2
|
||||||
|
dest: /etc/sudoers.d/capi_proxy_sudoers
|
||||||
|
|
||||||
|
# create service
|
||||||
|
- name: Run the systemd service role
|
||||||
|
import_role:
|
||||||
|
name: systemd_service
|
||||||
|
vars:
|
||||||
|
systemd_user_name: "{{ magnum_cluster_api_proxy_system_user_name }}"
|
||||||
|
systemd_group_name: "{{ magnum_cluster_api_proxy_system_group_name }}"
|
||||||
|
systemd_service_restart_changed: true
|
||||||
|
systemd_tempd_prefix: openstack
|
||||||
|
systemd_slice_name: magnum-cluster-api-proxy
|
||||||
|
systemd_lock_path: /var/lock/magnum-cluster-api-proxy
|
||||||
|
systemd_service_cpu_accounting: true
|
||||||
|
systemd_service_block_io_accounting: true
|
||||||
|
systemd_service_memory_accounting: true
|
||||||
|
systemd_service_tasks_accounting: true
|
||||||
|
systemd_services:
|
||||||
|
- service_name: magnum-cluster-api-proxy
|
||||||
|
execstarts:
|
||||||
|
- "{{ _bin ~ '/magnum-cluster-api-proxy' }}"
|
||||||
|
start_order: 1
|
6
mcapi_vexxhost/roles/proxy/templates/capi_sudoers.j2
Normal file
6
mcapi_vexxhost/roles/proxy/templates/capi_sudoers.j2
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# {{ ansible_managed }}
|
||||||
|
|
||||||
|
Defaults:{{ magnum_cluster_api_proxy_system_user_name }} !requiretty
|
||||||
|
Defaults:{{ magnum_cluster_api_proxy_system_user_name }} secure_path="{{ _bin }}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||||
|
|
||||||
|
{{ magnum_cluster_api_proxy_system_user_name }} ALL = (root) NOPASSWD: {{ _bin }}/privsep-helper
|
Loading…
x
Reference in New Issue
Block a user