Add ClusterAPI documentation to renderred docs
Change-Id: I4f802681d494bfce2b8fdc1127c7e916a29d9944
This commit is contained in:
Dmitriy Rabotyagov
parent
92f00c4486
commit
a0b16b4faf
1
doc/source/assets/mcapi-architecture.png
Symbolic link
1
doc/source/assets/mcapi-architecture.png
Symbolic link
@ -0,0 +1 @@
|
||||
../../../mcapi_vexxhost/assets/mcapi-architecture.png
|
||||
@ -7,6 +7,7 @@ OpenStack-Ansible Operator Tooling
|
||||
|
||||
swift_storage_mount_drives
|
||||
elk_metrics
|
||||
mcapi
|
||||
|
||||
OpenStack-Ansible Diff Generator
|
||||
--------------------------------
|
||||
|
||||
2
doc/source/mcapi.rst
Normal file
2
doc/source/mcapi.rst
Normal file
@ -0,0 +1,2 @@
|
||||
|
||||
.. include:: ../../mcapi_vexxhost/README.rst
|
||||
@ -1,3 +0,0 @@
|
||||
# Ansible Collection - osa_ops.mcapi_vexxhost
|
||||
|
||||
Documentation for the collection.
|
||||
370
mcapi_vexxhost/README.rst
Normal file
370
mcapi_vexxhost/README.rst
Normal file
@ -0,0 +1,370 @@
|
||||
Install vexxhost magnum-cluster-api driver
|
||||
##########################################
|
||||
|
||||
About this repository
|
||||
---------------------
|
||||
|
||||
This repository includes playbooks and roles to deploy the Vexxhost
|
||||
magnum-cluster-api driver for the OpenStack Magnum service.
|
||||
|
||||
The playbooks create a complete deployment including the control plane
|
||||
k8s cluster which should result in a ready-to-go experience for operators.
|
||||
|
||||
The following architectural features are present:
|
||||
|
||||
* The control plane k8s cluster is an integral part of the openstack-ansible
|
||||
deployment, and forms part of the foundational components alongside mariadb
|
||||
and rabbitmq.
|
||||
* The control plane k8s cluster is deployed on the infra hosts and integrated
|
||||
with the haproxy loadbalancer and OpenStack internal API endpoint, and not
|
||||
exposed outside of the deployment
|
||||
* SSL is supported between all components and configuration is
|
||||
possible to support different certificate authorities on the internal
|
||||
and external loadbalancer endpoints.
|
||||
* Control plane traffic can stay entirely within the management network
|
||||
if required
|
||||
* The magnum-cluster-api-proxy service is deployed to allow communication
|
||||
between the control plane and workload clusters when a floating IP is not
|
||||
attached to the workload cluster.
|
||||
|
||||
* It is possible to do a completely offline install for airgapped environments
|
||||
|
||||
The magnum-cluster-api driver for magnum can be found here https://github.com/vexxhost/magnum-cluster-api
|
||||
|
||||
Documentation for the Vexxhost magnum-cluster-api driver is here https://vexxhost.github.io/magnum-cluster-api/
|
||||
|
||||
The ansible collection used to deploy the controlplane k8s cluster is here https://github.com/vexxhost/ansible-collection-kubernetes
|
||||
|
||||
The ansible collection used to deploy the container runtime for the controlplane k8s cluster is here https://github.com/vexxhost/ansible-collection-containers
|
||||
|
||||
**These playbooks require Openstack-Ansible Caracal or later.**
|
||||
|
||||
Highlevel overview of the Magnum infrastructure these playbooks will
|
||||
build and operate against.
|
||||
|
||||
.. image:: assets/mcapi-architecture.png
|
||||
:scale: 100 %
|
||||
:alt: OSA Magnum Cluster API Architecture
|
||||
:align: center
|
||||
|
||||
Pre-requisites
|
||||
--------------
|
||||
|
||||
* An existing openstack-ansible deployment
|
||||
* Control plane using LXC containers, bare metal deployment is not tested
|
||||
* Core openstack services plus Octavia
|
||||
|
||||
OpenStack-Ansible Integration
|
||||
-----------------------------
|
||||
|
||||
The playbooks are distributed as an ansible collection, and integrate with
|
||||
Openstack-Ansible by adding the collection to the deployment host by
|
||||
adding the following to `/etc/openstack_deploy/user-collection-requirements.yml`
|
||||
under the collections key.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
collections:
|
||||
- name: vexxhost.kubernetes
|
||||
source: https://github.com/vexxhost/ansible-collection-kubernetes
|
||||
type: git
|
||||
version: main
|
||||
- name: osa_ops.mcapi_vexxhost
|
||||
type: git
|
||||
version: master
|
||||
source: https://opendev.org/openstack/openstack-ansible-ops#/mcapi_vexxhost
|
||||
|
||||
|
||||
The collections can then be installed with the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd /opt/openstack-ansible
|
||||
openstack-ansible scripts/get-ansible-collection-requirements.yml
|
||||
|
||||
The modules in the kubernetes collection require an additional python module
|
||||
to be present in the ansible-runtime python virtual environment. Specify this
|
||||
in /etc/openstack_deploy/user-ansible-venv-requirements.txt
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker-image-py
|
||||
|
||||
OpenStack-Ansible configuration for magnum-cluster-api driver
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Specify the deployment of the control plane k8s cluster in
|
||||
`/etc/openstack_deploy/env.d/k8s.yml`
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
component_skel:
|
||||
k8s_capi:
|
||||
belongs_to:
|
||||
- k8s_all
|
||||
|
||||
container_skel:
|
||||
k8s_container:
|
||||
belongs_to:
|
||||
- cluster-api_containers
|
||||
contains:
|
||||
- k8s_capi
|
||||
|
||||
physical_skel:
|
||||
cluster-api_containers:
|
||||
belongs_to:
|
||||
- all_containers
|
||||
cluster-api_hosts:
|
||||
belongs_to:
|
||||
- hosts
|
||||
|
||||
Define the physical hosts that will host the controlplane k8s
|
||||
cluster, this example is for an all-in-one deployment and should
|
||||
be adjusted to match a real deployment with multiple hosts if
|
||||
high availability is required.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
cluster-api_hosts:
|
||||
aio1:
|
||||
ip: 172.29.236.100
|
||||
|
||||
Integrate the control plane k8s cluster with the haproxy loadbalancer
|
||||
in `/etc/openstack-deploy/group_vars/k8s_all/haproxy_service.yml`
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
haproxy_k8s_service:
|
||||
haproxy_service_name: k8s
|
||||
haproxy_backend_nodes: "{{ groups['k8s_all'] | default([]) }}"
|
||||
haproxy_ssl: false
|
||||
haproxy_ssl_all_vips: false
|
||||
haproxy_port: 6443
|
||||
haproxy_balance_type: tcp
|
||||
haproxy_balance_alg: leastconn
|
||||
haproxy_interval: '15000'
|
||||
haproxy_backend_port: 6443
|
||||
haproxy_backend_rise: 2
|
||||
haproxy_backend_fall: 2
|
||||
haproxy_timeout_server: '15m'
|
||||
haproxy_timeout_client: '5m'
|
||||
haproxy_backend_options:
|
||||
- tcplog
|
||||
- ssl-hello-chk
|
||||
- log-health-checks
|
||||
haproxy_backend_httpcheck_options:
|
||||
- 'send hdr User-Agent "osa-haproxy-healthcheck" meth GET uri /healthz'
|
||||
haproxy_backend_server_options:
|
||||
- check-ssl
|
||||
- verify none
|
||||
haproxy_accept_both_protocols: "{{ k8s_accept_both_protocols | default(openstack_service_accept_both_protocols) }}"
|
||||
haproxy_service_enabled: "{{ groups['k8s_all'] is defined and groups['k8s_all'] | length > 0 }}"
|
||||
|
||||
k8s_haproxy_services:
|
||||
- "{{ haproxy_k8s_service | combine(haproxy_k8s_service_overrides | default({})) }}"
|
||||
|
||||
Configure the LXC container that will host the control plane k8s cluster to
|
||||
be suitable for running nested containers in `/etc/openstack-deploy/group_vars/k8s_all/main.yml`
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
lxc_container_config_list:
|
||||
- "lxc.apparmor.profile=unconfined"
|
||||
|
||||
lxc_container_mount_auto:
|
||||
- "proc:rw"
|
||||
- "sys:rw"
|
||||
|
||||
Set up config-overrides for the magnum service in `/etc/openstack-deploy/user_variables_magnum.yml`.
|
||||
Adjust the images and flavors here as necessary, these are just for demonstration. Upload as many
|
||||
images as you need for the different workload cluster kubernetes versions.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
#list the images to upload to glance here, or set to an empty list
|
||||
#to handle image uploading by some other means
|
||||
magnum_glance_images:
|
||||
- disk_format: qcow2
|
||||
distro: ubuntu
|
||||
file: https://object-storage.public.mtl1.vexxhost.net/swift/v1/a91f106f55e64246babde7402c21b87a/magnum-capi/ubuntu-2204-kube-v1.23.17.qcow2
|
||||
image_format: bare
|
||||
name: ubuntu-2204-kube-v1.23.17
|
||||
public: true
|
||||
|
||||
#the cluster templates cannot be created during the magnum installation
|
||||
#as the control plane k8s credentials must be in place first
|
||||
magnum_cluster_templates: []
|
||||
|
||||
#any flavors specified in the cluster template must already exist
|
||||
#the magnum playbook can create flavors, or set to an empty list
|
||||
#to handle flavor creation by some other means
|
||||
magnum_flavors:
|
||||
- cloud: default
|
||||
disk: 40
|
||||
name: m1.medium
|
||||
ram: 4096
|
||||
vcpus: 2
|
||||
|
||||
Set up config-overrides for the control plane k8s cluster in /etc/openstack-deploy/user_variables_k8s.yml`
|
||||
Attention must be given to the SSL configuration. Users and workload clusters will
|
||||
interact with the external endpoint and must trust the SSL certificate. The magnum
|
||||
service and cluster-api can be configured to interact with either the external or
|
||||
internal endpoint and must trust the SSL certificiate. Depending on the environment,
|
||||
these may be derived from different certificate authorities.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# connect ansible group, host and network addresses into control plane k8s deployment
|
||||
kubernetes_control_plane_group: k8s_all
|
||||
kubelet_hostname: "{{ ansible_facts['hostname'] }}"
|
||||
kubelet_node_ip: "{{ management_address }}"
|
||||
kubernetes_hostname: "{{ internal_lb_vip_address }}"
|
||||
kubernetes_non_init_namespace: true
|
||||
|
||||
# install the vexxhost magnum-cluster-api plugin into the magnum venv
|
||||
magnum_user_pip_packages:
|
||||
- git+https://github.com/vexxhost/magnum-cluster-api@main#egg=magnum-cluster-api
|
||||
|
||||
# make the required settings in magnum.conf
|
||||
magnum_config_overrides:
|
||||
drivers:
|
||||
# ensure that the external VIP CA is trusted by the workload cluster
|
||||
openstack_ca_file: '/usr/local/share/ca-certificates/ExampleCorpRoot.crt'
|
||||
capi_client:
|
||||
# ensure that the internal VIP CA is trusted by the CAPI driver
|
||||
ca_file: '/usr/local/share/ca-certificates/ExampleCorpRoot.crt'
|
||||
endpoint: 'internalURL'
|
||||
cluster_template:
|
||||
# the only permitted workload network driver is calico
|
||||
kubernetes_allowed_network_drivers: 'calico'
|
||||
kubernetes_default_network_driver: 'calico'
|
||||
certificates:
|
||||
# store certificates in the magnum database instead of barbican
|
||||
cert_manager_type: x509keypair
|
||||
|
||||
# Pick a range of addresses for the control plane k8s cluster cilium
|
||||
# network that do not collide with anything else in the deployment
|
||||
cilium_ipv4_cidr: 172.29.200.0/22
|
||||
|
||||
# Set this manually, or kube-proxy will try to do this - not possible
|
||||
# in a non-init namespace and will fail in LXC
|
||||
openstack_host_nf_conntrack_max: 1572864
|
||||
|
||||
# OSA containers do not run ssh so cannot use the ansible synchronize module
|
||||
upload_helm_chart_method: copy
|
||||
|
||||
Run the deployment
|
||||
------------------
|
||||
|
||||
For a new deployment
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Run the OSA playbooks/setup.yml playbooks as usual, following the normal
|
||||
deployment guide.
|
||||
|
||||
Run the magnum-cluster-api deployment
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
openstack-ansible osa_ops.mcapi_vexxhost.k8s_install
|
||||
|
||||
For an existing deployment
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Ensure that the python modules required for ansible are present:
|
||||
|
||||
.. code-block: bash
|
||||
|
||||
./scripts/bootstrap-ansible.sh
|
||||
|
||||
Alternatively, without re-running the bootstrap script:
|
||||
|
||||
.. code-block bash
|
||||
|
||||
/opt/ansible-runtime/bin/pip install docker-image-py
|
||||
|
||||
Add the magnum-cluser-api driver to the magnum service
|
||||
|
||||
.. code-block: bash
|
||||
|
||||
openstack-ansible playbooks/os-magnum-install.yml
|
||||
|
||||
Create the k8s control plane containers
|
||||
|
||||
.. code-block: bash
|
||||
|
||||
openstack-ansible playbooks/lxc-containers-create.yml --limit k8s_all
|
||||
|
||||
Run the magnum-cluster-api deployment
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
openstack-ansible osa_ops.mcapi_vexxhost.k8s_install
|
||||
|
||||
Optionally run a functional test of magnum-cluster-api
|
||||
------------------------------------------------------
|
||||
|
||||
This can be done quickly using the following playbook
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
openstack-ansible osa_ops.mcapi_vexxhost.functional_test
|
||||
|
||||
This playbook will create a neutron public network, download a prebuilt
|
||||
k8s glance image, create a nova flavor and a magnum cluster template.
|
||||
|
||||
It will then deploy the workload k8s cluster using magnum, and
|
||||
run a sonobouy "quick mode" test of the workload cluster.
|
||||
|
||||
This playbook is intended to be used on an openstack-ansible
|
||||
all-in-one deployment.
|
||||
|
||||
Use Magnum to create a workload cluster
|
||||
---------------------------------------
|
||||
|
||||
Upload Images
|
||||
|
||||
Create a cluster template
|
||||
|
||||
Create a workload cluster
|
||||
|
||||
Optional Components
|
||||
-------------------
|
||||
|
||||
Deploy the workload clusters with a local registry
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
TODO - describe how to do this
|
||||
|
||||
Deploy the control plane cluster from a local registry
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
TODO - describe how to do this
|
||||
|
||||
Use of magnum-cluster-api-proxy
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
TODO - describe what this is for
|
||||
|
||||
Troubleshooting
|
||||
---------------
|
||||
|
||||
Local testing
|
||||
-------------
|
||||
|
||||
An OpenStack-Ansible all-in-one configured with Magnum and Octavia is
|
||||
capable of running a functioning magnum-cluster-api deployment.
|
||||
|
||||
Sufficient memory should be available beyond the minimum 8G usually required
|
||||
for an all-in-one. A multinode workload cluster may require nova to boot several
|
||||
Ubuntu images in addition to an Octavia loadbalancer instance. 64G would
|
||||
be an appropriate amount of system RAM.
|
||||
|
||||
There also must be sufficient disk space in `/var/lib/nova/instances` to
|
||||
support the required number of instances - the normal minimum of 60G
|
||||
required for an all-in-one deployment will be insufficient, 500G would
|
||||
be plenty.
|
||||
|
||||
|
Before Width: | Height: | Size: 208 KiB After Width: | Height: | Size: 208 KiB |
@ -1,370 +1,3 @@
|
||||
Install vexxhost magnum-cluster-api driver
|
||||
##########################################
|
||||
|
||||
About this repository
|
||||
---------------------
|
||||
|
||||
This repository includes playbooks and roles to deploy the Vexxhost
|
||||
magnum-cluster-api driver for the OpenStack Magnum service.
|
||||
|
||||
The playbooks create a complete deployment including the control plane
|
||||
k8s cluster which should result in a ready-to-go experience for operators.
|
||||
|
||||
The following architectural features are present:
|
||||
|
||||
* The control plane k8s cluster is an integral part of the openstack-ansible
|
||||
deployment, and forms part of the foundational components alongside mariadb
|
||||
and rabbitmq.
|
||||
* The control plane k8s cluster is deployed on the infra hosts and integrated
|
||||
with the haproxy loadbalancer and OpenStack internal API endpoint, and not
|
||||
exposed outside of the deployment
|
||||
* SSL is supported between all components and configuration is
|
||||
possible to support different certificate authorities on the internal
|
||||
and external loadbalancer endpoints.
|
||||
* Control plane traffic can stay entirely within the management network
|
||||
if required
|
||||
* The magnum-cluster-api-proxy service is deployed to allow communication
|
||||
between the control plane and workload clusters when a floating IP is not
|
||||
attached to the workload cluster.
|
||||
|
||||
* It is possible to do a completely offline install for airgapped environments
|
||||
|
||||
The magnum-cluster-api driver for magnum can be found here https://github.com/vexxhost/magnum-cluster-api
|
||||
|
||||
Documentation for the Vexxhost magnum-cluster-api driver is here https://vexxhost.github.io/magnum-cluster-api/
|
||||
|
||||
The ansible collection used to deploy the controlplane k8s cluster is here https://github.com/vexxhost/ansible-collection-kubernetes
|
||||
|
||||
The ansible collection used to deploy the container runtime for the controlplane k8s cluster is here https://github.com/vexxhost/ansible-collection-containers
|
||||
|
||||
**These playbooks require Openstack-Ansible Caracal or later.**
|
||||
|
||||
Highlevel overview of the Magnum infrastructure these playbooks will
|
||||
build and operate against.
|
||||
|
||||
.. image:: mcapi-architecture.png
|
||||
:scale: 100 %
|
||||
:alt: OSA Magnum Cluster API Architecture
|
||||
:align: center
|
||||
|
||||
Pre-requisites
|
||||
--------------
|
||||
|
||||
* An existing openstack-ansible deployment
|
||||
* Control plane using LXC containers, bare metal deployment is not tested
|
||||
* Core openstack services plus Octavia
|
||||
|
||||
OpenStack-Ansible Integration
|
||||
-----------------------------
|
||||
|
||||
The playbooks are distributed as an ansible collection, and integrate with
|
||||
Openstack-Ansible by adding the collection to the deployment host by
|
||||
adding the following to `/etc/openstack_deploy/user-collection-requirements.yml`
|
||||
under the collections key.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
collections:
|
||||
- name: vexxhost.kubernetes
|
||||
source: https://github.com/vexxhost/ansible-collection-kubernetes
|
||||
type: git
|
||||
version: main
|
||||
- name: osa_ops.mcapi_vexxhost
|
||||
type: git
|
||||
version: master
|
||||
source: https://opendev.org/openstack/openstack-ansible-ops#/mcapi_vexxhost
|
||||
|
||||
|
||||
The collections can then be installed with the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd /opt/openstack-ansible
|
||||
openstack-ansible scripts/get-ansible-collection-requirements.yml
|
||||
|
||||
The modules in the kubernetes collection require an additional python module
|
||||
to be present in the ansible-runtime python virtual environment. Specify this
|
||||
in /etc/openstack_deploy/user-ansible-venv-requirements.txt
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
docker-image-py
|
||||
|
||||
OpenStack-Ansible configuration for magnum-cluster-api driver
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Specify the deployment of the control plane k8s cluster in
|
||||
`/etc/openstack_deploy/env.d/k8s.yml`
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
component_skel:
|
||||
k8s_capi:
|
||||
belongs_to:
|
||||
- k8s_all
|
||||
|
||||
container_skel:
|
||||
k8s_container:
|
||||
belongs_to:
|
||||
- cluster-api_containers
|
||||
contains:
|
||||
- k8s_capi
|
||||
|
||||
physical_skel:
|
||||
cluster-api_containers:
|
||||
belongs_to:
|
||||
- all_containers
|
||||
cluster-api_hosts:
|
||||
belongs_to:
|
||||
- hosts
|
||||
|
||||
Define the physical hosts that will host the controlplane k8s
|
||||
cluster, this example is for an all-in-one deployment and should
|
||||
be adjusted to match a real deployment with multiple hosts if
|
||||
high availability is required.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
cluster-api_hosts:
|
||||
aio1:
|
||||
ip: 172.29.236.100
|
||||
|
||||
Integrate the control plane k8s cluster with the haproxy loadbalancer
|
||||
in `/etc/openstack-deploy/group_vars/k8s_all/haproxy_service.yml`
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
haproxy_k8s_service:
|
||||
haproxy_service_name: k8s
|
||||
haproxy_backend_nodes: "{{ groups['k8s_all'] | default([]) }}"
|
||||
haproxy_ssl: false
|
||||
haproxy_ssl_all_vips: false
|
||||
haproxy_port: 6443
|
||||
haproxy_balance_type: tcp
|
||||
haproxy_balance_alg: leastconn
|
||||
haproxy_interval: '15000'
|
||||
haproxy_backend_port: 6443
|
||||
haproxy_backend_rise: 2
|
||||
haproxy_backend_fall: 2
|
||||
haproxy_timeout_server: '15m'
|
||||
haproxy_timeout_client: '5m'
|
||||
haproxy_backend_options:
|
||||
- tcplog
|
||||
- ssl-hello-chk
|
||||
- log-health-checks
|
||||
haproxy_backend_httpcheck_options:
|
||||
- 'send hdr User-Agent "osa-haproxy-healthcheck" meth GET uri /healthz'
|
||||
haproxy_backend_server_options:
|
||||
- check-ssl
|
||||
- verify none
|
||||
haproxy_accept_both_protocols: "{{ k8s_accept_both_protocols | default(openstack_service_accept_both_protocols) }}"
|
||||
haproxy_service_enabled: "{{ groups['k8s_all'] is defined and groups['k8s_all'] | length > 0 }}"
|
||||
|
||||
k8s_haproxy_services:
|
||||
- "{{ haproxy_k8s_service | combine(haproxy_k8s_service_overrides | default({})) }}"
|
||||
|
||||
Configure the LXC container that will host the control plane k8s cluster to
|
||||
be suitable for running nested containers in `/etc/openstack-deploy/group_vars/k8s_all/main.yml`
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
---
|
||||
lxc_container_config_list:
|
||||
- "lxc.apparmor.profile=unconfined"
|
||||
|
||||
lxc_container_mount_auto:
|
||||
- "proc:rw"
|
||||
- "sys:rw"
|
||||
|
||||
Set up config-overrides for the magnum service in `/etc/openstack-deploy/user_variables_magnum.yml`.
|
||||
Adjust the images and flavors here as necessary, these are just for demonstration. Upload as many
|
||||
images as you need for the different workload cluster kubernetes versions.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
#list the images to upload to glance here, or set to an empty list
|
||||
#to handle image uploading by some other means
|
||||
magnum_glance_images:
|
||||
- disk_format: qcow2
|
||||
distro: ubuntu
|
||||
file: https://object-storage.public.mtl1.vexxhost.net/swift/v1/a91f106f55e64246babde7402c21b87a/magnum-capi/ubuntu-2204-kube-v1.23.17.qcow2
|
||||
image_format: bare
|
||||
name: ubuntu-2204-kube-v1.23.17
|
||||
public: true
|
||||
|
||||
#the cluster templates cannot be created during the magnum installation
|
||||
#as the control plane k8s credentials must be in place first
|
||||
magnum_cluster_templates: []
|
||||
|
||||
#any flavors specified in the cluster template must already exist
|
||||
#the magnum playbook can create flavors, or set to an empty list
|
||||
#to handle flavor creation by some other means
|
||||
magnum_flavors:
|
||||
- cloud: default
|
||||
disk: 40
|
||||
name: m1.medium
|
||||
ram: 4096
|
||||
vcpus: 2
|
||||
|
||||
Set up config-overrides for the control plane k8s cluster in /etc/openstack-deploy/user_variables_k8s.yml`
|
||||
Attention must be given to the SSL configuration. Users and workload clusters will
|
||||
interact with the external endpoint and must trust the SSL certificate. The magnum
|
||||
service and cluster-api can be configured to interact with either the external or
|
||||
internal endpoint and must trust the SSL certificiate. Depending on the environment,
|
||||
these may be derived from different certificate authorities.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
# connect ansible group, host and network addresses into control plane k8s deployment
|
||||
kubernetes_control_plane_group: k8s_all
|
||||
kubelet_hostname: "{{ ansible_facts['hostname'] }}"
|
||||
kubelet_node_ip: "{{ management_address }}"
|
||||
kubernetes_hostname: "{{ internal_lb_vip_address }}"
|
||||
kubernetes_non_init_namespace: true
|
||||
|
||||
# install the vexxhost magnum-cluster-api plugin into the magnum venv
|
||||
magnum_user_pip_packages:
|
||||
- git+https://github.com/vexxhost/magnum-cluster-api@main#egg=magnum-cluster-api
|
||||
|
||||
# make the required settings in magnum.conf
|
||||
magnum_config_overrides:
|
||||
drivers:
|
||||
# ensure that the external VIP CA is trusted by the workload cluster
|
||||
openstack_ca_file: '/usr/local/share/ca-certificates/ExampleCorpRoot.crt'
|
||||
capi_client:
|
||||
# ensure that the internal VIP CA is trusted by the CAPI driver
|
||||
ca_file: '/usr/local/share/ca-certificates/ExampleCorpRoot.crt'
|
||||
endpoint: 'internalURL'
|
||||
cluster_template:
|
||||
# the only permitted workload network driver is calico
|
||||
kubernetes_allowed_network_drivers: 'calico'
|
||||
kubernetes_default_network_driver: 'calico'
|
||||
certificates:
|
||||
# store certificates in the magnum database instead of barbican
|
||||
cert_manager_type: x509keypair
|
||||
|
||||
# Pick a range of addresses for the control plane k8s cluster cilium
|
||||
# network that do not collide with anything else in the deployment
|
||||
cilium_ipv4_cidr: 172.29.200.0/22
|
||||
|
||||
# Set this manually, or kube-proxy will try to do this - not possible
|
||||
# in a non-init namespace and will fail in LXC
|
||||
openstack_host_nf_conntrack_max: 1572864
|
||||
|
||||
# OSA containers do not run ssh so cannot use the ansible synchronize module
|
||||
upload_helm_chart_method: copy
|
||||
|
||||
Run the deployment
|
||||
------------------
|
||||
|
||||
For a new deployment
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Run the OSA playbooks/setup.yml playbooks as usual, following the normal
|
||||
deployment guide.
|
||||
|
||||
Run the magnum-cluster-api deployment
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
openstack-ansible osa_ops.mcapi_vexxhost.k8s_install
|
||||
|
||||
For an existing deployment
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Ensure that the python modules required for ansible are present:
|
||||
|
||||
.. code-block: bash
|
||||
|
||||
./scripts/bootstrap-ansible.sh
|
||||
|
||||
Alternatively, without re-running the bootstrap script:
|
||||
|
||||
.. code-block bash
|
||||
|
||||
/opt/ansible-runtime/bin/pip install docker-image-py
|
||||
|
||||
Add the magnum-cluser-api driver to the magnum service
|
||||
|
||||
.. code-block: bash
|
||||
|
||||
openstack-ansible playbooks/os-magnum-install.yml
|
||||
|
||||
Create the k8s control plane containers
|
||||
|
||||
.. code-block: bash
|
||||
|
||||
openstack-ansible playbooks/lxc-containers-create.yml --limit k8s_all
|
||||
|
||||
Run the magnum-cluster-api deployment
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
openstack-ansible osa_ops.mcapi_vexxhost.k8s_install
|
||||
|
||||
Optionally run a functional test of magnum-cluster-api
|
||||
-----------------------------------------------------
|
||||
|
||||
This can be done quickly using the following playbook
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
openstack-ansible osa_ops.mcapi_vexxhost.functional_test
|
||||
|
||||
This playbook will create a neutron public network, download a prebuilt
|
||||
k8s glance image, create a nova flavor and a magnum cluster template.
|
||||
|
||||
It will then deploy the workload k8s cluster using magnum, and
|
||||
run a sonobouy "quick mode" test of the workload cluster.
|
||||
|
||||
This playbook is intended to be used on an openstack-ansible
|
||||
all-in-one deployment.
|
||||
|
||||
Use Magnum to create a workload cluster
|
||||
---------------------------------------
|
||||
|
||||
Upload Images
|
||||
|
||||
Create a cluster template
|
||||
|
||||
Create a workload cluster
|
||||
|
||||
Optional Components
|
||||
-------------------
|
||||
|
||||
Deploy the workload clusters with a local registry
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
TODO - describe how to do this
|
||||
|
||||
Deploy the control plane cluster from a local registry
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
TODO - describe how to do this
|
||||
|
||||
Use of magnum-cluster-api-proxy
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
TODO - describe what this is for
|
||||
|
||||
Troubleshooting
|
||||
---------------
|
||||
|
||||
Local testing
|
||||
-------------
|
||||
|
||||
An OpenStack-Ansible all-in-one configured with Magnum and Octavia is
|
||||
capable of running a functioning magnum-cluster-api deployment.
|
||||
|
||||
Sufficient memory should be available beyond the minimum 8G usually required
|
||||
for an all-in-one. A multinode workload cluster may require nova to boot several
|
||||
Ubuntu images in addition to an Octavia loadbalancer instance. 64G would
|
||||
be an appropriate amount of system RAM.
|
||||
|
||||
There also must be sufficient disk space in `/var/lib/nova/instances` to
|
||||
support the required number of instances - the normal minimum of 60G
|
||||
required for an all-in-one deployment will be insufficient, 500G would
|
||||
be plenty.
|
||||
|
||||
Documentation for the collection can be found via the link:
|
||||
https://docs.openstack.org/openstack-ansible-ops/latest/openstack-ansible-ops/mcapi.html
|
||||
|
||||
Loading…
Reference in New Issue
Block a user