Add airshipctl versions catalogue & replacements

This extracts the container, disk image, and Kubernetes versions that
are defined in airshipctl functions, into an `versions-airshipctl`
catalogue in the `airshipctl-catalogues` function.  The default version
values have been left in-place in their resources as well, and will
be used if the user decides not to apply the catalogue via a
transformer at the site level (it's not currently required to actually
use the catalogue or associate replacment rules).

Please see the READMEs in this change for more information.

Note: replacement of CAPI component container versions must be done
through a slightly different mechanism and will be in a
follow-on changeset.

Note that this also incorporates a fix to pin the Ironic images, which
are currently not compatible at the :latest tag.

Co-Authored-By: Alexander Noskov <anoskov@mirantis.com>
Change-Id: Iafe1d185a7e7d3932576055eda819d2270b02482
Relates-To: https://github.com/airshipit/airshipctl/issues/316

manifests/function/airshipctl-catalogues/README.md

@@ -0,0 +1,27 @@
+Function: airshipctl-catalogues
+===============================
+
+This function defines some default VariableCatalogue resources,
+which can be consumed and used (via ReplacementTransformer) to change the
+versioning and resource locations used by functions in the airshipctl project.
+More base catalogues will be added here in the future.
+
+This catalogue can be used as-is to simply apply defaults, or a different
+catalogue may be supplied (with the same ``versions-airshipctl`` name)
+as a kustomize resource.  The catalogue in this function can also be
+patched at the composite, type, or site level to reconfigure the versions.
+
+The versions info falls under these keys:
+
+* charts: Helm chart locations and versions
+
+* files: image file (etc) locations and versions
+
+* images: container image registries and versions
+
+* kubernetes: a standalone key for the Kubernetes version to use
+
+Versions that are defined for specific resources in specific functions
+(e.g., container images) are categorized in the catalogue according
+to the function and resource they will be applied to.
+E.g., ``images.baremetal_operator.ironic.dnsmasq``.

manifests/function/airshipctl-catalogues/kustomization.yaml

@@ -0,0 +1,4 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - versions-airshipctl.yaml

manifests/function/airshipctl-catalogues/versions-airshipctl.yaml

@@ -0,0 +1,49 @@
+# The default versions catalogue for functions hosted in the airshipctl project.
+# These values can be overridden at the site, type, etc levels as appropriate.
+apiVersion: airshipit.org/v1alpha1
+kind: VariableCatalogue
+metadata:
+  name: versions-airshipctl
+  labels:
+    airshipit.org/deploy-k8s: "false"
+
+charts:
+  dex:
+    # TODO: match this data to what is expected by helm-operator or image builder
+    location: TODO
+    reference: TODO
+    subpath: TODO
+    type: TODO
+
+files:
+  k8scontrol:
+    cluster_controlplane_image:
+      url: https://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img
+      checksum: 4a6909d1480ac30d676accd7b37ec711
+
+images:
+  capm3: # Images specific to the camp3 function; etc.
+    manager: quay.io/metal3-io/cluster-api-provider-metal3
+    auth_proxy: gcr.io/kubebuilder/kube-rbac-proxy:v0.4.0
+  cacpk:
+    manager: us.gcr.io/k8s-artifacts-prod/cluster-api/kubeadm-control-plane-controller:v0.3.3
+    auth_proxy: gcr.io/kubebuilder/kube-rbac-proxy:v0.4.1
+  cabpk:
+    manager: us.gcr.io/k8s-artifacts-prod/cluster-api/kubeadm-bootstrap-controller:v0.3.3
+    auth_proxy: gcr.io/kubebuilder/kube-rbac-proxy:v0.4.1
+  capi:
+    manager: us.gcr.io/k8s-artifacts-prod/cluster-api/cluster-api-controller:v0.3.3
+    auth_proxy: gcr.io/kubebuilder/kube-rbac-proxy:v0.4.1
+  baremetal_operator:
+    ironic: # ironic Deployment
+      init_bootstrap: centos
+      dnsmasq: quay.io/metal3-io/ironic:capm3-v0.4.0
+      httpd: quay.io/metal3-io/ironic:capm3-v0.4.0
+      ironic: quay.io/metal3-io/ironic:capm3-v0.4.0
+      ironic_inspector: quay.io/metal3-io/ironic-inspector:capm3-v0.4.0
+    metal3_baremetal_operator: # metal3-baremetal-operator Deployment
+      baremetal_operator: quay.io/metal3-io/baremetal-operator:capm3-v0.4.0
+      ironic_proxy: alpine/socat
+      ironic_inspector_proxy: alpine/socat
+
+kubernetes: v1.18.6

manifests/function/baremetal-operator/README.md

@@ -0,0 +1,11 @@
+Function: baremetal-operator
+============================
+
+This function defines a deployment of the Metal3 baremetal-operator,
+including both the operator itself and Ironic.
+
+Optional: a ``versions-airshipctl`` VariableCatalogue may be used to
+override the default container images.
+A base example for this catalogue can be found in the ``airshipctl-catalogues``
+function.  If using the catalogue, apply the ``replacements/`` entrypoint
+at the site level, as a Kustomize transformer.

manifests/function/baremetal-operator/replacements/kustomization.yaml

@@ -0,0 +1,4 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - versions.yaml

manifests/function/baremetal-operator/replacements/versions.yaml

@@ -0,0 +1,82 @@
+# These rules inject host-specific information from the `host-catalogue`
+# into the hostgenerator-m3 function's Template plugin config.
+apiVersion: airshipit.org/v1alpha1
+kind: ReplacementTransformer
+metadata:
+  name: baremetal-operator-versions-replacements
+replacements:
+# Container versions for the ironic Deployment
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.ironic.init_bootstrap
+  target:
+    objref:
+      kind: Deployment
+      name: ironic
+    fieldrefs: ["spec.template.spec.initContainers[name=init-bootstrap].image"]
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.ironic.dnsmasq
+  target:
+    objref:
+      kind: Deployment
+      name: ironic
+    fieldrefs: ["spec.template.spec.containers[name=dnsmasq].image"]
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.ironic.httpd
+  target:
+    objref:
+      kind: Deployment
+      name: ironic
+    fieldrefs: ["spec.template.spec.containers[name=httpd].image"]
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.ironic.ironic
+  target:
+    objref:
+      kind: Deployment
+      name: ironic
+    fieldrefs: ["spec.template.spec.containers[name=ironic].image"]
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.ironic.ironic_inspector
+  target:
+    objref:
+      kind: Deployment
+      name: ironic
+    fieldrefs: ["spec.template.spec.containers[name=ironic-inspector].image"]
+# Container versions for the metal3-baremetal-operator Deployment
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.metal3_baremetal_operator.baremetal_operator
+  target:
+    objref:
+      kind: Deployment
+      name: metal3-baremetal-operator
+    fieldrefs: ["spec.template.spec.containers[name=baremetal-operator].image"]
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.metal3_baremetal_operator.ironic_proxy
+  target:
+    objref:
+      kind: Deployment
+      name: metal3-baremetal-operator
+    fieldrefs: ["spec.template.spec.containers[name=ironic-proxy].image"]
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: images.baremetal_operator.metal3_baremetal_operator.ironic_inspector_proxy
+  target:
+    objref:
+      kind: Deployment
+      name: metal3-baremetal-operator
+    fieldrefs: ["spec.template.spec.containers[name=ironic-inspector-proxy].image"]
+

manifests/function/k8scontrol/README.md

@@ -0,0 +1,12 @@
+Function: k8scontrol
+====================
+
+This function defines a KubeADM and Metal3 control plane, including
+Cluster, Metal3Cluster, KubeadmControlPlane, and Metal3MachineTemplate
+resources.
+
+Optional: a ``versions-airshipctl`` VariableCatalogue may be used to
+override the default Kubernetes version and controlplane disk image.
+A base example for this catalogue can be found in the ``airshipctl-catalogues``
+function.  If using the catalogue, apply the ``replacements/`` entrypoint
+at the site level, as a Kubernetes transformer.

manifests/function/k8scontrol/replacements/kustomization.yaml

@@ -0,0 +1,4 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+resources:
+  - versions.yaml

manifests/function/k8scontrol/replacements/versions.yaml

@@ -0,0 +1,27 @@
+# These rules inject versioned artifacts into the k8scontrol function.
+apiVersion: airshipit.org/v1alpha1
+kind: ReplacementTransformer
+metadata:
+  name: k8scontrol-versions-replacements
+replacements:
+# Replace the Kubernetes version in the KubeadmControlPlane
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: kubernetes
+  target:
+    objref:
+      kind: KubeadmControlPlane
+      name: cluster-controlplane
+    fieldrefs: ["spec.version"]
+# Replace the controlplane disk image in the Metal3MachineTemplate
+- source:
+    objref:
+      name: versions-airshipctl
+    fieldref: files.k8scontrol.cluster_controlplane_image
+  target:
+    objref:
+      kind: Metal3MachineTemplate
+      name: cluster-controlplane
+    fieldrefs: ["spec.template.spec.image"]
+

manifests/site/test-site/ephemeral/controlplane/control-machine-template-patch.yaml

@@ -1,11 +0,0 @@
----
-apiVersion: infrastructure.cluster.x-k8s.io/v1alpha3
-kind: Metal3MachineTemplate
-metadata:
-  name: cluster-controlplane
-spec:
-  template:
-    spec:
-      image:
-        url: http://10.23.24.1:8099/target-image.qcow2
-        checksum: http://10.23.24.1:8099/target-image.qcow2.md5sum

manifests/site/test-site/ephemeral/controlplane/kustomization.yaml

@@ -4,6 +4,9 @@ resources:
   # TODO (dukov) It's recocommended to upload BareMetalHost objects separately
   # otherwise nodes will hang in 'registering' state for quite a long time
   - nodes
+  - ../../../../function/airshipctl-catalogues
   - ../../../../function/k8scontrol
 patchesStrategicMerge:
-  - control-machine-template-patch.yaml
+  - versions-catalogue-patch.yaml
+transformers:
+  - ../../../../function/k8scontrol/replacements

manifests/site/test-site/ephemeral/controlplane/versions-catalogue-patch.yaml

@@ -0,0 +1,11 @@
+# Patch the versions catalogue to use the site-specific local image URL
+# TODO: patch this in from a site-networking catalogue in the future
+apiVersion: airshipit.org/v1alpha1
+kind: VariableCatalogue
+metadata:
+  name: versions-airshipctl
+files:
+  k8scontrol:
+    cluster_controlplane_image:
+      url: http://10.23.24.1:8099/target-image.qcow2
+      checksum: http://10.23.24.1:8099/target-image.qcow2.md5sum

manifests/site/test-site/ephemeral/initinfra/kustomization.yaml

@@ -1,8 +1,11 @@
 resources:
   - ../../../../composite/infra
   - ../../shared/clusterctl
+  - ../../../../function/airshipctl-catalogues
   - ../../../../function/baremetal-operator
 patchesStrategicMerge:
   - patch_bmo_config.yaml
 commonLabels:
   airshipit.org/stage: initinfra
+transformers:
+  - ../../../../function/baremetal-operator/replacements

manifests/site/test-site/target/initinfra/kustomization.yaml

@@ -1,9 +1,12 @@
 resources:
   - ../../../../composite/infra
   - ../../shared/clusterctl
+  - ../../../../function/airshipctl-catalogues
   - ../../../../function/baremetal-operator
   - ../../../../function/helm-operator
 patchesStrategicMerge:
   - patch_bmo_config.yaml
 commonLabels:
   airshipit.org/stage: initinfra
+transformers:
+  - ../../../../function/baremetal-operator/replacements