openstack/openstack-helm-infra
Code Issues Proposed changes

Cleanup unused scripts

Browse Source 
Change-Id: I3bad13cc332fd439b3b56cfa5fc596255bc466f2
This commit is contained in:
Vladimir Kozhukalov 2024-04-23 17:08:36 -05:00
parent a3a348c7b3
commit 427b0163eb
251 changed files with 147 additions and 6282 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/source/index.rst
View File

@ -6,7 +6,6 @@ Contents:
.. toctree:: .. toctree::
:maxdepth: 2 :maxdepth: 2
install/index
contributor/contributing contributor/contributing
testing/index testing/index
monitoring/index monitoring/index

9
doc/source/install/index.rst
View File

@ -1,9 +0,0 @@
Installation
============
Contents:
.. toctree::
:maxdepth: 2
multinode

237
doc/source/install/multinode.rst
View File

@ -1,237 +0,0 @@
======================
Development Deployment
======================
Deploy Local Docker Registry
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/010-deploy-docker-registry.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/010-deploy-docker-registry.sh
Deploy Cluster and Namespace Ingress Controllers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/common/ingress.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/020-ingress.sh
Deploy Ceph
^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/030-ceph.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/030-ceph.sh
Activate the OSH-Infra namespace to be able to use Ceph
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/035-ceph-ns-activate.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/035-ceph-ns-activate.sh
Deploy LDAP
^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/040-ldap.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/040-ldap.sh
Deploy MariaDB
^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/045-mariadb.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/045-mariadb.sh
Deploy Prometheus
^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/050-prometheus.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/050-prometheus.sh
Deploy Alertmanager
^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/060-alertmanager.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/060-alertmanager.sh
Deploy Kube-State-Metrics
^^^^^^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/070-kube-state-metrics.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/070-kube-state-metrics.sh
Deploy Node Exporter
^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/080-node-exporter.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/080-node-exporter.sh
Deploy Process Exporter
^^^^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/085-process-exporter.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/085-process-exporter.sh
Deploy OpenStack Exporter
^^^^^^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/090-openstack-exporter.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/090-openstack-exporter.sh
Deploy Grafana
^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/100-grafana.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/100-grafana.sh
Deploy Nagios
^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/110-nagios.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/110-nagios.sh
Deploy Rados Gateway for OSH-Infra
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/115-radosgw-osh-infra.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/115-radosgw-osh-infra.sh
Deploy Elasticsearch
^^^^^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/120-elasticsearch.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/120-elasticsearch.sh
Deploy Fluentbit
^^^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/125-fluentbit.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/125-fluentbit.sh
Deploy Fluentd
^^^^^^^^^^^^^^
.. literalinclude:: ../../../tools/deployment/multinode/130-fluentd.sh
:language: shell
:lines: 1,17-
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./tools/deployment/multinode/130-fluentd.sh

1
tools/deployment/apparmor/000-install-packages.sh
View File

@ -1 +0,0 @@
../common/000-install-packages.sh

1
tools/deployment/apparmor/001-setup-apparmor-profiles.sh
View File

@ -1 +0,0 @@
../common/001-setup-apparmor-profiles.sh

1
tools/deployment/apparmor/005-deploy-k8s.sh
View File

@ -1 +0,0 @@
../../gate/deploy-k8s.sh

1
tools/deployment/apparmor/020-ceph.sh
View File

@ -1 +0,0 @@
../osh-infra-logging/020-ceph.sh

1
tools/deployment/apparmor/025-ceph-ns-activate.sh
View File

@ -1 +0,0 @@
../osh-infra-logging/025-ceph-ns-activate.sh

36
tools/deployment/apparmor/030-mariadb.sh
View File

@ -1,36 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make mariadb
: ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB:="$(./tools/deployment/common/get-values-overrides.sh mariadb)"}
#NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
helm upgrade --install mariadb ./mariadb \
--namespace=osh-infra \
--set monitoring.prometheus.enabled=true \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
# Delete the test pod if it still exists
kubectl delete pods -l application=mariadb,release_group=mariadb,component=test --namespace=osh-infra --ignore-not-found
#NOTE: Validate the deployment
helm test mariadb --namespace osh-infra

79
tools/deployment/apparmor/040-memcached.sh
View File

@ -1,79 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
namespace="osh-infra"
: ${OSH_INFRA_EXTRA_HELM_ARGS_MEMCACHED:="$(./tools/deployment/common/get-values-overrides.sh memcached)"}
# NOTE: Lint and package chart
make memcached
tee /tmp/memcached.yaml <<EOF
images:
tags:
apparmor_loader: google/apparmor-loader:latest
pod:
mandatory_access_control:
type: apparmor
memcached:
memcached: runtime/default
EOF
# NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
helm upgrade --install memcached ./memcached \
--namespace=$namespace \
--set pod.replicas.server=1 \
--values=/tmp/memcached.yaml \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_MEMCACHED}
# NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh $namespace
# Run a test. Note: the simple "cat /proc/1/attr/current" verification method
# will not work, as memcached has multiple processes running, so we have to
# find out which one is the memcached application process.
pod=$(kubectl -n $namespace get pod | grep memcached | awk '{print $1}')
unsorted_process_file="/tmp/unsorted_proc_list"
sorted_process_file="/tmp/proc_list"
expected_profile="docker-default (enforce)"
# Grab the processes (numbered directories) from the /proc directory,
# and then sort them. Highest proc number indicates most recent process.
kubectl -n $namespace exec $pod -- ls -1 /proc | grep -e "^[0-9]*$" > $unsorted_process_file
sort --numeric-sort $unsorted_process_file > $sorted_process_file
# The last/latest process in the list will actually be the "ls" command above,
# which isn't running any more, so remove it.
sed -i '$ d' $sorted_process_file
while IFS='' read -r process || [[ -n "$process" ]]; do
echo "Process ID: $process"
proc_name=`kubectl -n $namespace exec $pod -- cat /proc/$process/status | grep "Name:" | awk -F' ' '{print $2}'`
echo "Process Name: $proc_name"
profile=`kubectl -n $namespace exec $pod -- cat /proc/$process/attr/current`
echo "Profile running: $profile"
if test "$profile" != "$expected_profile"
then
if test "$proc_name" == "pause"
then
echo "Root process (pause) can run docker-default, it's ok."
else
echo "$profile is the WRONG PROFILE!!"
return 1
fi
fi
done < $sorted_process_file

175
tools/deployment/apparmor/050-libvirt.sh
View File

@ -1,175 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make libvirt
tee /tmp/libvirt.yaml <<EOF
images:
tags:
apparmor_loader: google/apparmor-loader:latest
pod:
mandatory_access_control:
type: apparmor
configmap_apparmor: true
libvirt-libvirt-default:
libvirt-libvirt-default: localhost/my-apparmor-v1
apparmor-loader: unconfined
conf:
apparmor_profiles:
my-apparmor-v1.profile: |-
#include <tunables/global>
@{LIBVIRT}="libvirt"
profile my-apparmor-v1 flags=(attach_disconnected) {
#include <abstractions/base>
#include <abstractions/dbus>
capability kill,
capability audit_write,
capability audit_control,
capability net_admin,
capability net_raw,
capability setgid,
capability sys_admin,
capability sys_module,
capability sys_ptrace,
capability sys_pacct,
capability sys_nice,
capability sys_chroot,
capability setuid,
capability dac_override,
capability dac_read_search,
capability fowner,
capability chown,
capability setpcap,
capability mknod,
capability fsetid,
capability audit_write,
capability ipc_lock,
# Needed for vfio
capability sys_resource,
mount options=(rw,rslave) -> /,
mount options=(rw, nosuid) -> /{var/,}run/libvirt/qemu/*.dev/,
mount options=(rw, move) /dev/ -> /{var/,}run/libvirt/qemu/*.dev/,
mount options=(rw, move) /dev/hugepages/ -> /{var/,}run/libvirt/qemu/*.hugepages/,
mount options=(rw, move) /dev/mqueue/ -> /{var/,}run/libvirt/qemu/*.mqueue/,
mount options=(rw, move) /dev/pts/ -> /{var/,}run/libvirt/qemu/*.pts/,
mount options=(rw, move) /dev/shm/ -> /{var/,}run/libvirt/qemu/*.shm/,
mount options=(rw, move) /{var/,}run/libvirt/qemu/*.dev/ -> /dev/,
mount options=(rw, move) /{var/,}run/libvirt/qemu/*.hugepages/ -> /dev/hugepages/,
mount options=(rw, move) /{var/,}run/libvirt/qemu/*.mqueue/ -> /dev/mqueue/,
mount options=(rw, move) /{var/,}run/libvirt/qemu/*.pts/ -> /dev/pts/,
mount options=(rw, move) /{var/,}run/libvirt/qemu/*.shm/ -> /dev/shm/,
network inet stream,
network inet dgram,
network inet6 stream,
network inet6 dgram,
network netlink raw,
network packet dgram,
network packet raw,
# for --p2p migrations
unix (send, receive) type=stream addr=none peer=(label=unconfined addr=none),
ptrace (trace) peer=unconfined,
ptrace (trace) peer=/usr/sbin/libvirtd,
ptrace (trace) peer=/usr/sbin/dnsmasq,
ptrace (trace) peer=libvirt-*,
signal (send) peer=/usr/sbin/dnsmasq,
signal (read, send) peer=libvirt-*,
signal (send) set=("kill", "term") peer=unconfined,
# For communication/control to qemu-bridge-helper
unix (send, receive) type=stream addr=none peer=(label=/usr/sbin/libvirtd//qemu_bridge_helper),
signal (send) set=("term") peer=/usr/sbin/libvirtd//qemu_bridge_helper,
# Very lenient profile for libvirtd since we want to first focus on confining
# the guests. Guests will have a very restricted profile.
/ r,
/** rwmkl,
/bin/* PUx,
/sbin/* PUx,
/usr/bin/* PUx,
/usr/sbin/virtlogd pix,
/usr/sbin/* PUx,
/{usr/,}lib/udev/scsi_id PUx,
/usr/{lib,lib64}/xen-common/bin/xen-toolstack PUx,
/usr/{lib,lib64}/xen/bin/* Ux,
/usr/lib/xen-*/bin/libxl-save-helper PUx,
# Required by nwfilter_ebiptables_driver.c:ebiptablesWriteToTempFile() to
# read and run an ebtables script.
/var/lib/libvirt/virtd* ixr,
# force the use of virt-aa-helper
audit deny /{usr/,}sbin/apparmor_parser rwxl,
audit deny /etc/apparmor.d/libvirt/** wxl,
audit deny /sys/kernel/security/apparmor/features rwxl,
audit deny /sys/kernel/security/apparmor/matching rwxl,
audit deny /sys/kernel/security/apparmor/.* rwxl,
/sys/kernel/security/apparmor/profiles r,
/usr/{lib,lib64}/libvirt/* PUxr,
/usr/{lib,lib64}/libvirt/libvirt_parthelper ix,
/usr/{lib,lib64}/libvirt/libvirt_iohelper ix,
/etc/libvirt/hooks/** rmix,
/etc/xen/scripts/** rmix,
# allow changing to our UUID-based named profiles
change_profile -> @{LIBVIRT}-[0-9a-f]*-[0-9a-f]*-[0-9a-f]*-[0-9a-f]*-[0-9a-f]*,
/usr/{lib,lib64,lib/qemu,libexec}/qemu-bridge-helper Cx -> qemu_bridge_helper,
# child profile for bridge helper process
profile qemu_bridge_helper {
#include <abstractions/base>
capability setuid,
capability setgid,
capability setpcap,
capability net_admin,
network inet stream,
# For communication/control from libvirtd
unix (send, receive) type=stream addr=none peer=(label=/usr/sbin/libvirtd),
signal (receive) set=("term") peer=/usr/sbin/libvirtd,
/dev/net/tun rw,
/etc/qemu/** r,
owner @{PROC}/*/status r,
/usr/{lib,lib64,lib/qemu,libexec}/qemu-bridge-helper rmix,
}
}
EOF
#NOTE: Deploy command
: ${OSH_EXTRA_HELM_ARGS_LIBVIRT:="$(./tools/deployment/common/get-values-overrides.sh libvirt)"}
helm upgrade --install libvirt ./libvirt \
--namespace=openstack \
--values=/tmp/libvirt.yaml \
--set network.backend="null" \
${OSH_EXTRA_HELM_ARGS} \
${OSH_EXTRA_HELM_ARGS_LIBVIRT}
#NOTE: Validate Deployment info
./tools/deployment/common/wait-for-pods.sh openstack

30
tools/deployment/apparmor/050-prometheus-alertmanager.sh
View File

@ -1,30 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make prometheus-alertmanager
: ${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS_ALERTMANAGER:="$(./tools/deployment/common/get-values-overrides.sh prometheus-alertmanager)"}
#NOTE: Deploy command
helm upgrade --install prometheus-alertmanager ./prometheus-alertmanager \
--namespace=osh-infra \
--set pod.replicas.alertmanager=1 \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS_ALERTMANAGER}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra

1
tools/deployment/apparmor/055-prometheus.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/050-prometheus.sh

1
tools/deployment/apparmor/060-prometheus-node-exporter.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/080-node-exporter.sh

30
tools/deployment/apparmor/065-prometheus-openstack-exporter.sh
View File

@ -1,30 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make prometheus-openstack-exporter
: ${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS_OPENSTACK_EXPORTER:="$(./tools/deployment/common/get-values-overrides.sh prometheus-openstack-exporter)"}
#NOTE: Deploy command
helm upgrade --install prometheus-openstack-exporter \
./prometheus-openstack-exporter \
--namespace=openstack \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS_OPENSTACK_EXPORTER}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack

30
tools/deployment/apparmor/070-prometheus-blackbox-exporter.sh
View File

@ -1,30 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make prometheus-blackbox-exporter
: ${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS_BLACKBOX_EXPORTER:="$(./tools/deployment/common/get-values-overrides.sh prometheus-blackbox-exporter)"}
#NOTE: Deploy command
helm upgrade --install prometheus-blackbox-exporter \
./prometheus-blackbox-exporter \
--namespace=openstack \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS_BLACKBOX_EXPORTER}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack

1
tools/deployment/apparmor/075-prometheus-process-exporter.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/090-process-exporter.sh

1
tools/deployment/apparmor/080-grafana.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/110-grafana.sh

30
tools/deployment/apparmor/085-rabbitmq.sh
View File

@ -1,30 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make rabbitmq
: ${OSH_INFRA_EXTRA_HELM_ARGS_RABBITMQ:="$(./tools/deployment/common/get-values-overrides.sh rabbitmq)"}
#NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
helm upgrade --install rabbitmq ./rabbitmq \
--namespace=osh-infra \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_RABBITMQ}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra

79
tools/deployment/apparmor/090-elasticsearch.sh
View File

@ -1,79 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make elasticsearch
#NOTE: Deploy command
tee /tmp/elasticsearch.yaml << EOF
dependencies:
static:
tests:
jobs: null
storage:
data:
enabled: false
master:
enabled: false
pod:
mandatory_access_control:
type: apparmor
elasticsearch-master:
elasticsearch-master: runtime/default
elasticsearch-data:
elasticsearch-data: runtime/default
elasticsearch-client:
elasticsearch-client: runtime/default
replicas:
client: 1
data: 1
master: 2
conf:
curator:
schedule: "0 */6 * * *"
action_file:
actions:
1:
action: delete_indices
description: >-
"Delete indices older than 365 days"
options:
timeout_override:
continue_if_exception: False
ignore_empty_list: True
disable_action: True
filters:
- filtertype: pattern
kind: prefix
value: logstash-
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 365
EOF
helm upgrade --install elasticsearch ./elasticsearch \
--namespace=osh-infra \
--values=/tmp/elasticsearch.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
# Delete the test pod if it still exists
kubectl delete pods -l application=elasticsearch,release_group=elasticsearch,component=test --namespace=osh-infra --ignore-not-found
helm test elasticsearch --namespace osh-infra

1
tools/deployment/apparmor/095-nagios.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/120-nagios.sh

37
tools/deployment/apparmor/100-fluentbit.sh
View File

@ -1,37 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make fluentbit
tee /tmp/fluentbit.yaml <<EOF
pod:
mandatory_access_control:
type: apparmor
fluentbit:
fluentbit: runtime/default
EOF
#NOTE: Deploy command
helm upgrade --install fluentbit ./fluentbit \
--namespace=osh-infra \
--values=/tmp/fluentbit.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
# Delete the test pod if it still exists
kubectl delete pods -l application=fluentbit,release_group=fluentbit,component=test --namespace=osh-infra --ignore-not-found
helm test fluentbit --namespace osh-infra

172
tools/deployment/apparmor/110-fluentd-daemonset.sh
View File

@ -1,172 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make fluentd
tee /tmp/fluentd-daemonset.yaml <<EOF
deployment:
type: DaemonSet
pod:
security_context:
fluentd:
pod:
runAsUser: 0
mandatory_access_control:
type: apparmor
fluentd:
fluentd: runtime/default
conf:
fluentd:
template: |
<source>
bind 0.0.0.0
port 24220
@type monitor_agent
</source>
<source>
<parse>
time_format %Y-%m-%dT%H:%M:%S.%NZ
@type json
</parse>
path /var/log/containers/*.log
read_from_head true
tag kubernetes.*
@type tail
</source>
<filter kubernetes.**>
@type kubernetes_metadata
</filter>
<source>
bind 0.0.0.0
port "#{ENV['FLUENTD_PORT']}"
@type forward
</source>
<match fluent.**>
@type null
</match>
<match libvirt>
<buffer>
chunk_limit_size 500K
flush_interval 5s
flush_thread_count 8
queue_limit_length 16
retry_forever false
retry_max_interval 30
</buffer>
host "#{ENV['ELASTICSEARCH_HOST']}"
include_tag_key true
logstash_format true
logstash_prefix libvirt
password "#{ENV['ELASTICSEARCH_PASSWORD']}"
port "#{ENV['ELASTICSEARCH_PORT']}"
@type elasticsearch
user "#{ENV['ELASTICSEARCH_USERNAME']}"
</match>
<match qemu>
<buffer>
chunk_limit_size 500K
flush_interval 5s
flush_thread_count 8
queue_limit_length 16
retry_forever false
retry_max_interval 30
</buffer>
host "#{ENV['ELASTICSEARCH_HOST']}"
include_tag_key true
logstash_format true
logstash_prefix qemu
password "#{ENV['ELASTICSEARCH_PASSWORD']}"
port "#{ENV['ELASTICSEARCH_PORT']}"
@type elasticsearch
user "#{ENV['ELASTICSEARCH_USERNAME']}"
</match>
<match journal.**>
<buffer>
chunk_limit_size 500K
flush_interval 5s
flush_thread_count 8
queue_limit_length 16
retry_forever false
retry_max_interval 30
</buffer>
host "#{ENV['ELASTICSEARCH_HOST']}"
include_tag_key true
logstash_format true
logstash_prefix journal
password "#{ENV['ELASTICSEARCH_PASSWORD']}"
port "#{ENV['ELASTICSEARCH_PORT']}"
@type elasticsearch
user "#{ENV['ELASTICSEARCH_USERNAME']}"
</match>
<match kernel>
<buffer>
chunk_limit_size 500K
flush_interval 5s
flush_thread_count 8
queue_limit_length 16
retry_forever false
retry_max_interval 30
</buffer>
host "#{ENV['ELASTICSEARCH_HOST']}"
include_tag_key true
logstash_format true
logstash_prefix kernel
password "#{ENV['ELASTICSEARCH_PASSWORD']}"
port "#{ENV['ELASTICSEARCH_PORT']}"
@type elasticsearch
user "#{ENV['ELASTICSEARCH_USERNAME']}"
</match>
<match **>
<buffer>
chunk_limit_size 500K
flush_interval 5s
flush_thread_count 8
queue_limit_length 16
retry_forever false
retry_max_interval 30
</buffer>
flush_interval 15s
host "#{ENV['ELASTICSEARCH_HOST']}"
include_tag_key true
logstash_format true
password "#{ENV['ELASTICSEARCH_PASSWORD']}"
port "#{ENV['ELASTICSEARCH_PORT']}"
@type elasticsearch
type_name fluent
user "#{ENV['ELASTICSEARCH_USERNAME']}"
</match>
EOF
#NOTE: Deploy command
helm upgrade --install fluentd-daemonset ./fluentd \
--namespace=osh-infra \
--values=/tmp/fluentd-daemonset.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
# Delete the test pod if it still exists
kubectl delete pods -l application=fluentd,release_group=fluentd-daemonset,component=test --namespace=osh-infra --ignore-not-found
helm test fluentd-daemonset --namespace osh-infra

1
tools/deployment/apparmor/115-node-problem-detector.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/075-node-problem-detector.sh

1
tools/deployment/apparmor/120-openvswitch.sh
View File

@ -1 +0,0 @@
../openstack-support/060-openvswitch.sh

65
tools/deployment/apparmor/140-ceph-radosgateway.sh
View File

@ -1,65 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
: ${OSH_INFRA_EXTRA_HELM_ARGS_CEPH_RGW:="$(./tools/deployment/common/get-values-overrides.sh ceph-rgw)"}
#NOTE: Lint and package chart
: ${OSH_INFRA_PATH:="../openstack-helm-infra"}
make -C ${OSH_INFRA_PATH} ceph-rgw
#NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
tee /tmp/radosgw-openstack.yaml <<EOF
endpoints:
identity:
namespace: openstack
object_store:
namespace: openstack
ceph_mon:
namespace: ceph
network:
public: 172.17.0.1/16
cluster: 172.17.0.1/16
deployment:
ceph: true
rgw_keystone_user_and_endpoints: true
bootstrap:
enabled: false
conf:
rgw_ks:
enabled: true
pod:
replicas:
rgw: 1
EOF
helm upgrade --install radosgw-openstack ${OSH_INFRA_PATH}/ceph-rgw \
--namespace=openstack \
--values=/tmp/radosgw-openstack.yaml \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_CEPH_RGW}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack
#NOTE: Validate Deployment info
export OS_CLOUD=openstack_helm
sleep 60 #NOTE(portdirect): Wait for ingress controller to update rules and restart Nginx
openstack service list
openstack endpoint list
# Delete the test pod if it still exists
kubectl delete pods -l application=ceph,release_group=radosgw-openstack,component=rgw-test --namespace=openstack --ignore-not-found
helm test radosgw-openstack --namespace openstack --timeout 900s

1
tools/deployment/apparmor/170-postgresql.sh
View File

@ -1 +0,0 @@
../common/postgresql.sh

7
tools/deployment/openstack-support-rook/025-ceph-ns-activate.sh → tools/deployment/ceph/ceph-ns-activate.sh
View File

@ -14,9 +14,6 @@
set -xe set -xe
#NOTE: Lint and package chart
make ceph-provisioners
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_EXTRA_HELM_ARGS:=""} : ${OSH_EXTRA_HELM_ARGS:=""}
tee /tmp/ceph-openstack-config.yaml <<EOF tee /tmp/ceph-openstack-config.yaml <<EOF
@ -39,7 +36,7 @@ conf:
enabled: false enabled: false
EOF EOF
: ${OSH_INFRA_EXTRA_HELM_ARGS_CEPH_NS_ACTIVATE:="$(./tools/deployment/common/get-values-overrides.sh ceph-provisioners)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_CEPH_NS_ACTIVATE:="$(helm osh get-values-overrides -c ceph-provisioners ${FEATURES})"}
helm upgrade --install ceph-openstack-config ./ceph-provisioners \ helm upgrade --install ceph-openstack-config ./ceph-provisioners \
--namespace=openstack \ --namespace=openstack \
@ -48,7 +45,7 @@ helm upgrade --install ceph-openstack-config ./ceph-provisioners \
${OSH_INFRA_EXTRA_HELM_ARGS_CEPH_NS_ACTIVATE} ${OSH_INFRA_EXTRA_HELM_ARGS_CEPH_NS_ACTIVATE}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack helm osh wait-for-pods openstack
helm test ceph-openstack-config --namespace openstack --timeout 600s helm test ceph-openstack-config --namespace openstack --timeout 600s

10
tools/deployment/osh-infra-logging-tls/030-radosgw-osh-infra.sh → tools/deployment/ceph/ceph-radosgw.sh
View File

@ -14,13 +14,9 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_EXTRA_HELM_ARGS_CEPH_RGW:="$(helm osh get-values-overrides -c ceph-rgw ${FEATURES})"}
make ceph-rgw
: ${OSH_EXTRA_HELM_ARGS_CEPH_RGW:="$(./tools/deployment/common/get-values-overrides.sh ceph-rgw)"}
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_EXTRA_HELM_ARGS:=""}
tee /tmp/radosgw-osh-infra.yaml <<EOF tee /tmp/radosgw-osh-infra.yaml <<EOF
endpoints: endpoints:
ceph_object_store: ceph_object_store:
@ -53,11 +49,11 @@ EOF
helm upgrade --install radosgw-osh-infra ./ceph-rgw \ helm upgrade --install radosgw-osh-infra ./ceph-rgw \
--namespace=osh-infra \ --namespace=osh-infra \
--values=/tmp/radosgw-osh-infra.yaml \ --values=/tmp/radosgw-osh-infra.yaml \
${OSH_EXTRA_HELM_ARGS} \ ${OSH_EXTRA_HELM_ARGS:=} \
${OSH_EXTRA_HELM_ARGS_CEPH_RGW} ${OSH_EXTRA_HELM_ARGS_CEPH_RGW}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra
# Delete the test pod if it still exists # Delete the test pod if it still exists
kubectl delete pods -l application=ceph,release_group=radosgw-osh-infra,component=rgw-test --namespace=osh-infra --ignore-not-found kubectl delete pods -l application=ceph,release_group=radosgw-osh-infra,component=rgw-test --namespace=osh-infra --ignore-not-found

27
tools/deployment/common/000-install-packages.sh
View File

@ -1,27 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
sudo apt-get update
sudo apt-get install --no-install-recommends -y \
ca-certificates \
git \
make \
nmap \
curl \
bc \
python3-pip \
dnsutils \
lvm2

20
tools/deployment/common/001-setup-apparmor-profiles.sh
View File

@ -1,20 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
# Ensure that apparmor is installed and enabled
sudo -H -E apt-get install -y apparmor
sudo systemctl enable apparmor && sudo systemctl start apparmor
sudo systemctl status apparmor.service

32
tools/deployment/common/030-nfs-provisioner.sh
View File

@ -1,32 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
make nfs-provisioner
#NOTE: Deploy nfs instance for logging, monitoring and alerting components
tee /tmp/nfs-provisioner.yaml << EOF
labels:
node_selector_key: openstack-control-plane
node_selector_value: enabled
storageclass:
name: general
EOF
helm upgrade --install nfs-provisioner \
./nfs-provisioner --namespace=nfs \
--values=/tmp/nfs-provisioner.yaml
#NOTE: Wait for deployment
./tools/deployment/common/wait-for-pods.sh nfs

0
tools/deployment/common/015-cert-manager.sh → tools/deployment/common/cert-manager.sh
View File

7
tools/deployment/common/daemonjob-controller.sh
View File

@ -14,10 +14,7 @@
set -xe set -xe
namespace="metacontroller" namespace="metacontroller"
: ${HELM_ARGS_DAEMONJOB_CONTROLLER:="$(./tools/deployment/common/get-values-overrides.sh daemonjob-controller)"} : ${HELM_ARGS_DAEMONJOB_CONTROLLER:="$(helm osh get-values-overrides -c daemonjob-controller ${FEATURES})"}
#NOTE: Lint and package chart
make daemonjob-controller
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install daemonjob-controller ./daemonjob-controller \ helm upgrade --install daemonjob-controller ./daemonjob-controller \
@ -26,7 +23,7 @@ helm upgrade --install daemonjob-controller ./daemonjob-controller \
${HELM_ARGS_DAEMONJOB_CONTROLLER} ${HELM_ARGS_DAEMONJOB_CONTROLLER}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh daemonjob-controller helm osh wait-for-pods daemonjob-controller
#NOTE: CompositeController succesfully deployed #NOTE: CompositeController succesfully deployed
composite_controller_cr=$(kubectl get compositecontrollers | awk '{print $1}') composite_controller_cr=$(kubectl get compositecontrollers | awk '{print $1}')

7
tools/deployment/common/010-deploy-docker-registry.sh → tools/deployment/common/deploy-docker-registry.sh
View File

@ -14,11 +14,6 @@
set -xe set -xe
#NOTE: Lint and package charts for deploying a local docker registry
make nfs-provisioner
make redis
make registry
for NAMESPACE in docker-nfs docker-registry; do for NAMESPACE in docker-nfs docker-registry; do
tee /tmp/${NAMESPACE}-ns.yaml << EOF tee /tmp/${NAMESPACE}-ns.yaml << EOF
apiVersion: v1 apiVersion: v1
@ -64,7 +59,7 @@ helm upgrade --install docker-registry ./registry \
--values=/tmp/docker-registry.yaml --values=/tmp/docker-registry.yaml
#NOTE: Wait for deployments #NOTE: Wait for deployments
./tools/deployment/common/wait-for-pods.sh docker-registry helm osh wait-for-pods docker-registry
# Delete the test pod if it still exists # Delete the test pod if it still exists
kubectl delete pods -l application=redis,release_group=docker-registry-redis,component=test --namespace=docker-registry --ignore-not-found kubectl delete pods -l application=redis,release_group=docker-registry-redis,component=test --namespace=docker-registry --ignore-not-found

5
tools/deployment/common/150-falco.sh → tools/deployment/common/falco.sh
View File

@ -14,12 +14,9 @@
set -xe set -xe
#NOTE: Lint and package chart
make falco
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install falco ./falco \ helm upgrade --install falco ./falco \
--namespace=kube-system --namespace=kube-system
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system helm osh wait-for-pods kube-system

8
tools/deployment/common/ingress.sh
View File

@ -34,7 +34,7 @@ helm upgrade --install ingress-nginx-cluster ingress-nginx/ingress-nginx \
--set controller.labels.app=ingress-api --set controller.labels.app=ingress-api
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system helm osh wait-for-pods kube-system
#NOTE: Deploy namespace ingress #NOTE: Deploy namespace ingress
helm upgrade --install ingress-nginx-openstack ingress-nginx/ingress-nginx \ helm upgrade --install ingress-nginx-openstack ingress-nginx/ingress-nginx \
@ -49,7 +49,7 @@ helm upgrade --install ingress-nginx-openstack ingress-nginx/ingress-nginx \
--set controller.labels.app=ingress-api --set controller.labels.app=ingress-api
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack helm osh wait-for-pods openstack
helm upgrade --install ingress-nginx-ceph ingress-nginx/ingress-nginx \ helm upgrade --install ingress-nginx-ceph ingress-nginx/ingress-nginx \
--version ${HELM_INGRESS_NGINX_VERSION} \ --version ${HELM_INGRESS_NGINX_VERSION} \
@ -63,7 +63,7 @@ helm upgrade --install ingress-nginx-ceph ingress-nginx/ingress-nginx \
--set controller.labels.app=ingress-api --set controller.labels.app=ingress-api
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh ceph helm osh wait-for-pods ceph
helm upgrade --install ingress-nginx-osh-infra ingress-nginx/ingress-nginx \ helm upgrade --install ingress-nginx-osh-infra ingress-nginx/ingress-nginx \
--version ${HELM_INGRESS_NGINX_VERSION} \ --version ${HELM_INGRESS_NGINX_VERSION} \
@ -77,4 +77,4 @@ helm upgrade --install ingress-nginx-osh-infra ingress-nginx/ingress-nginx \
--set controller.labels.app=ingress-api --set controller.labels.app=ingress-api
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra

11
tools/deployment/common/040-ldap.sh → tools/deployment/common/ldap.sh
View File

@ -14,16 +14,15 @@
set -xe set -xe
: ${OSH_INFRA_EXTRA_HELM_ARGS_LDAP:="$(./tools/deployment/common/get-values-overrides.sh ldap)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_LDAP:="$(helm osh get-values-overrides -c ldap ${FEATURES})"}
: ${NAMESPACE:="osh-infra"}
#NOTE: Pull images and lint chart
make ldap
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install ldap ./ldap \ helm upgrade --install ldap ./ldap \
--namespace=osh-infra \ --namespace=${NAMESPACE} \
--set bootstrap.enabled=true \ --set bootstrap.enabled=true \
${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_LDAP} ${OSH_INFRA_EXTRA_HELM_ARGS_LDAP}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods ${NAMESPACE}

0
tools/deployment/openstack-support-rook/040-memcached.sh → tools/deployment/common/memcached.sh
View File

7
tools/deployment/common/metacontroller.sh
View File

@ -14,10 +14,7 @@
set -xe set -xe
namespace="metacontroller" namespace="metacontroller"
: ${HELM_ARGS_METACONTROLLER:="$(./tools/deployment/common/get-values-overrides.sh metacontroller)"} : ${HELM_ARGS_METACONTROLLER:="$(helm osh get-values-overrides -c metacontroller ${FEATURES})"}
#NOTE: Lint and package chart
make metacontroller
#NOTE: Check no crd exists of APIGroup metacontroller.k8s.io #NOTE: Check no crd exists of APIGroup metacontroller.k8s.io
crds=$(kubectl get crd | awk '/metacontroller.k8s.io/{print $1}') crds=$(kubectl get crd | awk '/metacontroller.k8s.io/{print $1}')
@ -45,7 +42,7 @@ helm upgrade --install metacontroller ./metacontroller \
${HELM_ARGS_METACONTROLLER} ${HELM_ARGS_METACONTROLLER}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh metacontroller helm osh wait-for-pods metacontroller
#NOTE: Check crds of APIGroup metacontroller.k8s.io successfully created #NOTE: Check crds of APIGroup metacontroller.k8s.io successfully created
crds=$(kubectl get crd | awk '/metacontroller.k8s.io/{print $1}') crds=$(kubectl get crd | awk '/metacontroller.k8s.io/{print $1}')

40
tools/deployment/common/nagios.sh
View File

@ -1,40 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make nagios
#NOTE: Deploy command
tee /tmp/nagios.yaml << EOF
conf:
nagios:
query_es_clauses:
test_es_query:
hello: world
EOF
helm upgrade --install nagios ./nagios \
--namespace=osh-infra \
--values=/tmp/nagios.yaml \
--values=nagios/values_overrides/openstack-objects.yaml \
--values=nagios/values_overrides/postgresql-objects.yaml \
--values=nagios/values_overrides/elasticsearch-objects.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
#NOTE: Verify elasticsearch query clauses are functional by execing into pod
NAGIOS_POD=$(kubectl -n osh-infra get pods -l='application=nagios,component=monitoring' --output=jsonpath='{.items[0].metadata.name}')
kubectl exec $NAGIOS_POD -n osh-infra -c nagios -- cat /opt/nagios/etc/objects/query_es_clauses.json | python -m json.tool

3
tools/deployment/openstack-support-rook/007-namespace-config.sh → tools/deployment/common/namespace-config.sh
View File

@ -14,9 +14,6 @@
set -xe set -xe
#NOTE: Lint and package chart
make namespace-config
#NOTE: Deploy namespace configs #NOTE: Deploy namespace configs
for NAMESPACE in kube-system ceph openstack; do for NAMESPACE in kube-system ceph openstack; do
helm upgrade --install ${NAMESPACE}-namespace-config ./namespace-config \ helm upgrade --install ${NAMESPACE}-namespace-config ./namespace-config \

4
tools/deployment/osh-infra-monitoring-tls/030-nfs-provisioner.sh → tools/deployment/common/nfs-provisioner.sh
View File

@ -14,8 +14,6 @@
set -xe set -xe
make nfs-provisioner
tee /tmp/nfs-ns.yaml << EOF tee /tmp/nfs-ns.yaml << EOF
apiVersion: v1 apiVersion: v1
kind: Namespace kind: Namespace
@ -41,4 +39,4 @@ helm upgrade --install nfs-provisioner \
--values=/tmp/nfs-provisioner.yaml --values=/tmp/nfs-provisioner.yaml
#NOTE: Wait for deployment #NOTE: Wait for deployment
./tools/deployment/common/wait-for-pods.sh nfs helm osh wait-for-pods nfs

23
tools/deployment/common/prepare-charts.sh Executable file
View File

@ -0,0 +1,23 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
set -ex
# Build all OSH charts
make all
# Build all OSH charts (necessary for Openstack deployment)
(
cd ${OSH_PATH:-"../openstack-helm"} &&
make all
)

2
tools/deployment/common/prepare-k8s.sh
View File

@ -44,5 +44,3 @@ EOF
kubectl apply -f /tmp/${NAMESPACE}-ns.yaml kubectl apply -f /tmp/${NAMESPACE}-ns.yaml
done done
make all

0
tools/deployment/openstack-support-rook/030-rabbitmq.sh → tools/deployment/common/rabbitmq.sh
View File

89
tools/deployment/common/setup-ceph-loopback-device.sh
View File

@ -1,89 +0,0 @@
#!/bin/bash
set -ex
: ${CEPH_LOOPBACK_PATH:="/var/lib/openstack-helm"}
function setup_loopback_devices() {
osd_data_device="$1"
osd_wal_db_device="$2"
namespace=${CEPH_NAMESPACE}
sudo mkdir -p ${CEPH_LOOPBACK_PATH}/$namespace
sudo truncate -s 10G ${CEPH_LOOPBACK_PATH}/$namespace/ceph-osd-data-loopbackfile.img
sudo truncate -s 8G ${CEPH_LOOPBACK_PATH}/$namespace/ceph-osd-db-wal-loopbackfile.img
sudo -E bash -c "cat <<EOF > /etc/systemd/system/loops-setup.service
[Unit]
Description=Setup loop devices
DefaultDependencies=no
Conflicts=umount.target
Before=local-fs.target
After=systemd-udevd.service
Requires=systemd-udevd.service
[Service]
Type=oneshot
ExecStart=/sbin/losetup $osd_data_device '${CEPH_LOOPBACK_PATH}/$namespace/ceph-osd-data-loopbackfile.img'
ExecStart=/sbin/losetup $osd_wal_db_device '${CEPH_LOOPBACK_PATH}/$namespace/ceph-osd-db-wal-loopbackfile.img'
ExecStop=/sbin/losetup -d $osd_data_device
ExecStop=/sbin/losetup -d $osd_wal_db_device
TimeoutSec=60
RemainAfterExit=yes
[Install]
WantedBy=local-fs.target
Also=systemd-udevd.service
EOF"
sudo systemctl daemon-reload
sudo systemctl start loops-setup
sudo systemctl status loops-setup
sudo systemctl enable loops-setup
# let's verify the devices
sudo losetup -a
if losetup |grep -i $osd_data_device; then
echo "ceph osd data disk got created successfully"
else
echo "could not find ceph osd data disk so exiting"
exit 1
fi
if losetup |grep -i $osd_wal_db_device; then
echo "ceph osd wal/db disk got created successfully"
else
echo "could not find ceph osd wal/db disk so exiting"
exit 1
fi
}
while [[ "$#" > 0 ]]; do case $1 in
-d|--ceph-osd-data) OSD_DATA_DEVICE="$2"; shift;shift;;
-w|--ceph-osd-dbwal) OSD_DB_WAL_DEVICE="$2";shift;shift;;
-v|--verbose) VERBOSE=1;shift;;
*) echo "Unknown parameter passed: $1"; shift;;
esac; done
# verify params
if [ -z "$OSD_DATA_DEVICE" ]; then
OSD_DATA_DEVICE=/dev/loop0
echo "Ceph osd data device is not set so using ${OSD_DATA_DEVICE}"
else
ceph_osd_disk_name=`basename "$OSD_DATA_DEVICE"`
if losetup -a|grep $ceph_osd_disk_name; then
echo "Ceph osd data device is already in use, please double check and correct the device name"
exit 1
fi
fi
if [ -z "$OSD_DB_WAL_DEVICE" ]; then
OSD_DB_WAL_DEVICE=/dev/loop1
echo "Ceph osd db/wal device is not set so using ${OSD_DB_WAL_DEVICE}"
else
ceph_dbwal_disk_name=`basename "$OSD_DB_WAL_DEVICE"`
if losetup -a|grep $ceph_dbwal_disk_name; then
echo "Ceph osd dbwal device is already in use, please double check and correct the device name"
exit 1
fi
fi
: "${CEPH_NAMESPACE:="ceph"}"
# setup loopback devices for ceph osds
setup_loopback_devices $OSD_DATA_DEVICE $OSD_DB_WAL_DEVICE

15
tools/deployment/mariadb-operator-cluster/090-mariadb-backup-test.sh → tools/deployment/db/mariadb-backup.sh
View File

@ -14,27 +14,20 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_BACKUP:="$(helm osh get-values-overrides -c mariadb-backup ${FEATURES})"}
make mariadb-backup
: ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_BACKUP:="$(./tools/deployment/common/get-values-overrides.sh mariadb-backup)"}
#NOTE: Deploy command #NOTE: Deploy command
# Deploying downscaled cluster
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
helm upgrade --install mariadb-backup ./mariadb-backup \ helm upgrade --install mariadb-backup ./mariadb-backup \
--namespace=openstack \ --namespace=openstack \
--wait \ --wait \
--timeout 900s \ --timeout 900s \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_BACKUP} ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_BACKUP}
helm osh wait-for-pods openstack
./tools/deployment/common/wait-for-pods.sh openstack
kubectl create job --from=cronjob/mariadb-backup mariadb-backup-manual-001 -n openstack kubectl create job --from=cronjob/mariadb-backup mariadb-backup-manual-001 -n openstack
./tools/deployment/common/wait-for-pods.sh openstack helm osh wait-for-pods openstack
kubectl logs jobs/mariadb-backup-manual-001 -n openstack kubectl logs jobs/mariadb-backup-manual-001 -n openstack

15
tools/deployment/mariadb-operator-cluster/045-mariadb-operator-cluster.sh → tools/deployment/db/mariadb-operator-cluster.sh
View File

@ -19,16 +19,12 @@ set -xe
# install mariadb-operator # install mariadb-operator
helm repo add mariadb-operator https://mariadb-operator.github.io/mariadb-operator helm repo add mariadb-operator https://mariadb-operator.github.io/mariadb-operator
helm install mariadb-operator mariadb-operator/mariadb-operator --version ${MARIADB_OPERATOR_RELEASE} -n mariadb-operator helm upgrade --install mariadb-operator mariadb-operator/mariadb-operator --version ${MARIADB_OPERATOR_RELEASE} -n mariadb-operator
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh mariadb-operator helm osh wait-for-pods mariadb-operator
: ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_CLUSTER:="$(helm osh get-values-overrides -c mariadb-cluster ${FEATURES})"}
#NOTE: Lint and package chart
make mariadb-cluster
: ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_CLUSTER:="$(./tools/deployment/common/get-values-overrides.sh mariadb-cluster)"}
#NOTE: Deploy command #NOTE: Deploy command
# Deploying downscaled cluster # Deploying downscaled cluster
@ -41,9 +37,10 @@ helm upgrade --install mariadb-cluster ./mariadb-cluster \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_CLUSTER} ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_CLUSTER}
sleep 30
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack helm osh wait-for-pods openstack
kubectl get pods --namespace=openstack -o wide kubectl get pods --namespace=openstack -o wide
@ -61,7 +58,7 @@ helm upgrade --install mariadb-cluster ./mariadb-cluster \
${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_CLUSTER} ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_CLUSTER}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack helm osh wait-for-pods openstack
kubectl get pods --namespace=openstack -o wide kubectl get pods --namespace=openstack -o wide

10
tools/deployment/osh-infra-monitoring/045-mariadb.sh → tools/deployment/db/mariadb.sh
View File

@ -14,21 +14,17 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB:="$(helm osh get-values-overrides -c mariadb ${FEATURES})"}
make mariadb
: ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB:="$(./tools/deployment/common/get-values-overrides.sh mariadb)"}
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
helm upgrade --install mariadb ./mariadb \ helm upgrade --install mariadb ./mariadb \
--namespace=osh-infra \ --namespace=osh-infra \
--set monitoring.prometheus.enabled=true \ --set monitoring.prometheus.enabled=true \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB} ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra
# Delete the test pod if it still exists # Delete the test pod if it still exists
kubectl delete pods -l application=mariadb,release_group=mariadb,component=test --namespace=osh-infra --ignore-not-found kubectl delete pods -l application=mariadb,release_group=mariadb,component=test --namespace=osh-infra --ignore-not-found

7
tools/deployment/common/postgresql.sh → tools/deployment/db/postgresql.sh
View File

@ -14,12 +14,9 @@
set -xe set -xe
#NOTE: Lint and package chart
make postgresql
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""} : ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
: ${OSH_INFRA_EXTRA_HELM_ARGS_POSTGRESQL:="$(./tools/deployment/common/get-values-overrides.sh postgresql)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_POSTGRESQL:="$(helm osh get-values-overrides -c postgresql ${FEATURES})"}
helm upgrade --install postgresql ./postgresql \ helm upgrade --install postgresql ./postgresql \
--namespace=osh-infra \ --namespace=osh-infra \
@ -31,4 +28,4 @@ helm upgrade --install postgresql ./postgresql \
${OSH_INFRA_EXTRA_HELM_ARGS_POSTGRESQL} ${OSH_INFRA_EXTRA_HELM_ARGS_POSTGRESQL}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra

1
tools/deployment/elastic-beats/005-deploy-k8s.sh
View File

@ -1 +0,0 @@
../../gate/deploy-k8s.sh

1
tools/deployment/elastic-beats/030-ceph.sh
View File

@ -1 +0,0 @@
../multinode/030-ceph.sh

1
tools/deployment/elastic-beats/035-ceph-ns-activate.sh
View File

@ -1 +0,0 @@
../multinode/035-ceph-ns-activate.sh

1
tools/deployment/elastic-beats/040-ldap.sh
View File

@ -1 +0,0 @@
../common/040-ldap.sh

62
tools/deployment/elastic-beats/050-elasticsearch.sh
View File

@ -1,62 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make elasticsearch
#NOTE: Deploy command
tee /tmp/elasticsearch.yaml << EOF
manifests:
cron_curator: false
configmap_bin_curator: false
configmap_etc_curator: false
images:
tags:
elasticsearch: docker.io/openstackhelm/elasticsearch-s3:7_1_0-20191115
storage:
data:
requests:
storage: 20Gi
master:
requests:
storage: 5Gi
jobs:
verify_repositories:
cron: "*/10 * * * *"
monitoring:
prometheus:
enabled: false
pod:
replicas:
client: 1
data: 1
master: 2
conf:
elasticsearch:
config:
xpack:
security:
enabled: false
ilm:
enabled: false
EOF
helm upgrade --install elasticsearch ./elasticsearch \
--namespace=osh-infra \
--values=/tmp/elasticsearch.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra

55
tools/deployment/elastic-beats/060-kibana.sh
View File

@ -1,55 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make kibana
: ${OSH_INFRA_EXTRA_HELM_ARGS_KIBANA:="$(./tools/deployment/common/get-values-overrides.sh kibana)"}
tee /tmp/kibana.yaml << EOF
images:
tags:
kibana: docker.elastic.co/kibana/kibana:7.1.0
conf:
kibana:
xpack:
security:
enabled: false
spaces:
enabled: false
apm:
enabled: false
graph:
enabled: false
ml:
enabled: false
monitoring:
enabled: false
reporting:
enabled: false
canvas:
enabled: false
EOF
#NOTE: Deploy command
helm upgrade --install kibana ./kibana \
--namespace=osh-infra \
--values=/tmp/kibana.yaml
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_KIBANA}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra

1
tools/deployment/elastic-beats/070-kube-state-metrics.sh
View File

@ -1 +0,0 @@
../common/070-kube-state-metrics.sh

42
tools/deployment/elastic-beats/080-elastic-metricbeat.sh
View File

@ -1,42 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make elastic-metricbeat
tee /tmp/metricbeat.yaml << EOF
images:
tags:
metricbeat: docker.elastic.co/beats/metricbeat:7.1.0
conf:
metricbeat:
setup:
ilm:
enabled: false
endpoints:
elasticsearch:
namespace: osh-infra
kibana:
namespace: osh-infra
EOF
#NOTE: Deploy command
helm upgrade --install elastic-metricbeat ./elastic-metricbeat \
--namespace=kube-system \
--values=/tmp/metricbeat.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system

42
tools/deployment/elastic-beats/090-elastic-filebeat.sh
View File

@ -1,42 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make elastic-filebeat
tee /tmp/filebeat.yaml << EOF
images:
tags:
filebeat: docker.elastic.co/beats/filebeat:7.1.0
conf:
filebeat:
setup:
ilm:
enabled: false
endpoints:
elasticsearch:
namespace: osh-infra
kibana:
namespace: osh-infra
EOF
#NOTE: Deploy command
helm upgrade --install elastic-filebeat ./elastic-filebeat \
--namespace=kube-system \
--values=/tmp/filebeat.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system

42
tools/deployment/elastic-beats/100-elastic-packetbeat.sh
View File

@ -1,42 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make elastic-packetbeat
tee /tmp/packetbeat.yaml << EOF
images:
tags:
filebeat: docker.elastic.co/beats/packetbeat:7.1.0
conf:
packetbeat:
setup:
ilm:
enabled: false
endpoints:
elasticsearch:
namespace: osh-infra
kibana:
namespace: osh-infra
EOF
#NOTE: Deploy command
helm upgrade --install elastic-packetbeat ./elastic-packetbeat \
--namespace=kube-system \
--values=/tmp/packetbeat.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system

1
tools/deployment/federated-monitoring/000-install-packages.sh
View File

@ -1 +0,0 @@
../common/000-install-packages.sh

1
tools/deployment/federated-monitoring/005-deploy-k8s.sh
View File

@ -1 +0,0 @@
../../gate/deploy-k8s.sh

1
tools/deployment/federated-monitoring/020-nfs-provisioner.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/030-nfs-provisioner.sh

1
tools/deployment/federated-monitoring/030-ldap.sh
View File

@ -1 +0,0 @@
../common/040-ldap.sh

1
tools/deployment/federated-monitoring/040-kube-state-metrics.sh
View File

@ -1 +0,0 @@
../common/070-kube-state-metrics.sh

1
tools/deployment/federated-monitoring/050-node-exporter.sh
View File

@ -1 +0,0 @@
../common/080-node-exporter.sh

65
tools/deployment/federated-monitoring/060-prometheus.sh
View File

@ -1,65 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make prometheus
tee /tmp/prometheus-one.yaml << EOF
endpoints:
monitoring:
hosts:
default: prom-metrics-one
public: prometheus-one
manifests:
network_policy: false
EOF
tee /tmp/prometheus-two.yaml << EOF
endpoints:
monitoring:
hosts:
default: prom-metrics-two
public: prometheus-two
manifests:
network_policy: false
EOF
tee /tmp/prometheus-three.yaml << EOF
endpoints:
monitoring:
hosts:
default: prom-metrics-three
public: prometheus-three
manifests:
network_policy: false
EOF
#NOTE: Deploy command
for release in prometheus-one prometheus-two prometheus-three; do
rules_overrides=""
for rules_file in $(ls ./prometheus/values_overrides); do
rules_overrides="$rules_overrides --values=./prometheus/values_overrides/$rules_file"
done
helm upgrade --install prometheus-$release ./prometheus \
--namespace=osh-infra \
--values=/tmp/$release.yaml \
$rules_overrides
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
# Delete the test pod if it still exists
kubectl delete pods -l application=prometheus,release_group=prometheus-$release,component=test --namespace=osh-infra --ignore-not-found
helm test prometheus-$release --namespace osh-infra
done

63
tools/deployment/federated-monitoring/070-federated-prometheus.sh
View File

@ -1,63 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
tee /tmp/federated-prometheus.yaml << EOF
endpoints:
monitoring:
hosts:
default: prom-metrics-federate
public: prometheus-federate
manifests:
network_policy: false
conf:
prometheus:
scrape_configs:
template: |
global:
scrape_interval: 60s
evaluation_interval: 60s
scrape_configs:
- job_name: 'federate'
scrape_interval: 15s
honor_labels: true
metrics_path: '/federate'
params:
'match[]':
- '{__name__=~".+"}'
static_configs:
- targets:
- 'prometheus-one.osh-infra.svc.cluster.local:80'
- 'prometheus-two.osh-infra.svc.cluster.local:80'
- 'prometheus-three.osh-infra.svc.cluster.local:80'
EOF
#NOTE: Lint and package chart
make prometheus
#NOTE: Deploy command
helm upgrade --install federated-prometheus ./prometheus \
--namespace=osh-infra \
--values=/tmp/federated-prometheus.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
# Delete the test pod if it still exists
kubectl delete pods -l application=prometheus,release_group=federated-prometheus,component=test --namespace=osh-infra --ignore-not-found
helm test federated-prometheus --namespace osh-infra

1
tools/deployment/federated-monitoring/080-mariadb.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/045-mariadb.sh

165
tools/deployment/federated-monitoring/090-grafana.sh
View File

@ -1,165 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Lint and package chart
make grafana
tee /tmp/grafana.yaml << EOF
endpoints:
monitoring_one:
name: prometheus-one
namespace: osh-infra
auth:
user:
username: admin
password: changeme
hosts:
default: prom-metrics-one
public: prometheus-one
host_fqdn_override:
default: null
path:
default: null
scheme:
default: http
port:
api:
default: 80
public: 80
monitoring_two:
name: prometheus-two
namespace: osh-infra
auth:
user:
username: admin
password: changeme
hosts:
default: prom-metrics-two
public: prometheus-two
host_fqdn_override:
default: null
path:
default: null
scheme:
default: http
port:
api:
default: 80
public: 80
monitoring_three:
name: prometheus-three
namespace: osh-infra
auth:
user:
username: admin
password: changeme
hosts:
default: prom-metrics-three
public: prometheus-three
host_fqdn_override:
default: null
path:
default: null
scheme:
default: http
port:
api:
default: 80
public: 80
monitoring_federated:
name: prometheus-federate
namespace: osh-infra
auth:
user:
username: admin
password: changeme
hosts:
default: prom-metrics-federate
public: prometheus-federate
host_fqdn_override:
default: null
path:
default: null
scheme:
default: http
port:
api:
default: 80
public: 80
conf:
provisioning:
datasources:
template: |
apiVersion: 1
datasources:
- name: prometheus-one
type: prometheus
access: proxy
orgId: 1
editable: false
basicAuth: true
basicAuthUser: admin
secureJsonData:
basicAuthPassword: changeme
url: {{ tuple "monitoring_one" "internal" "api" . | include "helm-toolkit.endpoints.keystone_endpoint_uri_lookup" }}
- name: prometheus-two
type: prometheus
access: proxy
orgId: 1
editable: false
basicAuth: true
basicAuthUser: admin
secureJsonData:
basicAuthPassword: changeme
url: {{ tuple "monitoring_two" "internal" "api" . | include "helm-toolkit.endpoints.keystone_endpoint_uri_lookup" }}
- name: prometheus-three
type: prometheus
access: proxy
orgId: 1
editable: false
basicAuth: true
basicAuthUser: admin
secureJsonData:
basicAuthPassword: changeme
url: {{ tuple "monitoring_three" "internal" "api" . | include "helm-toolkit.endpoints.keystone_endpoint_uri_lookup" }}
- name: prometheus-federated
type: prometheus
access: proxy
orgId: 1
editable: false
basicAuth: true
basicAuthUser: admin
secureJsonData:
basicAuthPassword: changeme
url: {{ tuple "monitoring_federated" "internal" "api" . | include "helm-toolkit.endpoints.keystone_endpoint_uri_lookup" }}
EOF
#NOTE: Deploy command
helm upgrade --install grafana ./grafana \
--namespace=osh-infra \
--values=/tmp/grafana.yaml
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra
# Delete the test pod if it still exists
kubectl delete pods -l application=grafana,release_group=grafana,component=test --namespace=osh-infra --ignore-not-found
helm test grafana --namespace osh-infra
echo "Get list of all configured datasources in Grafana"
curl -u admin:password http://grafana.osh-infra.svc.cluster.local/api/datasources | jq -r .

33
tools/deployment/federated-monitoring/100-prometheus-selenium.sh
View File

@ -1,33 +0,0 @@
#!/bin/bash
set -xe
export CHROMEDRIVER="${CHROMEDRIVER:="/etc/selenium/chromedriver"}"
export ARTIFACTS_DIR="${ARTIFACTS_DIR:="/tmp/artifacts/"}"
export PROMETHEUS_USER="admin"
export PROMETHEUS_PASSWORD="changeme"
export PROMETHEUS_URI="prometheus-one.osh-infra.svc.cluster.local"
python3 tools/gate/selenium/prometheusSelenium.py
mv ${ARTIFACTS_DIR}/Prometheus_Command_Line_Flags.png ${ARTIFACTS_DIR}/Prometheus_One_Command_Line_Flags.png
mv ${ARTIFACTS_DIR}Prometheus_Dashboard.png ${ARTIFACTS_DIR}/Prometheus_One_Dashboard.png
mv ${ARTIFACTS_DIR}/Prometheus_Runtime_Info.png ${ARTIFACTS_DIR}/Prometheus_One_Runtime_Info.png
export PROMETHEUS_URI="prometheus-two.osh-infra.svc.cluster.local"
python3 tools/gate/selenium/prometheusSelenium.py
mv ${ARTIFACTS_DIR}/Prometheus_Command_Line_Flags.png ${ARTIFACTS_DIR}/Prometheus_Two_Command_Line_Flags.png
mv ${ARTIFACTS_DIR}/Prometheus_Dashboard.png ${ARTIFACTS_DIR}/Prometheus_Two_Dashboard.png
mv ${ARTIFACTS_DIR}/Prometheus_Runtime_Info.png ${ARTIFACTS_DIR}/Prometheus_Two_Runtime_Info.png
export PROMETHEUS_URI="prometheus-three.osh-infra.svc.cluster.local"
python3 tools/gate/selenium/prometheusSelenium.py
mv ${ARTIFACTS_DIR}/Prometheus_Command_Line_Flags.png ${ARTIFACTS_DIR}/Prometheus_Three_Command_Line_Flags.png
mv ${ARTIFACTS_DIR}/Prometheus_Dashboard.png ${ARTIFACTS_DIR}/Prometheus_Three_Dashboard.png
mv ${ARTIFACTS_DIR}/Prometheus_Runtime_Info.png ${ARTIFACTS_DIR}/Prometheus_Three_Runtime_Info.png
export PROMETHEUS_URI="prometheus-federate.osh-infra.svc.cluster.local"
python3 tools/gate/selenium/prometheusSelenium.py
mv ${ARTIFACTS_DIR}/Prometheus_Command_Line_Flags.png ${ARTIFACTS_DIR}/Prometheus_Federated_Command_Line_Flags.png
mv ${ARTIFACTS_DIR}/Prometheus_Dashboard.png ${ARTIFACTS_DIR}/Prometheus_Federated_Dashboard.png
mv ${ARTIFACTS_DIR}/Prometheus_Runtime_Info.png ${ARTIFACTS_DIR}/Prometheus_Federated_Runtime_Info.png

18
tools/deployment/keystone-auth/010-setup-client.sh
View File

@ -1,18 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
#NOTE: Move into openstack-helm root dir & Run client setup script
cd ${OSH_PATH:-"../openstack-helm/"}; ./tools/deployment/developer/nfs/020-setup-client.sh; cd -

1
tools/deployment/keystone-auth/030-nfs-provisioner.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/030-nfs-provisioner.sh

1
tools/deployment/keystone-auth/040-rabbitmq.sh
View File

@ -1 +0,0 @@
../openstack-support/030-rabbitmq.sh

1
tools/deployment/keystone-auth/050-memcached.sh
View File

@ -1 +0,0 @@
../openstack-support/040-memcached.sh

35
tools/deployment/keystone-auth/060-mariadb.sh
View File

@ -1,35 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
: ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB:="$(./tools/deployment/common/get-values-overrides.sh mariadb)"}
#NOTE: Lint and package chart
make mariadb
#NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
helm upgrade --install mariadb ./mariadb \
--namespace=openstack \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB}
#NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack
# Delete the test pod if it still exists
kubectl delete pods -l application=mariadb,release_group=mariadb,component=test --namespace=openstack --ignore-not-found
#NOTE: Validate the deployment
helm test mariadb --namespace openstack

153
tools/deployment/keystone-auth/080-check.sh
View File

@ -1,153 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
export OS_CLOUD=openstack_helm
function keystone_token () {
openstack token issue -f value -c id
}
function report_failed_policy () {
echo "$1 was $2 to perform $3, which contradicts current policy"
exit 1
}
function test_user_is_authorized () {
TOKEN=$(keystone_token)
if ! kubectl --kubeconfig /tmp/kubeconfig.yaml --token $TOKEN $1 ; then
report_failed_policy "$OS_USERNAME" "not allowed" "$1"
fi
}
function test_user_is_unauthorized () {
TOKEN=$(keystone_token)
if ! kubectl --kubeconfig /tmp/kubeconfig.yaml --token $TOKEN $1 ; then
echo "Denied, as expected by policy"
else
report_failed_policy "$OS_USERNAME" "allowed" "$1"
fi
}
sudo cp -va $HOME/.kube/config /tmp/kubeconfig.yaml
sudo kubectl --kubeconfig /tmp/kubeconfig.yaml config unset users.kubernetes-admin
# Test
# This issues token with admin role
TOKEN=$(keystone_token)
kubectl --kubeconfig /tmp/kubeconfig.yaml --token $TOKEN get pods
kubectl --kubeconfig /tmp/kubeconfig.yaml --token $TOKEN get pods -n openstack
kubectl --kubeconfig /tmp/kubeconfig.yaml --token $TOKEN get secrets -n openstack
# This is used to grab a pod name for the following tests
TEST_POD="$(kubectl get pods -n openstack | awk 'NR==2{print $1}')"
# create users
openstack user create --or-show --password password admin_k8cluster_user
openstack user create --or-show --password password admin_k8cluster_edit_user
openstack user create --or-show --password password admin_k8cluster_view_user
# create project
openstack project create --or-show openstack-system
openstack project create --or-show demoProject
# create roles
openstack role create --or-show openstackRole
openstack role create --or-show kube-system-admin
openstack role create --or-show admin_k8cluster
openstack role create --or-show admin_k8cluster_editor
openstack role create --or-show admin_k8cluster_viewer
# assign user role to project
openstack role add --project openstack-system --user bob --project-domain default --user-domain ldapdomain openstackRole
openstack role add --project demoProject --user alice --project-domain default --user-domain ldapdomain kube-system-admin
openstack role add --project demoProject --user admin_k8cluster_user --project-domain default --user-domain default admin_k8cluster
openstack role add --project demoProject --user admin_k8cluster_edit_user --project-domain default --user-domain default admin_k8cluster_editor
openstack role add --project demoProject --user admin_k8cluster_view_user --project-domain default --user-domain default admin_k8cluster_viewer
unset OS_CLOUD
export OS_AUTH_URL="http://keystone.openstack.svc.cluster.local/v3"
export OS_IDENTITY_API_VERSION="3"
export OS_PROJECT_NAME="openstack-system"
export OS_PASSWORD="password"
export OS_USERNAME="bob"
export OS_USER_DOMAIN_NAME="ldapdomain"
# Create files for secret generation
echo -n 'admin' > /tmp/user.txt
echo -n 'password' > /tmp/pass.txt
# See this does fail as the policy does not allow for a non-admin user
TOKEN=$(keystone_token)
test_user_is_unauthorized "get pods"
export OS_USERNAME="alice"
export OS_PROJECT_NAME="demoProject"
test_user_is_unauthorized "get pods -n openstack"
export OS_USER_DOMAIN_NAME="default"
#admin_k8cluser_user
export OS_USERNAME="admin_k8cluster_user"
RESOURCES=("pods" "configmaps" "endpoints" "persistentvolumeclaims" \
"replicationcontrollers" "secrets" "serviceaccounts" \
"services" "events" "limitranges" "namespace" \
"replicationcontrollers" "resourcequotas" "daemonsets" \
"deployments" "replicasets" "statefulsets" "jobs" \
"cronjobs" "poddisruptionbudgets" "serviceaccounts" \
"networkpolicies" "horizontalpodautoscalers")
for r in "${RESOURCES[@]}" ; do
test_user_is_authorized "get $r"
done
test_user_is_authorized "create secret generic test-secret --from-file=/tmp/user.txt --from-file=/tmp/pass.txt"
test_user_is_authorized "delete secret test-secret"
#admin_k8cluster_edit_user
export OS_USERNAME="admin_k8cluster_edit_user"
RESOURCES=("pods" "configmaps" "endpoints" "persistentvolumeclaims" \
"replicationcontrollers" "secrets" "serviceaccounts" \
"services" "events" "limitranges" "namespace" \
"replicationcontrollers" "resourcequotas" "daemonsets" \
"deployments" "replicasets" "statefulsets" "jobs" \
"cronjobs" "poddisruptionbudgets" "serviceaccounts" \
"networkpolicies" "horizontalpodautoscalers")
for r in "${RESOURCES[@]}" ; do
test_user_is_authorized "get $r"
done
test_user_is_authorized "create secret generic test-secret --from-file=/tmp/user.txt --from-file=/tmp/pass.txt"
test_user_is_authorized "delete secret test-secret"
test_user_is_authorized "logs -n openstack $TEST_POD --tail=5"
test_user_is_unauthorized "create namespace test"
#admin_k8cluster_view_user
export OS_USERNAME="admin_k8cluster_view_user"
RESOURCES=("pods" "configmaps" "endpoints" "persistentvolumeclaims" \
"replicationcontrollers" "services" "serviceaccounts" \
"replicationcontrollers" "resourcequotas" "namespaces" \
"daemonsets" "deployments" "replicasets" "statefulsets" \
"poddisruptionbudgets" "networkpolicies")
for r in "${RESOURCES[@]}" ; do
test_user_is_authorized "get $r"
done
test_user_is_authorized "logs -n openstack $TEST_POD --tail=5"
test_user_is_unauthorized "delete pod $TEST_POD -n openstack"
test_user_is_unauthorized "create namespace test"
test_user_is_unauthorized "get secrets"
test_user_is_unauthorized "create secret generic test-secret --from-file=/tmp/user.txt --from-file=/tmp/pass.txt"

7
tools/deployment/osh-infra-logging/050-elasticsearch.sh → tools/deployment/logging/elasticsearch.sh
View File

@ -14,9 +14,6 @@
set -xe set -xe
#NOTE: Lint and package chart
make elasticsearch
#NOTE: Deploy command #NOTE: Deploy command
tee /tmp/elasticsearch.yaml << EOF tee /tmp/elasticsearch.yaml << EOF
jobs: jobs:
@ -167,7 +164,7 @@ manifests:
object_bucket_claim: true object_bucket_claim: true
EOF EOF
: ${OSH_INFRA_EXTRA_HELM_ARGS_ELASTICSEARCH:="$(./tools/deployment/common/get-values-overrides.sh elasticsearch)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_ELASTICSEARCH:="$(helm osh get-values-overrides -c elasticsearch ${FEATURES})"}
helm upgrade --install elasticsearch ./elasticsearch \ helm upgrade --install elasticsearch ./elasticsearch \
--namespace=osh-infra \ --namespace=osh-infra \
@ -176,7 +173,7 @@ helm upgrade --install elasticsearch ./elasticsearch \
${OSH_INFRA_EXTRA_HELM_ARGS_ELASTICSEARCH} ${OSH_INFRA_EXTRA_HELM_ARGS_ELASTICSEARCH}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra
# Delete the test pod if it still exists # Delete the test pod if it still exists
kubectl delete pods -l application=elasticsearch,release_group=elasticsearch,component=test --namespace=osh-infra --ignore-not-found kubectl delete pods -l application=elasticsearch,release_group=elasticsearch,component=test --namespace=osh-infra --ignore-not-found

10
tools/deployment/common/fluentbit.sh → tools/deployment/logging/fluentbit.sh
View File

@ -14,16 +14,12 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTBIT:="$(helm osh get-values-overrides -c fluentbit ${FEATURES})"}
make fluentbit
: ${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTBIT:="$(./tools/deployment/common/get-values-overrides.sh fluentbit)"}
helm upgrade --install fluentbit ./fluentbit \ helm upgrade --install fluentbit ./fluentbit \
--namespace=osh-infra \ --namespace=osh-infra \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTBIT} ${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTBIT}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra

6
tools/deployment/common/fluentd.sh → tools/deployment/logging/fluentd.sh
View File

@ -14,9 +14,7 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTD:="$(helm osh get-values-overrides -c fluentd ${FEATURES})"}
make fluentd
: ${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTD:="$(./tools/deployment/common/get-values-overrides.sh fluentd)"}
tee /tmp/fluentd.yaml << EOF tee /tmp/fluentd.yaml << EOF
pod: pod:
@ -185,4 +183,4 @@ helm upgrade --install fluentd ./fluentd \
${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTD} ${OSH_INFRA_EXTRA_HELM_ARGS_FLUENTD}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra

7
tools/deployment/osh-infra-logging/070-kibana.sh → tools/deployment/logging/kibana.sh
View File

@ -14,10 +14,7 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_INFRA_EXTRA_HELM_ARGS_KIBANA:="$(helm osh get-values-overrides -c kibana ${FEATURES})"}
make kibana
: ${OSH_INFRA_EXTRA_HELM_ARGS_KIBANA:="$(./tools/deployment/common/get-values-overrides.sh kibana)"}
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install kibana ./kibana \ helm upgrade --install kibana ./kibana \
@ -27,4 +24,4 @@ helm upgrade --install kibana ./kibana \
${OSH_INFRA_EXTRA_HELM_ARGS_KIBANA} ${OSH_INFRA_EXTRA_HELM_ARGS_KIBANA}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra

1
tools/deployment/mariadb-operator-cluster/000-prepare-k8s.sh
View File

@ -1 +0,0 @@
../common/prepare-k8s.sh

1
tools/deployment/mariadb-operator-cluster/010-deploy-docker-registry.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/010-deploy-docker-registry.sh

1
tools/deployment/mariadb-operator-cluster/012-setup-client.sh
View File

@ -1 +0,0 @@
../common/setup-client.sh

1
tools/deployment/mariadb-operator-cluster/030-nfs-provisioner.sh
View File

@ -1 +0,0 @@
../osh-infra-monitoring/030-nfs-provisioner.sh

1
tools/deployment/mariadb-operator-cluster/040-rabbitmq.sh
View File

@ -1 +0,0 @@
../keystone-auth/040-rabbitmq.sh

1
tools/deployment/mariadb-operator-cluster/050-memcached.sh
View File

@ -1 +0,0 @@
../keystone-auth/050-memcached.sh

48
tools/deployment/mariadb-operator-cluster/070-keystone.sh
View File

@ -1,48 +0,0 @@
#!/bin/bash
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
set -xe
: ${OSH_PATH:="../openstack-helm"}
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
: ${OSH_EXTRA_HELM_ARGS:=""}
: ${OSH_EXTRA_HELM_ARGS_KEYSTONE:="$(HELM_CHART_ROOT_PATH=${OSH_PATH} ./tools/deployment/common/get-values-overrides.sh keystone)"}
# Install LDAP
make ldap
helm upgrade --install ldap ./ldap \
--namespace=openstack \
--set pod.replicas.server=1 \
--set bootstrap.enabled=true \
${OSH_INFRA_EXTRA_HELM_ARGS} \
${OSH_INFRA_EXTRA_HELM_ARGS_LDAP}
# Install Keystone
cd ${OSH_PATH}
make keystone
cd -
helm upgrade --install keystone ${OSH_PATH}/keystone \
--namespace=openstack \
--values=${OSH_PATH}/keystone/values_overrides/ldap.yaml \
--set network.api.ingress.classes.namespace=nginx \
--set endpoints.oslo_db.hosts.default=mariadb-server-primary \
${OSH_EXTRA_HELM_ARGS} \
${OSH_EXTRA_HELM_ARGS_KEYSTONE}
./tools/deployment/common/wait-for-pods.sh openstack
# Testing basic functionality
export OS_CLOUD=openstack_helm
sleep 30 #NOTE(portdirect): Wait for ingress controller to update rules and restart Nginx
openstack endpoint list

5
tools/deployment/osh-infra-monitoring/060-alertmanager.sh → tools/deployment/monitoring/alertmanager.sh
View File

@ -14,13 +14,10 @@
set -xe set -xe
#NOTE: Lint and package chart
make prometheus-alertmanager
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install prometheus-alertmanager ./prometheus-alertmanager \ helm upgrade --install prometheus-alertmanager ./prometheus-alertmanager \
--namespace=osh-infra \ --namespace=osh-infra \
--set pod.replicas.alertmanager=1 --set pod.replicas.alertmanager=1
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra

5
tools/deployment/common/blackbox-exporter.sh → tools/deployment/monitoring/blackbox-exporter.sh
View File

@ -14,12 +14,9 @@
set -xe set -xe
#NOTE: Lint and package chart
make prometheus-blackbox-exporter
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install prometheus-blackbox-exporter \ helm upgrade --install prometheus-blackbox-exporter \
./prometheus-blackbox-exporter --namespace=osh-infra ./prometheus-blackbox-exporter --namespace=osh-infra
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra

11
tools/deployment/osh-infra-monitoring/110-grafana.sh → tools/deployment/monitoring/grafana.sh
View File

@ -14,20 +14,17 @@
set -xe set -xe
#NOTE: Lint and package chart FEATURE_GATES="calico ceph containers coredns elasticsearch kubernetes nginx nodes openstack prometheus home_dashboard persistentvolume apparmor"
make grafana : ${OSH_INFRA_EXTRA_HELM_ARGS_GRAFANA:=$(helm osh get-values-overrides -c grafana ${FEATURE_GATES} ${FEATURES} 2>/dev/null)}
FEATURE_GATES="calico,ceph,containers,coredns,elasticsearch,kubernetes,nginx,nodes,openstack,prometheus,home_dashboard,persistentvolume,apparmor"
: ${OSH_INFRA_EXTRA_HELM_ARGS_GRAFANA:="$({ ./tools/deployment/common/get-values-overrides.sh grafana;} 2> /dev/null)"}
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install grafana ./grafana \ helm upgrade --install grafana ./grafana \
--namespace=osh-infra \ --namespace=osh-infra \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_GRAFANA} ${OSH_INFRA_EXTRA_HELM_ARGS_GRAFANA}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra
# Delete the test pod if it still exists # Delete the test pod if it still exists
kubectl delete pods -l application=grafana,release_group=grafana,component=test --namespace=osh-infra --ignore-not-found kubectl delete pods -l application=grafana,release_group=grafana,component=test --namespace=osh-infra --ignore-not-found

7
tools/deployment/common/070-kube-state-metrics.sh → tools/deployment/monitoring/kube-state-metrics.sh
View File

@ -14,15 +14,12 @@
set -xe set -xe
#NOTE: Lint and package chart
make prometheus-kube-state-metrics
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS_KUBE_STATE_METRICS:="$(./tools/deployment/common/get-values-overrides.sh prometheus-kube-state-metrics)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_KUBE_STATE_METRICS:="$(helm osh get-values-overrides -c prometheus-kube-state-metrics ${FEATURES})"}
helm upgrade --install prometheus-kube-state-metrics \ helm upgrade --install prometheus-kube-state-metrics \
./prometheus-kube-state-metrics --namespace=kube-system \ ./prometheus-kube-state-metrics --namespace=kube-system \
${OSH_INFRA_EXTRA_HELM_ARGS_KUBE_STATE_METRICS} ${OSH_INFRA_EXTRA_HELM_ARGS_KUBE_STATE_METRICS}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system helm osh wait-for-pods kube-system

11
tools/deployment/mariadb-operator-cluster/095-mariadb-prometheus-mysql-exporter.sh → tools/deployment/monitoring/mysql-exporter.sh
View File

@ -14,23 +14,18 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_MYSQL_EXPORTER:="$(helm osh get-values-overrides -c prometheus-mysql-exporter ${FEATURES})"}
make prometheus-mysql-exporter
: ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_MYSQL_EXPORTER:="$(./tools/deployment/common/get-values-overrides.sh prometheus-mysql-exporter)"}
#NOTE: Deploy command #NOTE: Deploy command
# Deploying downscaled cluster
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
helm upgrade --install prometheus-mysql-exporter ./prometheus-mysql-exporter \ helm upgrade --install prometheus-mysql-exporter ./prometheus-mysql-exporter \
--namespace=openstack \ --namespace=openstack \
--wait \ --wait \
--timeout 900s \ --timeout 900s \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_MYSQL_EXPORTER} ${OSH_INFRA_EXTRA_HELM_ARGS_MARIADB_MYSQL_EXPORTER}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack helm osh wait-for-pods openstack
kubectl get pods --namespace=openstack -o wide kubectl get pods --namespace=openstack -o wide

9
tools/deployment/osh-infra-monitoring-tls/120-nagios.sh → tools/deployment/monitoring/nagios.sh
View File

@ -14,19 +14,16 @@
set -xe set -xe
#NOTE: Lint and package chart : ${OSH_INFRA_EXTRA_HELM_ARGS_NAGIOS:="$(helm osh get-values-overrides -c nagios ${FEATURES})"}
make nagios
: ${OSH_INFRA_EXTRA_HELM_ARGS_NAGIOS:="$(./tools/deployment/common/get-values-overrides.sh nagios)"}
#NOTE: Deploy command #NOTE: Deploy command
helm upgrade --install nagios ./nagios \ helm upgrade --install nagios ./nagios \
--namespace=osh-infra \ --namespace=osh-infra \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_NAGIOS} ${OSH_INFRA_EXTRA_HELM_ARGS_NAGIOS}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra
# Delete the test pod if it still exists # Delete the test pod if it still exists
kubectl delete pods -l application=nagios,release_group=nagios,component=test --namespace=osh-infra --ignore-not-found kubectl delete pods -l application=nagios,release_group=nagios,component=test --namespace=osh-infra --ignore-not-found

7
tools/deployment/common/080-node-exporter.sh → tools/deployment/monitoring/node-exporter.sh
View File

@ -14,15 +14,12 @@
set -xe set -xe
#NOTE: Lint and package chart
make prometheus-node-exporter
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS_NODE_EXPORTER:="$(./tools/deployment/common/get-values-overrides.sh prometheus-node-exporter)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_NODE_EXPORTER:="$(helm osh get-values-overrides -c prometheus-node-exporter ${FEATURES})"}
helm upgrade --install prometheus-node-exporter \ helm upgrade --install prometheus-node-exporter \
./prometheus-node-exporter --namespace=kube-system \ ./prometheus-node-exporter --namespace=kube-system \
${OSH_INFRA_EXTRA_HELM_ARGS_NODE_EXPORTER} ${OSH_INFRA_EXTRA_HELM_ARGS_NODE_EXPORTER}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system helm osh wait-for-pods kube-system

5
tools/deployment/common/node-problem-detector.sh → tools/deployment/monitoring/node-problem-detector.sh
View File

@ -13,9 +13,6 @@
set -xe set -xe
#NOTE: Lint and package chart
make kubernetes-node-problem-detector
#NOTE: Deploy command #NOTE: Deploy command
tee /tmp/kubernetes-node-problem-detector.yaml << EOF tee /tmp/kubernetes-node-problem-detector.yaml << EOF
monitoring: monitoring:
@ -32,4 +29,4 @@ helm upgrade --install kubernetes-node-problem-detector \
--values=/tmp/kubernetes-node-problem-detector.yaml --values=/tmp/kubernetes-node-problem-detector.yaml
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system helm osh wait-for-pods kube-system

7
tools/deployment/common/openstack-exporter.sh → tools/deployment/monitoring/openstack-exporter.sh
View File

@ -14,11 +14,8 @@
set -xe set -xe
#NOTE: Lint and package chart
make prometheus-openstack-exporter
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS_OS_EXPORTER:="$(./tools/deployment/common/get-values-overrides.sh prometheus-openstack-exporter)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_OS_EXPORTER:="$(helm osh get-values-overrides -c prometheus-openstack-exporter ${FEATURES})"}
tee /tmp/prometheus-openstack-exporter.yaml << EOF tee /tmp/prometheus-openstack-exporter.yaml << EOF
manifests: manifests:
@ -37,4 +34,4 @@ helm upgrade --install prometheus-openstack-exporter \
${OSH_INFRA_EXTRA_HELM_ARGS_OS_EXPORTER} ${OSH_INFRA_EXTRA_HELM_ARGS_OS_EXPORTER}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh openstack helm osh wait-for-pods openstack

7
tools/deployment/common/090-process-exporter.sh → tools/deployment/monitoring/process-exporter.sh
View File

@ -14,15 +14,12 @@
set -xe set -xe
#NOTE: Lint and package chart
make prometheus-process-exporter
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS_PROCESS_EXPORTER:="$(./tools/deployment/common/get-values-overrides.sh prometheus-process-exporter)"} : ${OSH_INFRA_EXTRA_HELM_ARGS_PROCESS_EXPORTER:="$(helm osh get-values-overrides -c prometheus-process-exporter ${FEATURES})"}
helm upgrade --install prometheus-process-exporter \ helm upgrade --install prometheus-process-exporter \
./prometheus-process-exporter --namespace=kube-system \ ./prometheus-process-exporter --namespace=kube-system \
${OSH_INFRA_EXTRA_HELM_ARGS_PROCESS_EXPORTER} ${OSH_INFRA_EXTRA_HELM_ARGS_PROCESS_EXPORTER}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh kube-system helm osh wait-for-pods kube-system

11
tools/deployment/osh-infra-local-storage/040-prometheus.sh → tools/deployment/monitoring/prometheus.sh
View File

@ -14,20 +14,17 @@
set -xe set -xe
#NOTE: Lint and package chart FEATURE_GATES="alertmanager ceph elasticsearch kubernetes nodes openstack postgresql apparmor"
make prometheus : ${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS:="$(helm osh get-values-overrides -c prometheus ${FEATURE_GATES} ${FEATURES})"}
#NOTE: Deploy command #NOTE: Deploy command
: ${OSH_INFRA_EXTRA_HELM_ARGS:=""}
: ${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS:="$(./tools/deployment/common/get-values-overrides.sh prometheus)"}
helm upgrade --install prometheus ./prometheus \ helm upgrade --install prometheus ./prometheus \
--namespace=osh-infra \ --namespace=osh-infra \
${OSH_INFRA_EXTRA_HELM_ARGS} \ ${OSH_INFRA_EXTRA_HELM_ARGS:=} \
${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS} ${OSH_INFRA_EXTRA_HELM_ARGS_PROMETHEUS}
#NOTE: Wait for deploy #NOTE: Wait for deploy
./tools/deployment/common/wait-for-pods.sh osh-infra helm osh wait-for-pods osh-infra
# Delete the test pod if it still exists # Delete the test pod if it still exists
kubectl delete pods -l application=prometheus,release_group=prometheus,component=test --namespace=osh-infra --ignore-not-found kubectl delete pods -l application=prometheus,release_group=prometheus,component=test --namespace=osh-infra --ignore-not-found

Some files were not shown because too many files have changed in this diff Show More