diff --git a/doc/source/multi_node.rst b/doc/source/multi_node.rst index ea9dabe..181d338 100644 --- a/doc/source/multi_node.rst +++ b/doc/source/multi_node.rst @@ -1,8 +1,154 @@ -Setting up a multi nodes cluster Stackube -===================================== +Setting Up A Multi-nodes Stackube (Without HA For Now) +====================================================== This page describes how to setup a multi-nodes cluster of Stackube. -================= -TODO -================= \ No newline at end of file +Prerequisites +------------- + +Roles +~~~~~ + +A stackube deployment is comprised by four kinds of nodes: control, +network, compute, storage. + +- Control + + - The control node is where openstack/kubernetes/ceph's + control-plane will run. + - **At least one and only one node** (for now). + - Minimum hardware requirements: + + - Two network interfaces + + - One is for public network connection, with a public IP. + - The other one is for private network connection, with a + private IP and MTU >= 1600. + + - 8GB main memory + - 50GB disk space + +- Network + + - The network nodes are where neutron l3/lbaas/dhcp agents will run. + - At least one node. + - Minimum hardware requirements: + + - Two network interfaces + + - One is as neutron-external-interface. Public IP is not + needed. + - The other one is for private network connection, with a + private IP and MTU >= 1600. + + - 8GB main memory + - 50GB disk space + +- Compute + + - The compute nodes are where your workloads will run. + - At least one node. + - Minimum hardware requirements: + + - One network interface + + - For private network connection, with a private IP and MTU >= + 1600. + + - 8GB main memory + - 50GB disk space + +- Storage + + - The storage nodes are where ceph-osd(s) will run. + - At least one node. + - Minimum hardware requirements: + + - One network interface + + - For private network connection, with a private IP and MTU >= + 1600. + + - 8GB main memory + - 50GB disk space + +There is no conflict between any two roles. That means, all of the roles +could be deployed on the same node(s). + +Host OS +~~~~~~~ + +For now only CentOS 7.x is supported. + +Public IP Pool +~~~~~~~~~~~~~~ + +A number of public IPs are needed. + +Deploy +------ + +All instructions below **must be done on the control node.** + +1. SSH To The Control Node, And Become Root +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + sudo su - + +2. Enable Password-Less SSH +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The control node needs to ssh to all nodes when deploying. + +- Generate SSH keys on the control node. Leave the passphrase empty: + +:: + + ssh-keygen + + Generating public/private rsa key pair. + Enter file in which to save the key (/root/.ssh/id_rsa): + Enter passphrase (empty for no passphrase): + Enter same passphrase again: + Your identification has been saved in /root/.ssh/id_rsa. + Your public key has been saved in /root/.ssh/id_rsa.pub. + +- Copy the key to each node (including the control node itself): + +:: + + ssh-copy-id root@NODE_IP + +3. Clone Stackube Repo +~~~~~~~~~~~~~~~~~~~~~~ + +:: + + git clone https://git.openstack.org/openstack/stackube + +4. Edit The Config File +~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + cd stackube/install + vim config_example + +5. Do The Deploy +~~~~~~~~~~~~~~~~ + +:: + + bash deploy.sh config_example + +If failed, please **do remove** (as shown below) before deploy again. + +Remove +------ + +:: + + bash remove.sh config_example + diff --git a/install/README.md b/install/README.md new file mode 100644 index 0000000..33020b2 --- /dev/null +++ b/install/README.md @@ -0,0 +1,112 @@ +# Setting Up A Multi-nodes Stackube (Without HA For Now) + +This page describes how to setup a multi-nodes cluster of Stackube. + +## Prerequisites + +### Roles + +A stackube deployment is comprised by four kinds of nodes: control, network, compute, storage. + +- Control + - The control node is where openstack/kubernetes/ceph's control-plane will run. + - **At least one and only one node** (for now). + - Minimum hardware requirements: + - Two network interfaces + - One is for public network connection, with a public IP. + - The other one is for private network connection, with a private IP and MTU >= 1600. + - 8GB main memory + - 50GB disk space + +- Network + - The network nodes are where neutron l3/lbaas/dhcp agents will run. + - At least one node. + - Minimum hardware requirements: + - Two network interfaces + - One is as neutron-external-interface. Public IP is not needed. + - The other one is for private network connection, with a private IP and MTU >= 1600. + - 8GB main memory + - 50GB disk space + +- Compute + - The compute nodes are where your workloads will run. + - At least one node. + - Minimum hardware requirements: + - One network interface + - For private network connection, with a private IP and MTU >= 1600. + - 8GB main memory + - 50GB disk space + +- Storage + - The storage nodes are where ceph-osd(s) will run. + - At least one node. + - Minimum hardware requirements: + - One network interface + - For private network connection, with a private IP and MTU >= 1600. + - 8GB main memory + - 50GB disk space + +There is no conflict between any two roles. That means, all of the roles could be deployed on the same node(s). + +### Host OS +For now only CentOS 7.x is supported. + +### Public IP Pool +A number of public IPs are needed. + + +## Deploy + +All instructions below **must be done on the control node.** + +### 1. SSH To The Control Node, And Become Root +``` +sudo su - +``` + +### 2. Enable Password-Less SSH + +The control node needs to ssh to all nodes when deploying. + +- Generate SSH keys on the control node. Leave the passphrase empty: + +``` +ssh-keygen + +Generating public/private rsa key pair. +Enter file in which to save the key (/root/.ssh/id_rsa): +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in /root/.ssh/id_rsa. +Your public key has been saved in /root/.ssh/id_rsa.pub. +``` + +- Copy the key to each node (including the control node itself): +``` +ssh-copy-id root@NODE_IP +``` + +### 3. Clone Stackube Repo +``` +git clone https://git.openstack.org/openstack/stackube +``` + +### 4. Edit The Config File +``` +cd stackube/install +vim config_example +``` + +### 5. Do The Deploy +``` +bash deploy.sh config_example +``` + +If failed, please **do remove** (as shown below) before deploy again. + + + +## Remove +``` +bash remove.sh config_example +``` diff --git a/install/ceph/config_ceph/ceph-mon/ceph.conf b/install/ceph/config_ceph/ceph-mon/ceph.conf new file mode 100644 index 0000000..3cfc658 --- /dev/null +++ b/install/ceph/config_ceph/ceph-mon/ceph.conf @@ -0,0 +1,53 @@ +[global] +log file = /var/log/kolla/ceph/$cluster-$name.log +log to syslog = true +err to syslog = true + +fsid = __FSID__ +mon initial members = __PUBLIC_IP__ +mon host = __PUBLIC_IP__ +mon addr = __PUBLIC_IP__:6789 + +auth cluster required = cephx +auth service required = cephx +auth client required = cephx + +# NOTE(inc0): This line will mean that if ceph was upgraded, it will run as root +# until contents of /var/lib/ceph are chowned to ceph user. +# This change was introduced in Jewel version and we should include +# chown operation in upgrade procedure. https://bugs.launchpad.net/kolla/+bug/1620702 +setuser match path = /var/lib/ceph/$type/$cluster-$id + +# Starting with the Jewel release, the ceph-osd daemon will refuse to start if the configured +# max object name cannot be safely stored on ext4. If the cluster is only being used with short +# object names (e.g., RBD only), you can continue using ext4 by setting the following configuration option: +# Note: This may result in difficult-to-diagnose errors if you try to use RGW or other librados +# clients that do not properly handle or politely surface any resulting ENAMETOOLONG errors. +osd max object name len = 256 +osd max object namespace len = 64 + +osd journal size = 5120 +osd pool default size = 1 +osd pool default min size = 1 +osd pool default pg num = 64 +osd pool default pgp num = 64 + +osd crush chooseleaf type = 1 + +rbd default features = 3 + + +[client] +rbd cache = false +rbd cache size = 0 +rbd cache max dirty =0 +rbd cache target dirty = 0 +rbd cache writethrough until flush = false + + +[mon] +# NOTE(SamYaple): The monitor files have been known to grow very large. The +# only fix for that is to compact the files. +mon compact on start = true +mon cluster log file = /var/log/kolla/ceph/$cluster.log + diff --git a/install/ceph/config_ceph/ceph-mon/config.json b/install/ceph/config_ceph/ceph-mon/config.json new file mode 100644 index 0000000..03bb555 --- /dev/null +++ b/install/ceph/config_ceph/ceph-mon/config.json @@ -0,0 +1,39 @@ +{ + "command": "/usr/bin/ceph-mon -d -i __PUBLIC_IP__ --public-addr __PUBLIC_IP__:6789", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/ceph.conf", + "dest": "/etc/ceph/ceph.conf", + "owner": "ceph", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ceph.client.admin.keyring", + "dest": "/etc/ceph/ceph.client.admin.keyring", + "owner": "ceph", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/ceph.client.mon.keyring", + "dest": "/etc/ceph/ceph.client.mon.keyring", + "owner": "ceph", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/ceph.client.radosgw.keyring", + "dest": "/etc/ceph/ceph.client.radosgw.keyring", + "owner": "ceph", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/ceph.monmap", + "dest": "/etc/ceph/ceph.monmap", + "owner": "ceph", + "perm": "0600", + "optional": true + } + ] +} diff --git a/install/ceph/config_ceph/ceph-osd/add_osd.sh b/install/ceph/config_ceph/ceph-osd/add_osd.sh new file mode 100644 index 0000000..63564c5 --- /dev/null +++ b/install/ceph/config_ceph/ceph-osd/add_osd.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# + +set -x + +cp --remove-destination /var/lib/kolla/config_files/{ceph.client.admin.keyring,ceph.conf} /etc/ceph/ || exit 1 + +ceph osd crush add-bucket __PUBLIC_IP__ host || exit 1 +ceph osd crush move __PUBLIC_IP__ root=default || exit 1 + +num=`ceph osd create` || exit 1 +echo $num || exit 1 +mkdir -p /var/lib/ceph/osd/ceph-${num} || exit 1 +ceph-osd -i ${num} --mkfs --mkkey || exit 1 +ceph auth add osd.${num} osd 'allow *' mon 'allow profile osd' -i /var/lib/ceph/osd/ceph-${num}/keyring || exit 1 +ceph osd crush add osd.${num} 1.0 host=__PUBLIC_IP__ || exit 1 + +chown ceph:ceph /var/lib/ceph/osd -R || exit 1 + +ceph osd crush tree + +exit 0 + diff --git a/install/ceph/config_ceph/ceph-osd/config.json b/install/ceph/config_ceph/ceph-osd/config.json new file mode 100644 index 0000000..7bb9649 --- /dev/null +++ b/install/ceph/config_ceph/ceph-osd/config.json @@ -0,0 +1,17 @@ +{ + "command": "/usr/bin/ceph-osd -f -d --public-addr __PUBLIC_IP__ --cluster-addr __CLUSTER_IP__", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/ceph.conf", + "dest": "/etc/ceph/ceph.conf", + "owner": "ceph", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ceph.client.admin.keyring", + "dest": "/etc/ceph/ceph.client.admin.keyring", + "owner": "ceph", + "perm": "0600" + } + ] +} diff --git a/install/ceph/deploy_ceph_mon.sh b/install/ceph/deploy_ceph_mon.sh new file mode 100644 index 0000000..7cf1592 --- /dev/null +++ b/install/ceph/deploy_ceph_mon.sh @@ -0,0 +1,85 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``CEPH_MON_PUBLIC_IP`` +# - ``CEPH_FSID`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/ceph +chmod 777 /var/log/stackube/ceph + + +## config files +mkdir -p /etc/stackube/ceph +cp -a ${programDir}/config_ceph/ceph-mon /etc/stackube/ceph/ +sed -i "s/__FSID__/${CEPH_FSID}/g" /etc/stackube/ceph/ceph-mon/ceph.conf +sed -i "s/__PUBLIC_IP__/${CEPH_MON_PUBLIC_IP}/g" /etc/stackube/ceph/ceph-mon/ceph.conf +sed -i "s/__PUBLIC_IP__/${CEPH_MON_PUBLIC_IP}/g" /etc/stackube/ceph/ceph-mon/config.json + + +mkdir -p /var/lib/stackube/ceph/ceph_mon_config && \ +mkdir -p /var/lib/stackube/ceph/ceph_mon && \ +docker run --net host \ + --name stackube_ceph_bootstrap_mon \ + -v /etc/stackube/ceph/ceph-mon/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/ceph:/var/log/kolla/:rw \ + -v /var/lib/stackube/ceph/ceph_mon_config:/etc/ceph/:rw \ + -v /var/lib/stackube/ceph/ceph_mon:/var/lib/ceph/:rw \ + \ + -e "KOLLA_BOOTSTRAP=" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + -e "MON_IP=${CEPH_MON_PUBLIC_IP}" \ + -e "HOSTNAME=${CEPH_MON_PUBLIC_IP}" \ + kolla/centos-binary-ceph-mon:4.0.0 + +docker rm stackube_ceph_bootstrap_mon + + +docker run -d --net host \ + --name stackube_ceph_mon \ + -v /etc/stackube/ceph/ceph-mon/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/ceph:/var/log/kolla/:rw \ + -v /var/lib/stackube/ceph/ceph_mon_config:/etc/ceph/:rw \ + -v /var/lib/stackube/ceph/ceph_mon:/var/lib/ceph/:rw \ + \ + -e "KOLLA_SERVICE_NAME=ceph-mon" \ + -e "HOSTNAME=${CEPH_MON_PUBLIC_IP}" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + kolla/centos-binary-ceph-mon:4.0.0 + +sleep 5 + +docker exec stackube_ceph_mon ceph -s + + + +exit 0 diff --git a/install/ceph/deploy_ceph_osd.sh b/install/ceph/deploy_ceph_osd.sh new file mode 100644 index 0000000..c7326b3 --- /dev/null +++ b/install/ceph/deploy_ceph_osd.sh @@ -0,0 +1,82 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``CEPH_OSD_PUBLIC_IP``, ``CEPH_OSD_CLUSTER_IP``, +# - ``CEPH_OSD_DATA_DIR`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/ceph +chmod 777 /var/log/stackube/ceph + + +## config files +sed -i "s/__PUBLIC_IP__/${CEPH_OSD_PUBLIC_IP}/g" /etc/stackube/ceph/ceph-osd/add_osd.sh +sed -i "s/__PUBLIC_IP__/${CEPH_OSD_PUBLIC_IP}/g" /etc/stackube/ceph/ceph-osd/config.json +sed -i "s/__CLUSTER_IP__/${CEPH_OSD_CLUSTER_IP}/g" /etc/stackube/ceph/ceph-osd/config.json + + +## bootstrap +mkdir -p ${CEPH_OSD_DATA_DIR} +docker run --net host \ + --name stackube_ceph_bootstrap_osd \ + -v /etc/stackube/ceph/ceph-osd/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/ceph:/var/log/kolla/:rw \ + -v ${CEPH_OSD_DATA_DIR}:/var/lib/ceph/:rw \ + \ + kolla/centos-binary-ceph-osd:4.0.0 /bin/bash /var/lib/kolla/config_files/add_osd.sh + +docker rm stackube_ceph_bootstrap_osd + + +## run +theOsd=`ls ${CEPH_OSD_DATA_DIR}/osd/ | grep -- 'ceph-' | head -n 1` +[ "${theOsd}" ] +osdId=`echo $theOsd | awk -F\- '{print $NF}'` +[ "${osdId}" ] + +docker run -d --net host \ + --name stackube_ceph_osd_${osdId} \ + -v /etc/stackube/ceph/ceph-osd/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/ceph:/var/log/kolla/:rw \ + -v ${CEPH_OSD_DATA_DIR}:/var/lib/ceph/:rw \ + \ + -e "KOLLA_SERVICE_NAME=ceph-osd" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + -e "OSD_ID=${osdId}" \ + -e "JOURNAL_PARTITION=/var/lib/ceph/osd/ceph-${osdId}/journal" \ + \ + --restart unless-stopped \ + kolla/centos-binary-ceph-osd:4.0.0 + +sleep 5 + + + +exit 0 diff --git a/install/ceph/remove_ceph_from_node.sh b/install/ceph/remove_ceph_from_node.sh new file mode 100644 index 0000000..4e965f6 --- /dev/null +++ b/install/ceph/remove_ceph_from_node.sh @@ -0,0 +1,39 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -x + + +## remove docker containers +stackubeCephConstaners=`docker ps -a | awk '{print $NF}' | grep '^stackube_ceph_' ` +if [ "${stackubeCephConstaners}" ]; then + docker rm -f $stackubeCephConstaners || exit 1 +fi + +## rm dirs +rm -fr /etc/stackube/ceph /var/log/stackube/ceph /var/lib/stackube/ceph ${CEPH_OSD_DATA_DIR} || exit 1 + + + +exit 0 + diff --git a/install/config_example b/install/config_example new file mode 100644 index 0000000..eaa7be9 --- /dev/null +++ b/install/config_example @@ -0,0 +1,41 @@ +#!/bin/bash + + +# Control Node +# At least one and only one node (for now). +CONTROL_NODE_PUBLIC_IP='147.75.77.67' +CONTROL_NODE_PRIVATE_IP='192.168.1.1' + +# Network Node(s) +# At least one node. Could be more (separated by commas). +NETWORK_NODES_PRIVATE_IP='192.168.1.2' +NETWORK_NODES_NEUTRON_EXT_IF='eth2' + +# Compute Node(s) +# At least one node. Could be more (separated by commas). +COMPUTE_NODES_PRIVATE_IP='192.168.1.1,192.168.1.2' + +# Storage Node(s) +# At least one node. Could be more (separated by commas). +STORAGE_NODES_PRIVATE_IP='192.168.1.1,192.168.1.2' +STORAGE_NODES_CEPH_OSD_DATA_DIR='/var/lib/stackube/ceph/ceph_osd,/var/lib/stackube/ceph/ceph_osd' + + +# NEUTRON_PUBLIC_SUBNET='subnet-range;gateway;allocation-pool' +NEUTRON_PUBLIC_SUBNET='147.75.192.224/29;147.75.192.225;start=147.75.192.226,end=147.75.192.230' + + +#CEPH_FSID= + +#RABBITMQ_PWD= + +#MYSQL_ROOT_PWD= +#MYSQL_KEYSTONE_PWD= +#MYSQL_NEUTRON_PWD= +#MYSQL_CINDER_PWD= + +#KEYSTONE_ADMIN_PWD= +#KEYSTONE_NEUTRON_PWD= +#KEYSTONE_CINDER_PWD= + + diff --git a/install/deploy.sh b/install/deploy.sh new file mode 100644 index 0000000..43c2b1a --- /dev/null +++ b/install/deploy.sh @@ -0,0 +1,123 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + + +function usage { + echo " +Usage: + bash $(basename $0) CONFIG_FILE +" +} + +[ "$1" ] || { usage; exit 1; } +[ -f "$1" ] || { echo "Error: $1 not exists or not a file!"; exit 1; } + +source $(readlink -f $1) || { echo "'source $(readlink -f $1)' failed!"; exit 1; } + +[ "${CONTROL_NODE_PUBLIC_IP}" ] || { echo "Error: CONTROL_NODE_PUBLIC_IP not defined!"; exit 1; } +[ "${CONTROL_NODE_PRIVATE_IP}" ] || { echo "Error: CONTROL_NODE_PRIVATE_IP not defined!"; exit 1; } + +[ "${NETWORK_NODES_PRIVATE_IP}" ] || { echo "Error: NETWORK_NODES_PRIVATE_IP not defined!"; exit 1; } +[ "${NETWORK_NODES_NEUTRON_EXT_IF}" ] || { echo "Error: NETWORK_NODES_NEUTRON_EXT_IF not defined!"; exit 1; } + +[ "${COMPUTE_NODES_PRIVATE_IP}" ] || { echo "Error: COMPUTE_NODES_PRIVATE_IP not defined!"; exit 1; } + +[ "${STORAGE_NODES_PRIVATE_IP}" ] || { echo "Error: STORAGE_NODES_PRIVATE_IP not defined!"; exit 1; } +[ "${STORAGE_NODES_CEPH_OSD_DATA_DIR}" ] || { echo "Error: STORAGE_NODES_CEPH_OSD_DATA_DIR not defined!"; exit 1; } + +[ "${NEUTRON_PUBLIC_SUBNET}" ] || { echo "Error: NEUTRON_PUBLIC_SUBNET not defined!"; exit 1; } + + +##################### + + +function all_nodes_check_distro { + for IP in $1; do + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/{ensure_distro_supported.sh,lib_common.sh} root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/ensure_distro_supported.sh" + done +} + +function all_nodes_install_docker { + for IP in $1; do + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/ensure_docker_installed.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/ensure_docker_installed.sh" + done +} + + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log +logDir='/var/log/stackube' +logFile="${logDir}/install.log-$(date '+%Y-%m-%d_%H-%M-%S')" +mkdir -p ${logDir} + +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${NETWORK_NODES_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP} +${STORAGE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +{ + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') all_nodes_check_distro" + all_nodes_check_distro "${allIpList}" + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') all_nodes_install_docker" + all_nodes_install_docker "${allIpList}" + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') deploy_ceph" + bash ${programDir}/deploy_ceph.sh $(readlink -f $1) + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') deploy_openstack" + bash ${programDir}/deploy_openstack.sh $(readlink -f $1) + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') deploy_kubernetes" + bash ${programDir}/deploy_kubernetes.sh $(readlink -f $1) + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') All done!" + + echo " +Additional information: + * File /etc/stackube/openstack/admin-openrc.sh has been created. To use openstack command line tools you need to source the file. + * File /etc/kubernetes/admin.conf has been created. To use kubectl you need to do 'export KUBECONFIG=/etc/kubernetes/admin.conf'. + * The installation log file is available at: ${logFile} +" + +} 2>&1 | tee -a ${logFile} + + + + +exit 0 + + + + + + + diff --git a/install/deploy_ceph.sh b/install/deploy_ceph.sh new file mode 100644 index 0000000..de0c2e7 --- /dev/null +++ b/install/deploy_ceph.sh @@ -0,0 +1,65 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +source $(readlink -f $1) + +[ "${CONTROL_NODE_PRIVATE_IP}" ] +[ "${STORAGE_NODES_PRIVATE_IP}" ] +[ "${STORAGE_NODES_CEPH_OSD_DATA_DIR}" ] + + +# ceph-mon +export CEPH_MON_PUBLIC_IP="${CONTROL_NODE_PRIVATE_IP}" +export CEPH_FSID=${CEPH_FSID:-aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee} +/bin/bash ${programDir}/ceph/deploy_ceph_mon.sh + + +# ceph-osd +storageIpList=(`echo "${STORAGE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g'`) +osdDataDirList=(`echo "${STORAGE_NODES_CEPH_OSD_DATA_DIR}" | sed -e 's/,/\n/g'`) +[ ${#storageIpList[@]} -eq ${#osdDataDirList[@]} ] + +MAX=$((${#storageIpList[@]} - 1)) +for i in `seq 0 ${MAX}`; do + IP="${storageIpList[$i]}" + dataDir="${osdDataDirList[$i]}" + echo -e "\n------ ${IP} ${dataDir} ------" + ssh root@${IP} 'mkdir -p /etc/stackube/ceph /tmp/stackube_install' + scp -r ${programDir}/ceph/config_ceph/ceph-osd root@${IP}:/etc/stackube/ceph/ + scp -r /var/lib/stackube/ceph/ceph_mon_config/{ceph.client.admin.keyring,ceph.conf} root@${IP}:/etc/stackube/ceph/ceph-osd/ + + scp ${programDir}/ceph/deploy_ceph_osd.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "export CEPH_OSD_PUBLIC_IP='${IP}' + export CEPH_OSD_CLUSTER_IP='${IP}' + export CEPH_OSD_DATA_DIR='${dataDir}' + /bin/bash /tmp/stackube_install/deploy_ceph_osd.sh" +done + +docker exec stackube_ceph_mon ceph -s + + + diff --git a/install/deploy_kubernetes.sh b/install/deploy_kubernetes.sh new file mode 100644 index 0000000..6b23617 --- /dev/null +++ b/install/deploy_kubernetes.sh @@ -0,0 +1,148 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +source $(readlink -f $1) + +[ "${CONTROL_NODE_PUBLIC_IP}" ] +[ "${CONTROL_NODE_PRIVATE_IP}" ] +[ "${NETWORK_NODES_PRIVATE_IP}" ] +[ "${COMPUTE_NODES_PRIVATE_IP}" ] + + +export KUBERNETES_API_PUBLIC_IP="${CONTROL_NODE_PUBLIC_IP}" +export KUBERNETES_API_PRIVATE_IP="${CONTROL_NODE_PRIVATE_IP}" +export KEYSTONE_URL="https://${CONTROL_NODE_PRIVATE_IP}:5001/v2.0" +export KEYSTONE_ADMIN_URL="https://${CONTROL_NODE_PRIVATE_IP}:35358/v2.0" +export CLUSTER_CIDR="10.244.0.0/16" +export CLUSTER_GATEWAY="10.244.0.1" +export CONTAINER_CIDR="10.244.1.0/24" +export FRAKTI_VERSION="v1.0" + + +########## control & compute nodes ########## + +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +# hyperd frakti +for IP in ${allIpList}; do + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/kubernetes/deploy_hyperd_frakti.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "export FRAKTI_VERSION='${FRAKTI_VERSION}' + export STREAMING_SERVER_ADDR='${IP}' + /bin/bash /tmp/stackube_install/deploy_hyperd_frakti.sh" +done + +# kubeadm kubectl kubelet +for IP in ${allIpList}; do + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/kubernetes/deploy_kubeadm_kubectl_kubelet.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/deploy_kubeadm_kubectl_kubelet.sh" +done + + + +########## control node ########## + +# kubernetes master +sed -i "s|__KEYSTONE_URL__|${KEYSTONE_URL}|g" ${programDir}/kubernetes/kubeadm.yaml +sed -i "s|__POD_NET_CIDR__|${CLUSTER_CIDR}|g" ${programDir}/kubernetes/kubeadm.yaml +sed -i "s/__KUBERNETES_API_PUBLIC_IP__/${KUBERNETES_API_PUBLIC_IP}/g" ${programDir}/kubernetes/kubeadm.yaml +sed -i "s/__KUBERNETES_API_PRIVATE_IP__/${KUBERNETES_API_PRIVATE_IP}/g" ${programDir}/kubernetes/kubeadm.yaml +/bin/bash ${programDir}/kubernetes/deploy_kubernetes_init_master.sh +sleep 3 + + + +export KUBECONFIG=/etc/kubernetes/admin.conf + + +# install stackube addons +/bin/bash ${programDir}/kubernetes/deploy_kubernetes_install_stackube_addons.sh +sleep 10 + + +# add nodes +KUBEADM_TOKEN=`kubeadm token list | grep 'kubeadm init' | head -1 | awk '{print $1}'` +allIpList=`echo " +${COMPUTE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq | grep -v "${CONTROL_NODE_PRIVATE_IP}"` +for IP in ${allIpList}; do + ssh root@${IP} "kubeadm join --token ${KUBEADM_TOKEN} ${CONTROL_NODE_PRIVATE_IP}:6443" +done + + +# Enable schedule pods on the master (control node) if it's also designated as a compute node +set +e +check=`echo " +${COMPUTE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq | grep "${CONTROL_NODE_PRIVATE_IP}" ` +if [ "${check}" ]; then + kubectl taint nodes $(hostname) node-role.kubernetes.io/master- +fi +set -e + + +# certificate approve +sleep 5 +/bin/bash ${programDir}/kubernetes/deploy_kubernetes_certificate_approve.sh + + + +## check +sleep 3 +kubectl get nodes +kubectl get csr --all-namespaces + + + + +########## control (k8s master) & compute nodes ########### + +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +# install ovs for cni +for IP in ${allIpList}; do + ssh root@${IP} "yum install centos-release-openstack-ocata.noarch -y" + ssh root@${IP} "yum install openvswitch -y" +done + +# install ceph for kubelet +for IP in ${allIpList}; do + ssh root@${IP} "yum install centos-release-openstack-ocata.noarch -y" + ssh root@${IP} "yum install ceph -y" + ssh root@${IP} "systemctl disable ceph.target ceph-mds.target ceph-mon.target ceph-osd.target" + scp -r /var/lib/stackube/ceph/ceph_mon_config/* root@${IP}:/etc/ceph/ + ssh root@${IP} "ceph -s" + ssh root@${IP} "rbd -p cinder --id cinder --keyring=/etc/ceph/ceph.client.cinder.keyring ls" +done + + + + +exit 0 diff --git a/install/deploy_openstack.sh b/install/deploy_openstack.sh new file mode 100644 index 0000000..071eeab --- /dev/null +++ b/install/deploy_openstack.sh @@ -0,0 +1,280 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +source $(readlink -f $1) + +[ "${CONTROL_NODE_PRIVATE_IP}" ] + +[ "${NETWORK_NODES_PRIVATE_IP}" ] +#[ "${NETWORK_NODES_NEUTRON_EXT_IF}" ] + +[ "${NEUTRON_PUBLIC_SUBNET}" ] + +[ "${COMPUTE_NODES_PRIVATE_IP}" ] + +[ "${STORAGE_NODES_PRIVATE_IP}" ] +[ "${STORAGE_NODES_CEPH_OSD_DATA_DIR}" ] + + +export OPENSTACK_ENDPOINT_IP="${CONTROL_NODE_PRIVATE_IP}" +export KEYSTONE_API_IP="${CONTROL_NODE_PRIVATE_IP}" +export NEUTRON_API_IP="${CONTROL_NODE_PRIVATE_IP}" +export CINDER_API_IP="${CONTROL_NODE_PRIVATE_IP}" + +export MYSQL_HOST="${CONTROL_NODE_PRIVATE_IP}" +export MYSQL_ROOT_PWD=${MYSQL_ROOT_PWD:-MysqlRoot123} +export MYSQL_KEYSTONE_PWD=${MYSQL_KEYSTONE_PWD:-MysqlKeystone123} +export MYSQL_NEUTRON_PWD=${MYSQL_NEUTRON_PWD:-MysqlNeutron123} +export MYSQL_CINDER_PWD=${MYSQL_CINDER_PWD:-MysqlCinder123} + +export RABBITMQ_HOST="${CONTROL_NODE_PRIVATE_IP}" +export RABBITMQ_PWD=${RABBITMQ_PWD:-rabbitmq123} + +export KEYSTONE_ADMIN_PWD=${KEYSTONE_ADMIN_PWD:-KeystoneAdmin123} +export KEYSTONE_NEUTRON_PWD=${KEYSTONE_NEUTRON_PWD:-KeystoneNeutron123} +export KEYSTONE_CINDER_PWD=${KEYSTONE_CINDER_PWD:-KeystoneCinder123} + + + + +########## all nodes ########## + +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${NETWORK_NODES_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP} +${STORAGE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +# kolla-toolbox +for IP in ${allIpList}; do + ssh root@${IP} 'mkdir -p /etc/stackube/openstack /tmp/stackube_install' + scp -r ${programDir}/openstack/config_openstack/kolla-toolbox root@${IP}:/etc/stackube/openstack/ + + scp ${programDir}/openstack/deploy_openstack_kolla_toolbox.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/deploy_openstack_kolla_toolbox.sh" +done + + + +########## control node ########## + +# db, mq, haproxy +/bin/bash ${programDir}/openstack/deploy_openstack_mariadb.sh +/bin/bash ${programDir}/openstack/deploy_openstack_rabbitmq.sh +/bin/bash ${programDir}/openstack/deploy_openstack_haproxy.sh + +# keystone +/bin/bash ${programDir}/openstack/deploy_openstack_keystone.sh + + +# neutron server +function process_neutron_conf { + local configFile="$1" + sed -i "s/__RABBITMQ_HOST__/${RABBITMQ_HOST}/g" ${configFile} + sed -i "s/__RABBITMQ_PWD__/${RABBITMQ_PWD}/g" ${configFile} + sed -i "s/__NEUTRON_API_IP__/${NEUTRON_API_IP}/g" ${configFile} + sed -i "s/__MYSQL_HOST__/${MYSQL_HOST}/g" ${configFile} + sed -i "s/__OPENSTACK_ENDPOINT_IP__/${OPENSTACK_ENDPOINT_IP}/g" ${configFile} + sed -i "s/__KEYSTONE_NEUTRON_PWD__/${KEYSTONE_NEUTRON_PWD}/g" ${configFile} + sed -i "s/__MYSQL_NEUTRON_PWD__/${MYSQL_NEUTRON_PWD}/g" ${configFile} +} + +mkdir -p /etc/stackube/openstack +cp -a ${programDir}/openstack/config_openstack/neutron-server /etc/stackube/openstack/ +process_neutron_conf /etc/stackube/openstack/neutron-server/neutron.conf + +source /etc/stackube/openstack/admin-openrc.sh +cp -f ${OS_CACERT} /etc/stackube/openstack/neutron-server/haproxy-ca.crt + +/bin/bash ${programDir}/openstack/deploy_openstack_neutron_server.sh + + +## cinder api +function process_cinder_conf { + local cinderConfigFile="$1" + sed -i "s/__CINDER_API_IP__/${CINDER_API_IP}/g" ${cinderConfigFile} + sed -i "s/__RABBITMQ_HOST__/${RABBITMQ_HOST}/g" ${cinderConfigFile} + sed -i "s/__RABBITMQ_PWD__/${RABBITMQ_PWD}/g" ${cinderConfigFile} + sed -i "s/__MYSQL_CINDER_PWD__/${MYSQL_CINDER_PWD}/g" ${cinderConfigFile} + sed -i "s/__MYSQL_HOST__/${MYSQL_HOST}/g" ${cinderConfigFile} + sed -i "s/__OPENSTACK_ENDPOINT_IP__/${OPENSTACK_ENDPOINT_IP}/g" ${cinderConfigFile} + sed -i "s/__KEYSTONE_CINDER_PWD__/${KEYSTONE_CINDER_PWD}/g" ${cinderConfigFile} +} +mkdir -p /etc/stackube/openstack +cp -a ${programDir}/openstack/config_openstack/cinder-api /etc/stackube/openstack/ +process_cinder_conf /etc/stackube/openstack/cinder-api/cinder.conf + +source /etc/stackube/openstack/admin-openrc.sh +cp -f ${OS_CACERT} /etc/stackube/openstack/cinder-api/haproxy-ca.crt + +/bin/bash ${programDir}/openstack/deploy_openstack_cinder_api.sh + + +# cinder scheduler +mkdir -p /etc/stackube/openstack +cp -a ${programDir}/openstack/config_openstack/cinder-scheduler /etc/stackube/openstack/ +cp -f /etc/stackube/openstack/cinder-api/cinder.conf /etc/stackube/openstack/cinder-scheduler/ +/bin/bash ${programDir}/openstack/deploy_openstack_cinder_scheduler.sh + + +# cinder volume +docker exec stackube_ceph_mon ceph osd pool create cinder 128 128 +docker exec stackube_ceph_mon ceph auth get-or-create client.cinder mon 'allow r' \ + osd 'allow class-read object_prefix rbd_children, allow rwx pool=cinder' +docker exec stackube_ceph_mon /bin/bash -c 'ceph auth get-or-create client.cinder | tee /etc/ceph/ceph.client.cinder.keyring' + +for IP in ${CONTROL_NODE_PRIVATE_IP} ; do + ssh root@${IP} 'mkdir -p /etc/stackube/openstack /tmp/stackube_install' + scp -r ${programDir}/openstack/config_openstack/cinder-volume root@${IP}:/etc/stackube/openstack/ + scp -r /etc/stackube/openstack/cinder-api/cinder.conf \ + /var/lib/stackube/ceph/ceph_mon_config/{ceph.conf,ceph.client.cinder.keyring} root@${IP}:/etc/stackube/openstack/cinder-volume/ + + scp ${programDir}/openstack/deploy_openstack_cinder_volume.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/deploy_openstack_cinder_volume.sh" +done + + + + +########## network nodes ########## + +# neutron l3_agent +for IP in `echo ${NETWORK_NODES_PRIVATE_IP} | sed -e 's/,/ /g' ` ; do + ssh root@${IP} 'mkdir -p /etc/stackube/openstack /tmp/stackube_install' + scp -r ${programDir}/openstack/config_openstack/neutron-l3-agent root@${IP}:/etc/stackube/openstack/ + scp -r /etc/stackube/openstack/neutron-server/neutron.conf \ + ${programDir}/openstack/config_openstack/neutron-server/ml2_conf.ini root@${IP}:/etc/stackube/openstack/neutron-l3-agent/ + + scp ${programDir}/openstack/deploy_openstack_neutron_l3_agent.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "export OVSDB_IP='${IP}' + export ML2_LOCAL_IP='${IP}' + /bin/bash /tmp/stackube_install/deploy_openstack_neutron_l3_agent.sh" +done + + +# neutron dhcp_agent +for IP in `echo ${NETWORK_NODES_PRIVATE_IP} | sed -e 's/,/ /g' ` ; do + ssh root@${IP} 'mkdir -p /etc/stackube/openstack /tmp/stackube_install' + scp -r ${programDir}/openstack/config_openstack/neutron-dhcp-agent root@${IP}:/etc/stackube/openstack/ + scp -r /etc/stackube/openstack/neutron-server/neutron.conf \ + ${programDir}/openstack/config_openstack/neutron-server/ml2_conf.ini root@${IP}:/etc/stackube/openstack/neutron-dhcp-agent/ + + scp ${programDir}/openstack/deploy_openstack_neutron_dhcp_agent.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "export OVSDB_IP='${IP}' + export ML2_LOCAL_IP='${IP}' + /bin/bash /tmp/stackube_install/deploy_openstack_neutron_dhcp_agent.sh" +done + + +# neutron lbaas_agent +for IP in `echo ${NETWORK_NODES_PRIVATE_IP} | sed -e 's/,/ /g' ` ; do + ssh root@${IP} 'mkdir -p /etc/stackube/openstack /tmp/stackube_install' + scp -r ${programDir}/openstack/config_openstack/neutron-lbaas-agent root@${IP}:/etc/stackube/openstack/ + scp -r /etc/stackube/openstack/neutron-server/neutron.conf \ + ${programDir}/openstack/config_openstack/neutron-server/{ml2_conf.ini,neutron_lbaas.conf} root@${IP}:/etc/stackube/openstack/neutron-lbaas-agent/ + + scp ${programDir}/openstack/deploy_openstack_neutron_lbaas_agent.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "export OVSDB_IP='${IP}' + export ML2_LOCAL_IP='${IP}' + export KEYSTONE_API_IP='${KEYSTONE_API_IP}' + export KEYSTONE_NEUTRON_PWD='${KEYSTONE_NEUTRON_PWD}' + /bin/bash /tmp/stackube_install/deploy_openstack_neutron_lbaas_agent.sh" +done + + + + + +########## control & network & compute nodes ########## + +# openvswitch agent (deploy on control node for k8s master) +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${NETWORK_NODES_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` +for IP in ${allIpList}; do + ssh root@${IP} 'mkdir -p /etc/stackube/openstack /tmp/stackube_install' + scp -r ${programDir}/openstack/config_openstack/{openvswitch-db-server,openvswitch-vswitchd,neutron-openvswitch-agent} root@${IP}:/etc/stackube/openstack/ + scp -r /etc/stackube/openstack/neutron-server/neutron.conf ${programDir}/openstack/config_openstack/neutron-server/ml2_conf.ini root@${IP}:/etc/stackube/openstack/neutron-openvswitch-agent/ + + scp ${programDir}/openstack/deploy_openstack_neutron_openvswitch_agent.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "export OVSDB_IP='${IP}' + export ML2_LOCAL_IP='${IP}' + /bin/bash /tmp/stackube_install/deploy_openstack_neutron_openvswitch_agent.sh" +done + +# network nodes: NEUTRON_EXT_IF +networkIpList=(`echo "${NETWORK_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g'`) +neutronExtIfList=(`echo "${NETWORK_NODES_NEUTRON_EXT_IF}" | sed -e 's/,/\n/g'`) +[ ${#networkIpList[@]} -eq ${#neutronExtIfList[@]} ] +MAX=$((${#networkIpList[@]} - 1)) +for i in `seq 0 ${MAX}`; do + IP="${networkIpList[$i]}" + extIf="${neutronExtIfList[$i]}" + echo -e "\n------ ${IP} ${extIf} ------" + ssh root@${IP} "docker exec stackube_openstack_openvswitch_db /usr/local/bin/kolla_ensure_openvswitch_configured br-ex ${extIf}" +done + + + + + +######### compute node ############ + +# certificate for kubestack +allIpList=`echo " +${COMPUTE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` +for IP in ${allIpList}; do + scp -r /etc/stackube/openstack/certificates/CA/int-ca/ca-chain.pem root@${IP}:/usr/share/pki/ca-trust-source/anchors/stackube-chain.pem + ssh root@${IP} "update-ca-trust" +done + + + + + +######### control node ############ + +# create public network and subnet +yum install centos-release-openstack-ocata.noarch -y +yum install python-openstackclient -y + +source /etc/stackube/openstack/admin-openrc.sh +openstack network create --external --provider-physical-network physnet1 --provider-network-type flat public_1 + +# NEUTRON_PUBLIC_SUBNET='subnet-range;gateway;allocation-pool' +SUBNET=`echo "${NEUTRON_PUBLIC_SUBNET}" | awk -F\; '{print $1}'` +GATEWAY=`echo "${NEUTRON_PUBLIC_SUBNET}" | awk -F\; '{print $2}'` +POOL=`echo "${NEUTRON_PUBLIC_SUBNET}" | awk -F\; '{print $3}'` +openstack subnet create public_1-subnet_1 \ + --subnet-range "${SUBNET}" --gateway "${GATEWAY}" --allocation-pool "${POOL}" --no-dhcp --network public_1 + + +# check +openstack network list +openstack subnet list +openstack endpoint list diff --git a/install/ensure_distro_supported.sh b/install/ensure_distro_supported.sh new file mode 100644 index 0000000..c998a6a --- /dev/null +++ b/install/ensure_distro_supported.sh @@ -0,0 +1,42 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + + +source ${programDir}/lib_common.sh || { echo "Error: 'source ${programDir}/lib_common.sh' failed!"; exit 1; } + +MSG='Sorry, only CentOS 7.x supported for now.' + +if ! is_fedora; then + echo ${MSG} + exit 1 +fi + +mainVersion=`echo ${os_RELEASE} | awk -F\. '{print $1}' ` +if [ "${os_VENDOR}" == "CentOS" ] && [ "${mainVersion}" == "7" ]; then + true +else + echo ${MSG} + exit 1 +fi + + +exit 0 + diff --git a/install/ensure_docker_installed.sh b/install/ensure_docker_installed.sh new file mode 100644 index 0000000..b0690c8 --- /dev/null +++ b/install/ensure_docker_installed.sh @@ -0,0 +1,48 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +set -x + +systemctl start docker &> /dev/null + +sleep 2 + +docker info &> /dev/null + +if [ "$?" != "0" ]; then + cat > /etc/yum.repos.d/docker.repo << EOF +[docker-repo] +name=Docker main Repository +baseurl=https://yum.dockerproject.org/repo/main/centos/7 +enabled=1 +gpgcheck=1 +gpgkey=https://yum.dockerproject.org/gpg +EOF + yum install docker-engine-1.12.6 docker-engine-selinux-1.12.6 -y || exit 1 + #sed -i 's|ExecStart=.*|ExecStart=/usr/bin/dockerd --storage-opt dm.mountopt=nodiscard --storage-opt dm.blkdiscard=false|g' /usr/lib/systemd/system/docker.service + sed -i 's|ExecStart=.*|ExecStart=/usr/bin/dockerd -s overlay |g' /usr/lib/systemd/system/docker.service + systemctl daemon-reload || exit 1 + systemctl enable docker || exit 1 + systemctl start docker || exit 1 +fi + +sleep 5 + +docker info &> /dev/null || exit 1 + + +exit 0 + diff --git a/install/kubernetes/deploy_hyperd_frakti.sh b/install/kubernetes/deploy_hyperd_frakti.sh new file mode 100644 index 0000000..13a37b6 --- /dev/null +++ b/install/kubernetes/deploy_hyperd_frakti.sh @@ -0,0 +1,112 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``STREAMING_SERVER_ADDR`` +# - ``FRAKTI_VERSION`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## install libvirtd +yum install -y libvirt + + +## install hyperd +CENTOS7_QEMU_HYPER="http://hypercontainer-install.s3.amazonaws.com/qemu-hyper-2.4.1-3.el7.centos.x86_64.rpm" +CENTOS7_HYPERSTART="https://s3-us-west-1.amazonaws.com/hypercontainer-build/1.0-rc2/centos/hyperstart-0.8.1-1.el7.centos.x86_64.rpm" +CENTOS7_HYPER="https://s3-us-west-1.amazonaws.com/hypercontainer-build/1.0-rc2/centos/hyper-container-0.8.1-1.el7.centos.x86_64.rpm" + +if rpm -qa | grep "hyper-container-0.8.1-1.el7.centos.x86_64" ; then + true +else + set -e + yum install -y ${CENTOS7_QEMU_HYPER} ${CENTOS7_HYPERSTART} ${CENTOS7_HYPER} + set +e +fi +set -e + +cat > /etc/hyper/config << EOF +Kernel=/var/lib/hyper/kernel +Initrd=/var/lib/hyper/hyper-initrd.img +Hypervisor=qemu +StorageDriver=overlay +gRPCHost=127.0.0.1:22318 + +EOF + + +## install frakti +set +e +[ -f /usr/bin/frakti ] && rm -f /usr/bin/frakti +set -e +curl -sSL https://github.com/kubernetes/frakti/releases/download/${FRAKTI_VERSION}/frakti -o /usr/bin/frakti +chmod +x /usr/bin/frakti + +dockerInfo=`docker info ` +cgroup_driver=`echo "${dockerInfo}" | awk '/Cgroup Driver/{print $3}' ` +[ "${cgroup_driver}" ] + +echo "[Unit] +Description=Hypervisor-based container runtime for Kubernetes +Documentation=https://github.com/kubernetes/frakti +After=network.target +[Service] +ExecStart=/usr/bin/frakti --v=3 \ + --log-dir=/var/log/frakti \ + --logtostderr=false \ + --cgroup-driver=${cgroup_driver} \ + --listen=/var/run/frakti.sock \ + --streaming-server-addr=${STREAMING_SERVER_ADDR} \ + --hyper-endpoint=127.0.0.1:22318 +MountFlags=shared +#TasksMax=8192 +LimitNOFILE=1048576 +LimitNPROC=1048576 +LimitCORE=infinity +TimeoutStartSec=0 +Restart=on-abnormal +[Install] +WantedBy=multi-user.target +" > /lib/systemd/system/frakti.service + + +## start services +systemctl daemon-reload +systemctl enable hyperd frakti libvirtd +systemctl restart hyperd libvirtd +sleep 5 +systemctl restart frakti +sleep 5 + +## check +hyperctl list +pgrep -f '/usr/bin/frakti' +[ -e /var/run/frakti.sock ] + + + +exit 0 diff --git a/install/kubernetes/deploy_kubeadm_kubectl_kubelet.sh b/install/kubernetes/deploy_kubeadm_kubectl_kubelet.sh new file mode 100644 index 0000000..0c3dd49 --- /dev/null +++ b/install/kubernetes/deploy_kubeadm_kubectl_kubelet.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + + +setenforce 0 +sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config + + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## install kubeadm kubectl kubelet +cat > /etc/yum.repos.d/kubernetes.repo << EOF +[kubernetes] +name=Kubernetes +baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64 +enabled=1 +gpgcheck=1 +repo_gpgcheck=1 +gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg + https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg +EOF + +yum install -y kubelet-1.7.4-0 kubeadm-1.7.4-0 kubectl-1.7.4-0 + +# configure_kubelet +unitFile='/etc/systemd/system/kubelet.service.d/10-kubeadm.conf' +sed -i '/^Environment="KUBELET_EXTRA_ARGS=/d' ${unitFile} +sed -i '/\[Service\]/aEnvironment="KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=/var/run/frakti.sock --feature-gates=AllAlpha=true"' ${unitFile} + + +systemctl daemon-reload +systemctl enable kubelet + + + +exit 0 diff --git a/install/kubernetes/deploy_kubernetes_certificate_approve.sh b/install/kubernetes/deploy_kubernetes_certificate_approve.sh new file mode 100644 index 0000000..327dd15 --- /dev/null +++ b/install/kubernetes/deploy_kubernetes_certificate_approve.sh @@ -0,0 +1,41 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -x + + +export KUBECONFIG=/etc/kubernetes/admin.conf + +for i in `seq 1 30`; do + aaa=`kubectl get csr --all-namespaces | grep Pending | awk '{print $1}'` + if [ "$aaa" ]; then + for i in $aaa; do + kubectl certificate approve $i || exit 1 + done + sleep 5 + else + break + fi +done + + +exit 0 diff --git a/install/kubernetes/deploy_kubernetes_init_master.sh b/install/kubernetes/deploy_kubernetes_init_master.sh new file mode 100644 index 0000000..afd3424 --- /dev/null +++ b/install/kubernetes/deploy_kubernetes_init_master.sh @@ -0,0 +1,31 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +kubeadm init --config ${programDir}/kubeadm.yaml + + +exit 0 diff --git a/install/kubernetes/deploy_kubernetes_install_stackube_addons.sh b/install/kubernetes/deploy_kubernetes_install_stackube_addons.sh new file mode 100644 index 0000000..2b40e02 --- /dev/null +++ b/install/kubernetes/deploy_kubernetes_install_stackube_addons.sh @@ -0,0 +1,79 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``KUBERNETES_API_PUBLIC_IP`` +# - ``CLUSTER_CIDR``, ``CLUSTER_GATEWAY``, +# - ``KEYSTONE_ADMIN_URL`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## install stackube addons +kubectl -n kube-system delete deployment kube-dns +kubectl -n kube-system delete daemonset kube-proxy + +source /etc/stackube/openstack/admin-openrc.sh +netList=`openstack network list --long -f value` +public_network=$(echo "${netList}" | grep External | grep ' public_1 ' | awk '{print $1}') +[ "${public_network}" ] +nnn=`echo "${public_network}" | wc -l` +[ $nnn -eq 1 ] + +cinderKeyring=`cat /var/lib/stackube/ceph/ceph_mon_config/ceph.client.cinder.keyring` +keyring=`echo "${cinderKeyring}" | grep 'key = ' | awk -F\ \=\ '{print $2}'` +[ "${keyring}" ] + +cat > ${programDir}/stackube-configmap.yaml < /dev/null 2>&1; then + kubeadm reset || exit 1 +fi + + +systemctl stop hyperd kubelet +yum remove -y kubelet kubeadm kubectl qemu-hyper hyperstart hyper-container || exit 1 +rm -fr /etc/kubernetes /var/lib/kubelet /var/run/kubernetes + +systemctl stop frakti +rm -f /usr/bin/frakti /lib/systemd/system/frakti.service || exit 1 +systemctl daemon-reload + + + +exit 0 + diff --git a/install/lib_common.sh b/install/lib_common.sh new file mode 100644 index 0000000..aaddac8 --- /dev/null +++ b/install/lib_common.sh @@ -0,0 +1,196 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# Distro Functions +# ================ + +# Determine OS Vendor, Release and Update + +# +# NOTE : For portability, you almost certainly do not want to use +# these variables directly! The "is_*" functions defined below this +# bundle up compatible platforms under larger umbrellas that we have +# determinted are compatible enough (e.g. is_ubuntu covers Ubuntu & +# Debian, is_fedora covers RPM-based distros). Higher-level functions +# such as "install_package" further abstract things in better ways. +# +# ``os_VENDOR`` - vendor name: ``Ubuntu``, ``Fedora``, etc +# ``os_RELEASE`` - major release: ``16.04`` (Ubuntu), ``23`` (Fedora) +# ``os_PACKAGE`` - package type: ``deb`` or ``rpm`` +# ``os_CODENAME`` - vendor's codename for release: ``xenial`` + +declare -g os_VENDOR os_RELEASE os_PACKAGE os_CODENAME + +# Make a *best effort* attempt to install lsb_release packages for the +# user if not available. Note can't use generic install_package* +# because they depend on this! +function _ensure_lsb_release { + if [[ -x $(command -v lsb_release 2>/dev/null) ]]; then + return + fi + + if [[ -x $(command -v apt-get 2>/dev/null) ]]; then + sudo apt-get install -y lsb-release + elif [[ -x $(command -v zypper 2>/dev/null) ]]; then + # XXX: old code paths seem to have assumed SUSE platforms also + # had "yum". Keep this ordered above yum so we don't try to + # install the rh package. suse calls it just "lsb" + sudo zypper -n install lsb + elif [[ -x $(command -v dnf 2>/dev/null) ]]; then + sudo dnf install -y redhat-lsb-core + elif [[ -x $(command -v yum 2>/dev/null) ]]; then + # all rh patforms (fedora, centos, rhel) have this pkg + sudo yum install -y redhat-lsb-core + else + die $LINENO "Unable to find or auto-install lsb_release" + fi +} + +# GetOSVersion +# Set the following variables: +# - os_RELEASE +# - os_CODENAME +# - os_VENDOR +# - os_PACKAGE +function GetOSVersion { + # We only support distros that provide a sane lsb_release + _ensure_lsb_release + + os_RELEASE=$(lsb_release -r -s) + os_CODENAME=$(lsb_release -c -s) + os_VENDOR=$(lsb_release -i -s) + + if [[ $os_VENDOR =~ (Debian|Ubuntu|LinuxMint) ]]; then + os_PACKAGE="deb" + else + os_PACKAGE="rpm" + fi + + typeset -xr os_VENDOR + typeset -xr os_RELEASE + typeset -xr os_PACKAGE + typeset -xr os_CODENAME +} + +# Translate the OS version values into common nomenclature +# Sets global ``DISTRO`` from the ``os_*`` values +declare -g DISTRO + +function GetDistro { + GetOSVersion + if [[ "$os_VENDOR" =~ (Ubuntu) || "$os_VENDOR" =~ (Debian) || \ + "$os_VENDOR" =~ (LinuxMint) ]]; then + # 'Everyone' refers to Ubuntu / Debian / Mint releases by + # the code name adjective + DISTRO=$os_CODENAME + elif [[ "$os_VENDOR" =~ (Fedora) ]]; then + # For Fedora, just use 'f' and the release + DISTRO="f$os_RELEASE" + elif [[ "$os_VENDOR" =~ (openSUSE) ]]; then + DISTRO="opensuse-$os_RELEASE" + elif [[ "$os_VENDOR" =~ (SUSE LINUX) ]]; then + # just use major release + DISTRO="sle${os_RELEASE%.*}" + elif [[ "$os_VENDOR" =~ (Red.*Hat) || \ + "$os_VENDOR" =~ (CentOS) || \ + "$os_VENDOR" =~ (Scientific) || \ + "$os_VENDOR" =~ (OracleServer) || \ + "$os_VENDOR" =~ (Virtuozzo) ]]; then + # Drop the . release as we assume it's compatible + # XXX re-evaluate when we get RHEL10 + DISTRO="rhel${os_RELEASE::1}" + elif [[ "$os_VENDOR" =~ (XenServer) ]]; then + DISTRO="xs${os_RELEASE%.*}" + elif [[ "$os_VENDOR" =~ (kvmibm) ]]; then + DISTRO="${os_VENDOR}${os_RELEASE::1}" + else + # We can't make a good choice here. Setting a sensible DISTRO + # is part of the problem, but not the major issue -- we really + # only use DISTRO in the code as a fine-filter. + # + # The bigger problem is categorising the system into one of + # our two big categories as Ubuntu/Debian-ish or + # Fedora/CentOS-ish. + # + # The setting of os_PACKAGE above is only set to "deb" based + # on a hard-coded list of vendor names ... thus we will + # default to thinking unknown distros are RPM based + # (ie. is_ubuntu does not match). But the platform will then + # also not match in is_fedora, because that also has a list of + # names. + # + # So, if you are reading this, getting your distro supported + # is really about making sure it matches correctly in these + # functions. Then you can choose a sensible way to construct + # DISTRO based on your distros release approach. + die $LINENO "Unable to determine DISTRO, can not continue." + fi + typeset -xr DISTRO +} + +# Utility function for checking machine architecture +# is_arch arch-type +function is_arch { + [[ "$(uname -m)" == "$1" ]] +} + +# Determine if current distribution is an Oracle distribution +# is_oraclelinux +function is_oraclelinux { + if [[ -z "$os_VENDOR" ]]; then + GetOSVersion + fi + + [ "$os_VENDOR" = "OracleServer" ] +} + + +# Determine if current distribution is a Fedora-based distribution +# (Fedora, RHEL, CentOS, etc). +# is_fedora +function is_fedora { + if [[ -z "$os_VENDOR" ]]; then + GetOSVersion + fi + + [ "$os_VENDOR" = "Fedora" ] || [ "$os_VENDOR" = "Red Hat" ] || \ + [ "$os_VENDOR" = "RedHatEnterpriseServer" ] || \ + [ "$os_VENDOR" = "CentOS" ] || [ "$os_VENDOR" = "OracleServer" ] || \ + [ "$os_VENDOR" = "Virtuozzo" ] || [ "$os_VENDOR" = "kvmibm" ] +} + + +# Determine if current distribution is a SUSE-based distribution +# (openSUSE, SLE). +# is_suse +function is_suse { + if [[ -z "$os_VENDOR" ]]; then + GetOSVersion + fi + + [[ "$os_VENDOR" =~ (openSUSE) || "$os_VENDOR" == "SUSE LINUX" ]] +} + + +# Determine if current distribution is an Ubuntu-based distribution +# It will also detect non-Ubuntu but Debian-based distros +# is_ubuntu +function is_ubuntu { + if [[ -z "$os_PACKAGE" ]]; then + GetOSVersion + fi + [ "$os_PACKAGE" = "deb" ] +} diff --git a/install/lib_tls.sh b/install/lib_tls.sh new file mode 100644 index 0000000..76f7b61 --- /dev/null +++ b/install/lib_tls.sh @@ -0,0 +1,377 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# !! source _before_ any services that use ``SERVICE_HOST`` +# +# Dependencies: +# +# - ``DEST``, ``DATA_DIR`` must be defined +# - ``HOST_IP``, ``SERVICE_HOST`` +# - ``KEYSTONE_TOKEN_FORMAT`` must be defined + +# Entry points: +# +# - configure_CA +# - init_CA +# - cleanup_CA +# - make_root_CA +# - make_int_CA +# - make_cert ca-dir cert-name "common-name" ["alt-name" ...] + + + +# Defaults +# -------- + +# TODO: support more distributions +function is_fedora { + # Always true + return 0 +} + +# Check if this is a valid ipv4 address string +function is_ipv4_address { + local address=$1 + local regex='([0-9]{1,3}.){3}[0-9]{1,3}' + # TODO(clarkb) make this more robust + if [[ "$address" =~ $regex ]] ; then + return 0 + else + return 1 + fi +} + + +SSL_BUNDLE_FILE="$DATA_DIR/ca-bundle.pem" +TLS_IP=${TLS_IP:-$SERVICE_IP} + +STACKUBE_HOSTNAME=$(hostname -f) +STACKUBE_CERT_NAME=stackube-cert +STACKUBE_CERT=$DATA_DIR/$STACKUBE_CERT_NAME.pem + +# CA configuration +ROOT_CA_DIR=${ROOT_CA_DIR:-$DATA_DIR/CA/root-ca} +INT_CA_DIR=${INT_CA_DIR:-$DATA_DIR/CA/int-ca} + +ORG_NAME="OpenStack" +ORG_UNIT_NAME="Stackube" + + +# CA Functions +# ============ + +# There may be more than one, get specific +OPENSSL=${OPENSSL:-/usr/bin/openssl} + +# Do primary CA configuration +function configure_CA { + # build common config file + + # Verify ``TLS_IP`` is good + if [[ -n "$HOST_IP" && "$HOST_IP" != "$TLS_IP" ]]; then + # auto-discover has changed the IP + TLS_IP=$HOST_IP + fi +} + +# Creates a new CA directory structure +# create_CA_base ca-dir +function create_CA_base { + local ca_dir=$1 + + if [[ -d $ca_dir ]]; then + # Bail out it exists + return 0 + fi + + local i + for i in certs crl newcerts private; do + mkdir -p $ca_dir/$i + done + chmod 710 $ca_dir/private + echo "01" >$ca_dir/serial + cp /dev/null $ca_dir/index.txt +} + +# Create a new CA configuration file +# create_CA_config ca-dir common-name +function create_CA_config { + local ca_dir=$1 + local common_name=$2 + + echo " +[ ca ] +default_ca = CA_default + +[ CA_default ] +dir = $ca_dir +policy = policy_match +database = \$dir/index.txt +serial = \$dir/serial +certs = \$dir/certs +crl_dir = \$dir/crl +new_certs_dir = \$dir/newcerts +certificate = \$dir/cacert.pem +private_key = \$dir/private/cacert.key +RANDFILE = \$dir/private/.rand +default_md = sha256 + +[ req ] +default_bits = 2048 +default_md = sha256 + +prompt = no +distinguished_name = ca_distinguished_name + +x509_extensions = ca_extensions + +[ ca_distinguished_name ] +organizationName = $ORG_NAME +organizationalUnitName = $ORG_UNIT_NAME Certificate Authority +commonName = $common_name + +[ policy_match ] +countryName = optional +stateOrProvinceName = optional +organizationName = match +organizationalUnitName = optional +commonName = supplied + +[ ca_extensions ] +basicConstraints = critical,CA:true +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid:always, issuer +keyUsage = cRLSign, keyCertSign + +" >$ca_dir/ca.conf +} + +# Create a new signing configuration file +# create_signing_config ca-dir +function create_signing_config { + local ca_dir=$1 + + echo " +[ ca ] +default_ca = CA_default + +[ CA_default ] +dir = $ca_dir +policy = policy_match +database = \$dir/index.txt +serial = \$dir/serial +certs = \$dir/certs +crl_dir = \$dir/crl +new_certs_dir = \$dir/newcerts +certificate = \$dir/cacert.pem +private_key = \$dir/private/cacert.key +RANDFILE = \$dir/private/.rand +default_md = default + +[ req ] +default_bits = 1024 +default_md = sha1 + +prompt = no +distinguished_name = req_distinguished_name + +x509_extensions = req_extensions + +[ req_distinguished_name ] +organizationName = $ORG_NAME +organizationalUnitName = $ORG_UNIT_NAME Server Farm + +[ policy_match ] +countryName = optional +stateOrProvinceName = optional +organizationName = match +organizationalUnitName = optional +commonName = supplied + +[ req_extensions ] +basicConstraints = CA:false +subjectKeyIdentifier = hash +authorityKeyIdentifier = keyid:always, issuer +keyUsage = digitalSignature, keyEncipherment, keyAgreement +extendedKeyUsage = serverAuth, clientAuth +subjectAltName = \$ENV::SUBJECT_ALT_NAME + +" >$ca_dir/signing.conf +} + +# Create root and intermediate CAs +# init_CA +function init_CA { + # Ensure CAs are built + make_root_CA $ROOT_CA_DIR + make_int_CA $INT_CA_DIR $ROOT_CA_DIR + + # Create the CA bundle + cat $ROOT_CA_DIR/cacert.pem $INT_CA_DIR/cacert.pem >>$INT_CA_DIR/ca-chain.pem + cat $INT_CA_DIR/ca-chain.pem >> $SSL_BUNDLE_FILE + + if is_fedora; then + sudo cp $INT_CA_DIR/ca-chain.pem /usr/share/pki/ca-trust-source/anchors/stackube-chain.pem + sudo update-ca-trust + elif is_suse; then + sudo cp $INT_CA_DIR/ca-chain.pem /usr/share/pki/trust/anchors/stackube-chain.pem + sudo update-ca-certificates + elif is_ubuntu; then + sudo cp $INT_CA_DIR/ca-chain.pem /usr/local/share/ca-certificates/stackube-int.crt + sudo cp $ROOT_CA_DIR/cacert.pem /usr/local/share/ca-certificates/stackube-root.crt + sudo update-ca-certificates + fi +} + +# Create an initial server cert +# init_cert +function init_cert { + if [[ ! -r $STACKUBE_CERT ]]; then + if [[ -n "$TLS_IP" ]]; then + # Lie to let incomplete match routines work + TLS_IP="DNS:$TLS_IP,IP:$TLS_IP" + fi + make_cert $INT_CA_DIR $STACKUBE_CERT_NAME $STACKUBE_HOSTNAME "$TLS_IP" + + # Create a cert bundle + cat $INT_CA_DIR/private/$STACKUBE_CERT_NAME.key $INT_CA_DIR/$STACKUBE_CERT_NAME.crt $INT_CA_DIR/cacert.pem >$STACKUBE_CERT + fi +} + +# make_cert creates and signs a new certificate with the given commonName and CA +# make_cert ca-dir cert-name "common-name" ["alt-name" ...] +function make_cert { + local ca_dir=$1 + local cert_name=$2 + local common_name=$3 + local alt_names=$4 + + if [ "$common_name" != "$SERVICE_HOST" ]; then + if [[ -z "$alt_names" ]]; then + alt_names="DNS:$SERVICE_HOST" + else + alt_names="$alt_names,DNS:$SERVICE_HOST" + fi + if is_ipv4_address "$SERVICE_HOST" ; then + alt_names="$alt_names,IP:$SERVICE_HOST" + fi + fi + + # Only generate the certificate if it doesn't exist yet on the disk + if [ ! -r "$ca_dir/$cert_name.crt" ]; then + # Generate a signing request + $OPENSSL req \ + -sha1 \ + -newkey rsa \ + -nodes \ + -keyout $ca_dir/private/$cert_name.key \ + -out $ca_dir/$cert_name.csr \ + -subj "/O=${ORG_NAME}/OU=${ORG_UNIT_NAME} Servers/CN=${common_name}" + + if [[ -z "$alt_names" ]]; then + alt_names="DNS:${common_name}" + else + alt_names="DNS:${common_name},${alt_names}" + fi + + # Sign the request valid for 1 year + SUBJECT_ALT_NAME="$alt_names" \ + $OPENSSL ca -config $ca_dir/signing.conf \ + -extensions req_extensions \ + -days 3650 \ + -notext \ + -in $ca_dir/$cert_name.csr \ + -out $ca_dir/$cert_name.crt \ + -subj "/O=${ORG_NAME}/OU=${ORG_UNIT_NAME} Servers/CN=${common_name}" \ + -batch + fi +} + +# Make an intermediate CA to sign everything else +# make_int_CA ca-dir signing-ca-dir +function make_int_CA { + local ca_dir=$1 + local signing_ca_dir=$2 + + # Create the root CA + create_CA_base $ca_dir + create_CA_config $ca_dir 'Intermediate CA' + create_signing_config $ca_dir + + if [ ! -r "$ca_dir/cacert.pem" ]; then + # Create a signing certificate request + $OPENSSL req -config $ca_dir/ca.conf \ + -sha1 \ + -newkey rsa \ + -nodes \ + -keyout $ca_dir/private/cacert.key \ + -out $ca_dir/cacert.csr \ + -outform PEM + + # Sign the intermediate request valid for 1 year + $OPENSSL ca -config $signing_ca_dir/ca.conf \ + -extensions ca_extensions \ + -days 3650 \ + -notext \ + -in $ca_dir/cacert.csr \ + -out $ca_dir/cacert.pem \ + -batch + fi +} + +# Make a root CA to sign other CAs +# make_root_CA ca-dir +function make_root_CA { + local ca_dir=$1 + + # Create the root CA + create_CA_base $ca_dir + create_CA_config $ca_dir 'Root CA' + + if [ ! -r "$ca_dir/cacert.pem" ]; then + # Create a self-signed certificate valid for 5 years + $OPENSSL req -config $ca_dir/ca.conf \ + -x509 \ + -nodes \ + -newkey rsa \ + -days 21360 \ + -keyout $ca_dir/private/cacert.key \ + -out $ca_dir/cacert.pem \ + -outform PEM + fi +} + + + + +# Cleanup Functions +# ================= + +# Clean up the CA files +# cleanup_CA +function cleanup_CA { + if is_fedora; then + sudo rm -f /usr/share/pki/ca-trust-source/anchors/stackube-chain.pem + sudo update-ca-trust + elif is_ubuntu; then + sudo rm -f /usr/local/share/ca-certificates/stackube-int.crt + sudo rm -f /usr/local/share/ca-certificates/stackube-root.crt + sudo update-ca-certificates + fi + + rm -rf "$INT_CA_DIR" "$ROOT_CA_DIR" "$STACKUBE_CERT" +} + diff --git a/install/openstack/config_openstack/cinder-api/cinder.conf b/install/openstack/config_openstack/cinder-api/cinder.conf new file mode 100644 index 0000000..95fee52 --- /dev/null +++ b/install/openstack/config_openstack/cinder-api/cinder.conf @@ -0,0 +1,56 @@ +[DEFAULT] +debug = False +log_dir = /var/log/kolla/cinder +use_forwarded_for = true +use_stderr = False +enable_v1_api = false +osapi_volume_workers = 4 +volume_name_template = volume-%s + +os_region_name = RegionOne +enabled_backends = rbd-1 + +osapi_volume_listen = __CINDER_API_IP__ +osapi_volume_listen_port = 8776 +api_paste_config = /etc/cinder/api-paste.ini + +auth_strategy = keystone +transport_url = rabbit://openstack:__RABBITMQ_PWD__@__RABBITMQ_HOST__:5672 + +[oslo_messaging_notifications] +driver = noop + +[database] +connection = mysql+pymysql://cinder:__MYSQL_CINDER_PWD__@__MYSQL_HOST__:3306/cinder +max_retries = -1 + +[keystone_authtoken] +auth_uri = https://__OPENSTACK_ENDPOINT_IP__:5001/v3 +auth_url = https://__OPENSTACK_ENDPOINT_IP__:35358/v3 +auth_type = password +project_domain_id = default +user_domain_id = default +project_name = service +username = cinder +password = __KEYSTONE_CINDER_PWD__ +cafile = /etc/cinder/haproxy-ca.crt + + +[oslo_concurrency] +lock_path = /var/lib/cinder/tmp + +[rbd-1] +volume_driver = cinder.volume.drivers.rbd.RBDDriver +rbd_pool = cinder +rbd_ceph_conf = /etc/ceph/ceph.conf +rbd_flatten_volume_from_snapshot = false +rbd_max_clone_depth = 5 +rbd_store_chunk_size = 4 +rados_connect_timeout = -1 +rbd_user = cinder +rbd_secret_uuid = b2ec8922-0647-4885-9d6a-94d2688f35a3 +report_discard_supported = True + +[privsep_entrypoint] +helper_command = sudo cinder-rootwrap /etc/cinder/rootwrap.conf privsep-helper --config-file /etc/cinder/cinder.conf + diff --git a/install/openstack/config_openstack/cinder-api/config.json b/install/openstack/config_openstack/cinder-api/config.json new file mode 100644 index 0000000..a012df7 --- /dev/null +++ b/install/openstack/config_openstack/cinder-api/config.json @@ -0,0 +1,37 @@ +{ + "command": "cinder-api --config-file /etc/cinder/cinder.conf", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/cinder.conf", + "dest": "/etc/cinder/cinder.conf", + "owner": "cinder", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/cinder/policy.json", + "owner": "cinder", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/haproxy-ca.crt", + "dest": "/etc/cinder/haproxy-ca.crt", + "owner": "cinder", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/lib/cinder", + "owner": "cinder:cinder", + "recurse": true + }, + { + "path": "/var/log/kolla/cinder", + "owner": "cinder:cinder", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/cinder-scheduler/config.json b/install/openstack/config_openstack/cinder-scheduler/config.json new file mode 100644 index 0000000..9260ec0 --- /dev/null +++ b/install/openstack/config_openstack/cinder-scheduler/config.json @@ -0,0 +1,30 @@ +{ + "command": "cinder-scheduler --config-file /etc/cinder/cinder.conf", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/cinder.conf", + "dest": "/etc/cinder/cinder.conf", + "owner": "cinder", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/cinder/policy.json", + "owner": "cinder", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/lib/cinder", + "owner": "cinder:cinder", + "recurse": true + }, + { + "path": "/var/log/kolla/cinder", + "owner": "cinder:cinder", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/cinder-volume/config.json b/install/openstack/config_openstack/cinder-volume/config.json new file mode 100644 index 0000000..a2b2305 --- /dev/null +++ b/install/openstack/config_openstack/cinder-volume/config.json @@ -0,0 +1,51 @@ +{ + "command": "cinder-volume --config-file /etc/cinder/cinder.conf", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/cinder.conf", + "dest": "/etc/cinder/cinder.conf", + "owner": "cinder", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ceph.*", + "dest": "/etc/ceph/", + "owner": "cinder", + "perm": "0700", + "optional": false + }, + { + "source": "/var/lib/kolla/config_files/ceph.conf", + "dest": "/etc/ceph/ceph.conf", + "owner": "cinder", + "perm": "0600", + "optional": false + }, + { + "source": "/var/lib/kolla/config_files/nfs_shares", + "dest": "/etc/cinder/nfs_shares", + "owner": "cinder", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/cinder/policy.json", + "owner": "cinder", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/lib/cinder", + "owner": "cinder:cinder", + "recurse": true + }, + { + "path": "/var/log/kolla/cinder", + "owner": "cinder:cinder", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/haproxy/config.json b/install/openstack/config_openstack/haproxy/config.json new file mode 100644 index 0000000..4f894d8 --- /dev/null +++ b/install/openstack/config_openstack/haproxy/config.json @@ -0,0 +1,18 @@ +{ + "command": "/usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/haproxy.cfg", + "dest": "/etc/haproxy/haproxy.cfg", + "owner": "root", + "perm": "0644" + }, + { + "source": "/var/lib/kolla/config_files/haproxy.pem", + "dest": "/etc/haproxy/haproxy.pem", + "owner": "root", + "perm": "0600", + "optional": false + } + ] +} diff --git a/install/openstack/config_openstack/haproxy/haproxy.cfg b/install/openstack/config_openstack/haproxy/haproxy.cfg new file mode 100644 index 0000000..a4a8c0d --- /dev/null +++ b/install/openstack/config_openstack/haproxy/haproxy.cfg @@ -0,0 +1,61 @@ +global + chroot /var/lib/haproxy + user haproxy + group haproxy + daemon + log 127.0.0.1 local2 + maxconn 4000 + stats socket /var/lib/kolla/haproxy/haproxy.sock + ssl-default-bind-ciphers DEFAULT:!MEDIUM:!3DES + ssl-default-bind-options no-sslv3 no-tlsv10 + tune.ssl.default-dh-param 4096 + +defaults + log global + mode http + option redispatch + option httplog + option forwardfor + retries 3 + timeout http-request 10s + timeout queue 1m + timeout connect 10s + timeout client 30m + timeout server 30m + timeout check 10s + + + + +listen keystone_internal_external + bind __OPENSTACK_ENDPOINT_IP__:5001 ssl crt /etc/haproxy/haproxy.pem + http-request del-header X-Forwarded-Proto if { ssl_fc } + http-request set-header X-Forwarded-Proto https if { ssl_fc } + server stackube __KEYSTONE_API_IP__:5000 check inter 2000 rise 2 fall 5 + +listen keystone_admin + bind __OPENSTACK_ENDPOINT_IP__:35358 ssl crt /etc/haproxy/haproxy.pem + http-request del-header X-Forwarded-Proto if { ssl_fc } + http-request set-header X-Forwarded-Proto https if { ssl_fc } + server stackube __KEYSTONE_API_IP__:35357 check inter 2000 rise 2 fall 5 + + + +listen neutron_server + bind __OPENSTACK_ENDPOINT_IP__:9697 ssl crt /etc/haproxy/haproxy.pem + server stackube __NEUTRON_API_IP__:9696 check inter 2000 rise 2 fall 5 + + + +listen cinder_api + bind __OPENSTACK_ENDPOINT_IP__:8777 ssl crt /etc/haproxy/haproxy.pem + http-request del-header X-Forwarded-Proto if { ssl_fc } + http-request set-header X-Forwarded-Proto https if { ssl_fc } + server stackube __CINDER_API_IP__:8776 check inter 2000 rise 2 fall 5 + + + + + + + diff --git a/install/openstack/config_openstack/keystone/config.json b/install/openstack/config_openstack/keystone/config.json new file mode 100644 index 0000000..b55dfa3 --- /dev/null +++ b/install/openstack/config_openstack/keystone/config.json @@ -0,0 +1,47 @@ +{ + "command": "/usr/sbin/httpd", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/keystone.conf", + "dest": "/etc/keystone/keystone.conf", + "owner": "keystone", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/keystone-paste.ini", + "dest": "/etc/keystone/keystone-paste.ini", + "owner": "keystone", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/domains", + "dest": "/etc/keystone/domains", + "owner": "keystone", + "perm": "0700", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/keystone/policy.json", + "owner": "keystone", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/wsgi-keystone.conf", + "dest": "/etc/httpd/conf.d/wsgi-keystone.conf", + "owner": "keystone", + "perm": "0644" + } + ], + "permissions": [ + { + "path": "/var/log/kolla", + "owner": "keystone:kolla" + }, + { + "path": "/var/log/kolla/keystone/keystone.log", + "owner": "keystone:keystone" + } + ] +} diff --git a/install/openstack/config_openstack/keystone/keystone-paste.ini b/install/openstack/config_openstack/keystone/keystone-paste.ini new file mode 100644 index 0000000..2fb07b4 --- /dev/null +++ b/install/openstack/config_openstack/keystone/keystone-paste.ini @@ -0,0 +1,76 @@ +[filter:debug] +use = egg:oslo.middleware#debug + +[filter:request_id] +use = egg:oslo.middleware#request_id + +[filter:build_auth_context] +use = egg:keystone#build_auth_context + +[filter:token_auth] +use = egg:keystone#token_auth + +[filter:json_body] +use = egg:keystone#json_body + +[filter:cors] +use = egg:oslo.middleware#cors +oslo_config_project = keystone + +[filter:ec2_extension] +use = egg:keystone#ec2_extension + +[filter:ec2_extension_v3] +use = egg:keystone#ec2_extension_v3 + +[filter:s3_extension] +use = egg:keystone#s3_extension + +[filter:url_normalize] +use = egg:keystone#url_normalize + +[filter:sizelimit] +use = egg:oslo.middleware#sizelimit + +[app:public_service] +use = egg:keystone#public_service + +[app:service_v3] +use = egg:keystone#service_v3 + +[app:admin_service] +use = egg:keystone#admin_service + +[pipeline:public_api] +pipeline = cors sizelimit url_normalize request_id build_auth_context token_auth json_body ec2_extension public_service + +[pipeline:admin_api] +pipeline = cors sizelimit url_normalize request_id build_auth_context token_auth json_body ec2_extension s3_extension admin_service + +[pipeline:api_v3] +pipeline = cors sizelimit url_normalize request_id build_auth_context token_auth json_body ec2_extension_v3 s3_extension service_v3 + +[app:public_version_service] +use = egg:keystone#public_version_service + +[app:admin_version_service] +use = egg:keystone#admin_version_service + +[pipeline:public_version_api] +pipeline = cors sizelimit url_normalize public_version_service + +[pipeline:admin_version_api] +pipeline = cors sizelimit url_normalize admin_version_service + +[composite:main] +use = egg:Paste#urlmap +/v2.0 = public_api +/v3 = api_v3 +/ = public_version_api + +[composite:admin] +use = egg:Paste#urlmap +/v2.0 = admin_api +/v3 = api_v3 +/ = admin_version_api + diff --git a/install/openstack/config_openstack/keystone/keystone.conf b/install/openstack/config_openstack/keystone/keystone.conf new file mode 100644 index 0000000..07f9833 --- /dev/null +++ b/install/openstack/config_openstack/keystone/keystone.conf @@ -0,0 +1,14 @@ +[DEFAULT] +debug = False +log_file = /var/log/kolla/keystone/keystone.log +use_stderr = True +secure_proxy_ssl_header = HTTP_X_FORWARDED_PROTO + +[database] +connection = mysql+pymysql://keystone:__MYSQL_KWYSTONE_PWD__@__MYSQL_HOST__:3306/keystone +max_retries = -1 + +[token] +revoke_by_id = False +provider = uuid + diff --git a/install/openstack/config_openstack/keystone/wsgi-keystone.conf b/install/openstack/config_openstack/keystone/wsgi-keystone.conf new file mode 100644 index 0000000..4a63cf0 --- /dev/null +++ b/install/openstack/config_openstack/keystone/wsgi-keystone.conf @@ -0,0 +1,30 @@ +Listen __KEYSTONE_API_IP__:5000 +Listen __KEYSTONE_API_IP__:35357 + + + WSGIDaemonProcess keystone-public processes=4 threads=1 user=keystone group=keystone display-name=%{GROUP} python-path=/usr/lib/python2.7/site-packages + WSGIProcessGroup keystone-public + WSGIScriptAlias / /var/www/cgi-bin/keystone/main + WSGIApplicationGroup %{GLOBAL} + WSGIPassAuthorization On + = 2.4> + ErrorLogFormat "%{cu}t %M" + + ErrorLog "/var/log/kolla/keystone/keystone-apache-public-error.log" + LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"" logformat + CustomLog "/var/log/kolla/keystone/keystone-apache-public-access.log" logformat + + + + WSGIDaemonProcess keystone-admin processes=4 threads=1 user=keystone group=keystone display-name=%{GROUP} python-path=/usr/lib/python2.7/site-packages + WSGIProcessGroup keystone-admin + WSGIScriptAlias / /var/www/cgi-bin/keystone/admin + WSGIApplicationGroup %{GLOBAL} + WSGIPassAuthorization On + = 2.4> + ErrorLogFormat "%{cu}t %M" + + ErrorLog "/var/log/kolla/keystone/keystone-apache-admin-error.log" + LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"" logformat + CustomLog "/var/log/kolla/keystone/keystone-apache-admin-access.log" logformat + diff --git a/install/openstack/config_openstack/kolla-toolbox/config.json b/install/openstack/config_openstack/kolla-toolbox/config.json new file mode 100644 index 0000000..7f0dc05 --- /dev/null +++ b/install/openstack/config_openstack/kolla-toolbox/config.json @@ -0,0 +1,10 @@ +{ + "command": "sleep infinity", + "config_files": [], + "permissions": [ + { + "path": "/var/log/kolla/ansible.log", + "owner": "ansible:ansible" + } + ] +} diff --git a/install/openstack/config_openstack/neutron-dhcp-agent/config.json b/install/openstack/config_openstack/neutron-dhcp-agent/config.json new file mode 100644 index 0000000..dbd6e0b --- /dev/null +++ b/install/openstack/config_openstack/neutron-dhcp-agent/config.json @@ -0,0 +1,55 @@ +{ + "command": "neutron-dhcp-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/dhcp_agent.ini", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/neutron.conf", + "dest": "/etc/neutron/neutron.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ml2_conf.ini", + "dest": "/etc/neutron/plugins/ml2/ml2_conf.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/dhcp_agent.ini", + "dest": "/etc/neutron/dhcp_agent.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/dnsmasq.conf", + "dest": "/etc/neutron/dnsmasq.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/neutron/policy.json", + "owner": "neutron", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/haproxy-ca.crt", + "dest": "/etc/neutron/haproxy-ca.crt", + "owner": "neutron", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/log/kolla/neutron", + "owner": "neutron:neutron", + "recurse": true + }, + { + "path": "/var/lib/neutron/kolla", + "owner": "neutron:neutron", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/neutron-dhcp-agent/dhcp_agent.ini b/install/openstack/config_openstack/neutron-dhcp-agent/dhcp_agent.ini new file mode 100644 index 0000000..e0779dd --- /dev/null +++ b/install/openstack/config_openstack/neutron-dhcp-agent/dhcp_agent.ini @@ -0,0 +1,6 @@ +[DEFAULT] +dnsmasq_config_file = /etc/neutron/dnsmasq.conf +enable_isolated_metadata = False +force_metadata = False +dnsmasq_dns_servers = 8.8.8.8,8.8.4.4 + diff --git a/install/openstack/config_openstack/neutron-dhcp-agent/dnsmasq.conf b/install/openstack/config_openstack/neutron-dhcp-agent/dnsmasq.conf new file mode 100644 index 0000000..91566c4 --- /dev/null +++ b/install/openstack/config_openstack/neutron-dhcp-agent/dnsmasq.conf @@ -0,0 +1 @@ +log-facility=/var/log/kolla/neutron/dnsmasq.log diff --git a/install/openstack/config_openstack/neutron-l3-agent/config.json b/install/openstack/config_openstack/neutron-l3-agent/config.json new file mode 100644 index 0000000..2be7cc4 --- /dev/null +++ b/install/openstack/config_openstack/neutron-l3-agent/config.json @@ -0,0 +1,55 @@ +{ + "command": "neutron-l3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/l3_agent.ini --config-file /etc/neutron/fwaas_driver.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/neutron.conf", + "dest": "/etc/neutron/neutron.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ml2_conf.ini", + "dest": "/etc/neutron/plugins/ml2/ml2_conf.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/fwaas_driver.ini", + "dest": "/etc/neutron/fwaas_driver.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/l3_agent.ini", + "dest": "/etc/neutron/l3_agent.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/neutron/policy.json", + "owner": "neutron", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/haproxy-ca.crt", + "dest": "/etc/neutron/haproxy-ca.crt", + "owner": "neutron", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/log/kolla/neutron", + "owner": "neutron:neutron", + "recurse": true + }, + { + "path": "/var/lib/neutron/kolla", + "owner": "neutron:neutron", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/neutron-l3-agent/fwaas_driver.ini b/install/openstack/config_openstack/neutron-l3-agent/fwaas_driver.ini new file mode 100644 index 0000000..977b3a9 --- /dev/null +++ b/install/openstack/config_openstack/neutron-l3-agent/fwaas_driver.ini @@ -0,0 +1,2 @@ +[fwaas] + diff --git a/install/openstack/config_openstack/neutron-l3-agent/l3_agent.ini b/install/openstack/config_openstack/neutron-l3-agent/l3_agent.ini new file mode 100644 index 0000000..9546856 --- /dev/null +++ b/install/openstack/config_openstack/neutron-l3-agent/l3_agent.ini @@ -0,0 +1,3 @@ +[DEFAULT] +agent_mode = legacy + diff --git a/install/openstack/config_openstack/neutron-lbaas-agent/config.json b/install/openstack/config_openstack/neutron-lbaas-agent/config.json new file mode 100644 index 0000000..b863a70 --- /dev/null +++ b/install/openstack/config_openstack/neutron-lbaas-agent/config.json @@ -0,0 +1,55 @@ +{ + "command": "neutron-lbaasv2-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/lbaas_agent.ini --config-file /etc/neutron/neutron_lbaas.conf", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/neutron.conf", + "dest": "/etc/neutron/neutron.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/lbaas_agent.ini", + "dest": "/etc/neutron/lbaas_agent.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ml2_conf.ini", + "dest": "/etc/neutron/plugins/ml2/ml2_conf.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/neutron_lbaas.conf", + "dest": "/etc/neutron/neutron_lbaas.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/neutron/policy.json", + "owner": "neutron", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/haproxy-ca.crt", + "dest": "/etc/neutron/haproxy-ca.crt", + "owner": "neutron", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/log/kolla/neutron", + "owner": "neutron:neutron", + "recurse": true + }, + { + "path": "/var/lib/neutron/kolla", + "owner": "neutron:neutron", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/neutron-lbaas-agent/lbaas_agent.ini b/install/openstack/config_openstack/neutron-lbaas-agent/lbaas_agent.ini new file mode 100644 index 0000000..72a5377 --- /dev/null +++ b/install/openstack/config_openstack/neutron-lbaas-agent/lbaas_agent.ini @@ -0,0 +1,7 @@ +[DEFAULT] +debug = False +device_driver = neutron_lbaas.drivers.haproxy.namespace_driver.HaproxyNSDriver + +[haproxy] +user_group = haproxy + diff --git a/install/openstack/config_openstack/neutron-openvswitch-agent/config.json b/install/openstack/config_openstack/neutron-openvswitch-agent/config.json new file mode 100644 index 0000000..3a94959 --- /dev/null +++ b/install/openstack/config_openstack/neutron-openvswitch-agent/config.json @@ -0,0 +1,38 @@ +{ + "command": "neutron-openvswitch-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/neutron.conf", + "dest": "/etc/neutron/neutron.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ml2_conf.ini", + "dest": "/etc/neutron/plugins/ml2/ml2_conf.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/neutron/policy.json", + "owner": "neutron", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/haproxy-ca.crt", + "dest": "/etc/neutron/haproxy-ca.crt", + "owner": "neutron", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/log/kolla/neutron", + "owner": "neutron:neutron", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/neutron-server/config.json b/install/openstack/config_openstack/neutron-server/config.json new file mode 100644 index 0000000..91c4690 --- /dev/null +++ b/install/openstack/config_openstack/neutron-server/config.json @@ -0,0 +1,50 @@ +{ + "command": "neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/neutron_lbaas.conf --config-file /etc/neutron/neutron_vpnaas.conf", + "config_files": [ + { + "source": "/var/lib/kolla/config_files/neutron.conf", + "dest": "/etc/neutron/neutron.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/neutron_lbaas.conf", + "dest": "/etc/neutron/neutron_lbaas.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/neutron_vpnaas.conf", + "dest": "/etc/neutron/neutron_vpnaas.conf", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/ml2_conf.ini", + "dest": "/etc/neutron/plugins/ml2/ml2_conf.ini", + "owner": "neutron", + "perm": "0600" + }, + { + "source": "/var/lib/kolla/config_files/policy.json", + "dest": "/etc/neutron/policy.json", + "owner": "neutron", + "perm": "0600", + "optional": true + }, + { + "source": "/var/lib/kolla/config_files/haproxy-ca.crt", + "dest": "/etc/neutron/haproxy-ca.crt", + "owner": "neutron", + "perm": "0600", + "optional": true + } + ], + "permissions": [ + { + "path": "/var/log/kolla/neutron", + "owner": "neutron:neutron", + "recurse": true + } + ] +} diff --git a/install/openstack/config_openstack/neutron-server/ml2_conf.ini b/install/openstack/config_openstack/neutron-server/ml2_conf.ini new file mode 100644 index 0000000..45148ff --- /dev/null +++ b/install/openstack/config_openstack/neutron-server/ml2_conf.ini @@ -0,0 +1,28 @@ +[ml2] +type_drivers = flat,vlan,vxlan +tenant_network_types = vxlan +mechanism_drivers = openvswitch,l2population + +[ml2_type_vlan] +network_vlan_ranges = + +[ml2_type_flat] +flat_networks = physnet1 + +[ml2_type_vxlan] +vni_ranges = 1:10000 +vxlan_group = 239.1.1.1 + +[securitygroup] +firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver + +[agent] +tunnel_types = vxlan +l2_population = true +arp_responder = true + +[ovs] +bridge_mappings = physnet1:br-ex +ovsdb_connection = tcp:__OVSDB_IP__:6640 +local_ip = __LOCAL_IP__ + diff --git a/install/openstack/config_openstack/neutron-server/neutron.conf b/install/openstack/config_openstack/neutron-server/neutron.conf new file mode 100644 index 0000000..1a738aa --- /dev/null +++ b/install/openstack/config_openstack/neutron-server/neutron.conf @@ -0,0 +1,41 @@ +[DEFAULT] +debug = False +log_dir = /var/log/kolla/neutron +use_stderr = False +bind_host = __NEUTRON_API_IP__ +bind_port = 9696 +api_paste_config = /usr/share/neutron/api-paste.ini +endpoint_type = internalURL +api_workers = 4 +interface_driver = openvswitch +allow_overlapping_ips = true +core_plugin = ml2 +service_plugins = router,neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2 +transport_url = rabbit://openstack:__RABBITMQ_PWD__@__RABBITMQ_HOST__:5672 + + +[oslo_concurrency] +lock_path = /var/lib/neutron/tmp + +[agent] +root_helper = sudo neutron-rootwrap /etc/neutron/rootwrap.conf + +[database] +connection = mysql+pymysql://neutron:__MYSQL_NEUTRON_PWD__@__MYSQL_HOST__:3306/neutron +max_retries = -1 + +[keystone_authtoken] +auth_uri = https://__OPENSTACK_ENDPOINT_IP__:5001/v3 +auth_url = https://__OPENSTACK_ENDPOINT_IP__:35358/v3 +auth_type = password +project_domain_id = default +user_domain_id = default +project_name = service +username = neutron +password = __KEYSTONE_NEUTRON_PWD__ +cafile = /etc/neutron/haproxy-ca.crt + + +[oslo_messaging_notifications] +driver = noop + diff --git a/install/openstack/config_openstack/neutron-server/neutron_lbaas.conf b/install/openstack/config_openstack/neutron-server/neutron_lbaas.conf new file mode 100644 index 0000000..f0faa36 --- /dev/null +++ b/install/openstack/config_openstack/neutron-server/neutron_lbaas.conf @@ -0,0 +1,12 @@ +[service_providers] +service_provider = LOADBALANCERV2:Haproxy:neutron_lbaas.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default + +[service_auth] +auth_url = http://__KEYSTONE_API_IP__:5000/v2.0 +admin_tenant_name = service +admin_user = neutron +admin_password = __NEUTRON_KEYSTONE_PWD__ +auth_version = 2 +region = RegionOne +endpoint_type = internal + diff --git a/install/openstack/config_openstack/neutron-server/neutron_vpnaas.conf b/install/openstack/config_openstack/neutron-server/neutron_vpnaas.conf new file mode 100644 index 0000000..e69de29 diff --git a/install/openstack/config_openstack/openvswitch-db-server/config.json b/install/openstack/config_openstack/openvswitch-db-server/config.json new file mode 100644 index 0000000..ce00d35 --- /dev/null +++ b/install/openstack/config_openstack/openvswitch-db-server/config.json @@ -0,0 +1,4 @@ +{ + "command": "start-ovsdb-server __OVSDB_IP__ ", + "config_files": [] +} diff --git a/install/openstack/config_openstack/openvswitch-vswitchd/config.json b/install/openstack/config_openstack/openvswitch-vswitchd/config.json new file mode 100644 index 0000000..97308d8 --- /dev/null +++ b/install/openstack/config_openstack/openvswitch-vswitchd/config.json @@ -0,0 +1,4 @@ +{ + "command": "/usr/sbin/ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/kolla/openvswitch/ovs-vswitchd.log", + "config_files": [] +} diff --git a/install/openstack/deploy_openstack_cinder_api.sh b/install/openstack/deploy_openstack_cinder_api.sh new file mode 100644 index 0000000..54e8913 --- /dev/null +++ b/install/openstack/deploy_openstack_cinder_api.sh @@ -0,0 +1,159 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OPENSTACK_ENDPOINT_IP`` +# - ``MYSQL_HOST``, ``MYSQL_ROOT_PWD`` +# - ``KEYSTONE_ADMIN_PWD`` +# - ``KEYSTONE_CINDER_PWD``, ``MYSQL_CINDER_PWD``must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## register - Creating the Cinder service and endpoint +## v1 +for IF in 'admin' 'internal' 'public'; do + echo ${IF} + docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m kolla_keystone_service \ + -a "service_name=cinder + service_type=volume + description='Openstack Block Storage' + endpoint_region=RegionOne + url='https://${OPENSTACK_ENDPOINT_IP}:8777/v1/%(tenant_id)s' + interface='${IF}' + region_name=RegionOne + auth='{{ openstack_keystone_auth }}' + verify=False " \ + -e "{'openstack_keystone_auth': { + 'auth_url': 'https://${OPENSTACK_ENDPOINT_IP}:35358/v3', + 'username': 'admin', + 'password': '${KEYSTONE_ADMIN_PWD}', + 'project_name': 'admin', + 'domain_name': 'default' } + }" +done + +## v2 +for VER in 'v2' ; do + echo -e "\n--- ${VER} ---" + for IF in 'admin' 'internal' 'public'; do + echo ${IF} + docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m kolla_keystone_service \ + -a "service_name=cinder${VER} + service_type=volume${VER} + description='Openstack Block Storage' + endpoint_region=RegionOne + url='https://${OPENSTACK_ENDPOINT_IP}:8777/${VER}/%(tenant_id)s' + interface='${IF}' + region_name=RegionOne + auth='{{ openstack_keystone_auth }}' + verify=False " \ + -e "{'openstack_keystone_auth': { + 'auth_url': 'https://${OPENSTACK_ENDPOINT_IP}:35358/v3', + 'username': 'admin', + 'password': '${KEYSTONE_ADMIN_PWD}', + 'project_name': 'admin', + 'domain_name': 'default' } + }" + done +done + + +## register - Creating the Cinder project, user, and role +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m kolla_keystone_user \ + -a "project=service + user=cinder + password=${KEYSTONE_CINDER_PWD} + role=admin + region_name=RegionOne + auth='{{ openstack_keystone_auth }}' + verify=False " \ + -e "{'openstack_keystone_auth': { + 'auth_url': 'https://${OPENSTACK_ENDPOINT_IP}:35358/v3', + 'username': 'admin', + 'password': '${KEYSTONE_ADMIN_PWD}', + 'project_name': 'admin', + 'domain_name': 'default' } + }" + + + +# bootstrap - Creating Cinder database +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m mysql_db \ + -a "login_host=${MYSQL_HOST} + login_port=3306 + login_user=root + login_password=${MYSQL_ROOT_PWD} + name=cinder" + +# bootstrap - Creating Cinder database user and setting permissions +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m mysql_user \ + -a "login_host=${MYSQL_HOST} + login_port=3306 + login_user=root + login_password=${MYSQL_ROOT_PWD} + name=cinder + password=${MYSQL_CINDER_PWD} + host=% + priv='cinder.*:ALL' + append_privs=yes" + + + +# bootstrap_service - Running Cinder bootstrap container +docker run --net host \ + --name stackube_openstack_bootstrap_cinder \ + -v /etc/stackube/openstack/cinder-api/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -e "KOLLA_BOOTSTRAP=" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + kolla/centos-binary-cinder-api:4.0.0 + +sleep 2 +docker rm stackube_openstack_bootstrap_cinder + + +## start_container - cinder-api +docker run -d --net host \ + --name stackube_openstack_cinder_api \ + -v /etc/stackube/openstack/cinder-api/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + \ + -e "KOLLA_SERVICE_NAME=cinder-api" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + kolla/centos-binary-cinder-api:4.0.0 + +sleep 5 + + +exit 0 diff --git a/install/openstack/deploy_openstack_cinder_scheduler.sh b/install/openstack/deploy_openstack_cinder_scheduler.sh new file mode 100644 index 0000000..bb7b234 --- /dev/null +++ b/install/openstack/deploy_openstack_cinder_scheduler.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``API_IP``, ``RABBITMQ_PWD`` +# - ``KEYSTONE_ADMIN_PWD`` +# - ``KEYSTONE_CINDER_PWD``, ``MYSQL_CINDER_PWD``must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## start_container - cinder-scheduler +docker run -d --net host \ + --name stackube_openstack_cinder_scheduler \ + -v /etc/stackube/openstack/cinder-scheduler/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + \ + -e "KOLLA_SERVICE_NAME=cinder-scheduler" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + kolla/centos-binary-cinder-scheduler:4.0.0 + +sleep 5 + + + +exit 0 diff --git a/install/openstack/deploy_openstack_cinder_volume.sh b/install/openstack/deploy_openstack_cinder_volume.sh new file mode 100644 index 0000000..4fde69a --- /dev/null +++ b/install/openstack/deploy_openstack_cinder_volume.sh @@ -0,0 +1,60 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``API_IP``, ``RABBITMQ_PWD`` +# - ``KEYSTONE_ADMIN_PWD`` +# - ``KEYSTONE_CINDER_PWD``, ``MYSQL_CINDER_PWD``must be defined +# + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## start_container - cinder-volume +docker run -d --net host \ + --name stackube_openstack_cinder_volume \ + -v /etc/stackube/openstack/cinder-volume/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /run/:/run/:shared \ + -v /dev/:/dev/:rw \ + \ + -e "KOLLA_SERVICE_NAME=cinder-volume" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-cinder-volume:4.0.0 + +sleep 5 + + + +exit 0 diff --git a/install/openstack/deploy_openstack_haproxy.sh b/install/openstack/deploy_openstack_haproxy.sh new file mode 100644 index 0000000..feaf0db --- /dev/null +++ b/install/openstack/deploy_openstack_haproxy.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OPENSTACK_ENDPOINT_IP`` +# - ``KEYSTONE_API_IP`` +# - ``NEUTRON_API_IP`` +# - ``CINDER_API_IP`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## make certificates +HOST_IP=${OPENSTACK_ENDPOINT_IP} +SERVICE_HOST=${OPENSTACK_ENDPOINT_IP} +SERVICE_IP=${OPENSTACK_ENDPOINT_IP} +DATA_DIR='/etc/stackube/openstack/certificates' +source ${programDir}/../lib_tls.sh +mkdir -p ${DATA_DIR} +init_CA +init_cert + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## config files +mkdir -p /etc/stackube/openstack +cp -a ${programDir}/config_openstack/haproxy /etc/stackube/openstack/ +sed -i "s/__OPENSTACK_ENDPOINT_IP__/${OPENSTACK_ENDPOINT_IP}/g" /etc/stackube/openstack/haproxy/haproxy.cfg +sed -i "s/__KEYSTONE_API_IP__/${KEYSTONE_API_IP}/g" /etc/stackube/openstack/haproxy/haproxy.cfg +sed -i "s/__NEUTRON_API_IP__/${NEUTRON_API_IP}/g" /etc/stackube/openstack/haproxy/haproxy.cfg +sed -i "s/__CINDER_API_IP__/${CINDER_API_IP}/g" /etc/stackube/openstack/haproxy/haproxy.cfg +# STACKUBE_CERT defined in lib_tls.sh +cat ${STACKUBE_CERT} > /etc/stackube/openstack/haproxy/haproxy.pem + + +## run +docker run -d --net host \ + --name stackube_openstack_haproxy \ + -v /etc/stackube/openstack/haproxy/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + \ + -e "KOLLA_SERVICE_NAME=haproxy" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-haproxy:4.0.0 + + +exit 0 + diff --git a/install/openstack/deploy_openstack_keystone.sh b/install/openstack/deploy_openstack_keystone.sh new file mode 100644 index 0000000..17c23ba --- /dev/null +++ b/install/openstack/deploy_openstack_keystone.sh @@ -0,0 +1,121 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OPENSTACK_ENDPOINT_IP``, ``KEYSTONE_API_IP`` +# - ``MYSQL_HOST``, ``MYSQL_ROOT_PWD`` +# - ``MYSQL_KEYSTONE_PWD``, ``KEYSTONE_ADMIN_PWD`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## create db +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m mysql_db \ + -a "login_host=${MYSQL_HOST} + login_port=3306 + login_user=root + login_password=${MYSQL_ROOT_PWD} + name=keystone" + +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m mysql_user \ + -a "login_host=${MYSQL_HOST} + login_port=3306 + login_user=root + login_password=${MYSQL_ROOT_PWD} + name=keystone + password=${MYSQL_KEYSTONE_PWD} + host=% + priv=keystone.*:ALL + append_privs=yes " + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## config files +mkdir -p /etc/stackube/openstack +cp -a ${programDir}/config_openstack/keystone /etc/stackube/openstack/ +sed -i "s/__MYSQL_HOST__/${MYSQL_HOST}/g" /etc/stackube/openstack/keystone/keystone.conf +sed -i "s/__MYSQL_KWYSTONE_PWD__/${MYSQL_KEYSTONE_PWD}/g" /etc/stackube/openstack/keystone/keystone.conf +sed -i "s/__KEYSTONE_API_IP__/${KEYSTONE_API_IP}/g" /etc/stackube/openstack/keystone/wsgi-keystone.conf + + +# bootstrap_service +docker run --net host \ + --name stackube_openstack_bootstrap_keystone \ + -v /etc/stackube/openstack/keystone/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -e "KOLLA_BOOTSTRAP=" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + kolla/centos-binary-keystone:4.0.0 + +docker rm stackube_openstack_bootstrap_keystone + +docker run -d --net host \ + --name stackube_openstack_keystone \ + -v /etc/stackube/openstack/keystone/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -e "KOLLA_SERVICE_NAME=keystone" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + --restart unless-stopped \ + kolla/centos-binary-keystone:4.0.0 + +sleep 10 + +# register +docker exec stackube_openstack_keystone kolla_keystone_bootstrap admin ${KEYSTONE_ADMIN_PWD} admin admin \ + https://${OPENSTACK_ENDPOINT_IP}:35358/v3 \ + https://${OPENSTACK_ENDPOINT_IP}:5001/v3 \ + https://${OPENSTACK_ENDPOINT_IP}:5001/v3 \ + RegionOne + +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m os_keystone_role -a "name=_member_ auth='{{ openstack_keystone_auth }}' verify=False" \ + -e "{'openstack_keystone_auth': { + 'auth_url': 'https://${OPENSTACK_ENDPOINT_IP}:35358/v3', + 'username': 'admin', + 'password': '${KEYSTONE_ADMIN_PWD}', + 'project_name': 'admin', + 'domain_name': 'default' } + }" + + +cat > /etc/stackube/openstack/admin-openrc.sh << EOF +export OS_PROJECT_DOMAIN_NAME=default +export OS_USER_DOMAIN_NAME=default +export OS_PROJECT_NAME=admin +export OS_TENANT_NAME=admin +export OS_USERNAME=admin +export OS_PASSWORD=${KEYSTONE_ADMIN_PWD} +export OS_AUTH_URL=https://${OPENSTACK_ENDPOINT_IP}:35358/v3 +export OS_INTERFACE=internal +export OS_IDENTITY_API_VERSION=3 +export OS_CACERT=/etc/stackube/openstack/certificates/CA/int-ca/ca-chain.pem +EOF + +exit 0 + diff --git a/install/openstack/deploy_openstack_kolla_toolbox.sh b/install/openstack/deploy_openstack_kolla_toolbox.sh new file mode 100644 index 0000000..df2e50b --- /dev/null +++ b/install/openstack/deploy_openstack_kolla_toolbox.sh @@ -0,0 +1,55 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## kolla-toolbox +docker run -d --net host \ + --name stackube_openstack_kolla_toolbox \ + -v /run/:/run/:shared \ + -v /dev/:/dev/:rw \ + -v /etc/stackube/openstack/kolla-toolbox/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -e "KOLLA_SERVICE_NAME=kolla-toolbox" \ + -e "ANSIBLE_LIBRARY=/usr/share/ansible" \ + -e "ANSIBLE_NOCOLOR=1" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-kolla-toolbox:4.0.0 + +sleep 5 + + +exit 0 + diff --git a/install/openstack/deploy_openstack_mariadb.sh b/install/openstack/deploy_openstack_mariadb.sh new file mode 100644 index 0000000..6799bf0 --- /dev/null +++ b/install/openstack/deploy_openstack_mariadb.sh @@ -0,0 +1,46 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``MYSQL_ROOT_PWD`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## mariadb +mkdir -p /var/lib/stackube/openstack/mariadb && \ +docker run -d \ + --name stackube_openstack_mariadb \ + --net host \ + -e MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PWD} \ + -v /var/lib/stackube/openstack/mariadb:/var/lib/mysql \ + --restart unless-stopped \ + mariadb:5.5 + +sleep 5 + +exit 0 + diff --git a/install/openstack/deploy_openstack_neutron_dhcp_agent.sh b/install/openstack/deploy_openstack_neutron_dhcp_agent.sh new file mode 100644 index 0000000..eef0e92 --- /dev/null +++ b/install/openstack/deploy_openstack_neutron_dhcp_agent.sh @@ -0,0 +1,58 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OVSDB_IP`` +# - ``ML2_LOCAL_IP`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## start_container - neutron-dhcp-agent +sed -i "s/__OVSDB_IP__/${OVSDB_IP}/g" /etc/stackube/openstack/neutron-dhcp-agent/ml2_conf.ini +sed -i "s/__LOCAL_IP__/${ML2_LOCAL_IP}/g" /etc/stackube/openstack/neutron-dhcp-agent/ml2_conf.ini + +docker run -d --net host \ + --name stackube_openstack_neutron_dhcp_agent \ + -v /etc/stackube/openstack/neutron-dhcp-agent/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /run:/run:shared \ + \ + -e "KOLLA_SERVICE_NAME=neutron-dhcp-agent" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-neutron-dhcp-agent:4.0.0 + + + +exit 0 diff --git a/install/openstack/deploy_openstack_neutron_l3_agent.sh b/install/openstack/deploy_openstack_neutron_l3_agent.sh new file mode 100644 index 0000000..3bd1aa8 --- /dev/null +++ b/install/openstack/deploy_openstack_neutron_l3_agent.sh @@ -0,0 +1,71 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OVSDB_IP`` +# - ``ML2_LOCAL_IP`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## sysctl +sed -i '/^net\.ipv4\.ip_forward=/d' /etc/sysctl.conf +sed -i '/^net\.ipv4\.conf\.all\.rp_filter=/d' /etc/sysctl.conf +sed -i '/^net\.ipv4\.conf\.default\.rp_filter=/d' /etc/sysctl.conf + +echo ' +net.ipv4.ip_forward=1 +net.ipv4.conf.all.rp_filter=0 +net.ipv4.conf.default.rp_filter=0 +' >> /etc/sysctl.conf + +sysctl -p + + +## start_container - neutron-l3-agent +sed -i "s/__OVSDB_IP__/${OVSDB_IP}/g" /etc/stackube/openstack/neutron-l3-agent/ml2_conf.ini +sed -i "s/__LOCAL_IP__/${ML2_LOCAL_IP}/g" /etc/stackube/openstack/neutron-l3-agent/ml2_conf.ini + +docker run -d --net host \ + --name stackube_openstack_neutron_l3_agent \ + -v /etc/stackube/openstack/neutron-l3-agent/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /run:/run:shared \ + \ + -e "KOLLA_SERVICE_NAME=neutron-l3-agent" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-neutron-l3-agent:4.0.0 + + +exit 0 diff --git a/install/openstack/deploy_openstack_neutron_lbaas_agent.sh b/install/openstack/deploy_openstack_neutron_lbaas_agent.sh new file mode 100644 index 0000000..cd4a2a9 --- /dev/null +++ b/install/openstack/deploy_openstack_neutron_lbaas_agent.sh @@ -0,0 +1,79 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OVSDB_IP``, ``ML2_LOCAL_IP`` +# - ``KEYSTONE_API_IP``, ``KEYSTONE_NEUTRON_PWD`` must be defined +# + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +# bootstrap_service - Running Neutron lbaas bootstrap container +sed -i "s/__OVSDB_IP__/${OVSDB_IP}/g" /etc/stackube/openstack/neutron-lbaas-agent/ml2_conf.ini +sed -i "s/__LOCAL_IP__/${ML2_LOCAL_IP}/g" /etc/stackube/openstack/neutron-lbaas-agent/ml2_conf.ini + +sed -i "s/__KEYSTONE_API_IP__/${KEYSTONE_API_IP}/g" /etc/stackube/openstack/neutron-lbaas-agent/neutron_lbaas.conf +sed -i "s/__NEUTRON_KEYSTONE_PWD__/${KEYSTONE_NEUTRON_PWD}/g" /etc/stackube/openstack/neutron-lbaas-agent/neutron_lbaas.conf + +docker run --net host \ + --name stackube_openstack_bootstrap_neutron_lbaas_agent \ + -v /etc/stackube/openstack/neutron-lbaas-agent/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /run/netns/:/run/netns/:shared \ + -v /run:/run:shared \ + \ + -e "KOLLA_BOOTSTRAP=" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --privileged \ + kolla/centos-binary-neutron-lbaas-agent:4.0.0 + +sleep 2 +docker rm stackube_openstack_bootstrap_neutron_lbaas_agent + + +## start_container - neutron-lbaas-agent +docker run -d --net host \ + --name stackube_openstack_neutron_lbaas_agent \ + -v /etc/stackube/openstack/neutron-lbaas-agent/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /run/netns/:/run/netns/:shared \ + -v /run:/run:shared \ + \ + -e "KOLLA_SERVICE_NAME=neutron-lbaas-agent" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-neutron-lbaas-agent:4.0.0 + + +exit 0 diff --git a/install/openstack/deploy_openstack_neutron_openvswitch_agent.sh b/install/openstack/deploy_openstack_neutron_openvswitch_agent.sh new file mode 100644 index 0000000..59df836 --- /dev/null +++ b/install/openstack/deploy_openstack_neutron_openvswitch_agent.sh @@ -0,0 +1,98 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OVSDB_IP`` +# - ``ML2_LOCAL_IP`` must be defined +# + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +## openvswitch-db-server +sed -i "s/__OVSDB_IP__/${OVSDB_IP}/g" /etc/stackube/openstack/openvswitch-db-server/config.json +mkdir -p /var/lib/stackube/openstack/openvswitch +docker run -d --net host \ + --name stackube_openstack_openvswitch_db \ + -v /etc/stackube/openstack/openvswitch-db-server/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /var/lib/stackube/openstack/openvswitch/:/var/lib/openvswitch/:rw \ + -v /run:/run:shared \ + \ + -e "KOLLA_SERVICE_NAME=openvswitch-db" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + kolla/centos-binary-openvswitch-db-server:4.0.0 + +sleep 5 + +# config br +docker exec stackube_openstack_openvswitch_db /usr/local/bin/kolla_ensure_openvswitch_configured br-ex + + +## openvswitch-vswitchd +docker run -d --net host \ + --name stackube_openstack_openvswitch_vswitchd \ + -v /etc/stackube/openstack/openvswitch-vswitchd/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /run:/run:shared \ + -v /lib/modules:/lib/modules:ro \ + \ + -e "KOLLA_SERVICE_NAME=openvswitch-vswitchd" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-openvswitch-vswitchd:4.0.0 + +sleep 5 + + +## start_container - neutron-openvswitch-agent +sed -i "s/__OVSDB_IP__/${OVSDB_IP}/g" /etc/stackube/openstack/neutron-openvswitch-agent/ml2_conf.ini +sed -i "s/__LOCAL_IP__/${ML2_LOCAL_IP}/g" /etc/stackube/openstack/neutron-openvswitch-agent/ml2_conf.ini + + +docker run -d --net host \ + --name stackube_openstack_neutron_openvswitch_agent \ + -v /etc/stackube/openstack/neutron-openvswitch-agent/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -v /run:/run:shared \ + -v /lib/modules:/lib/modules:ro \ + \ + -e "KOLLA_SERVICE_NAME=neutron-openvswitch-agent" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + --privileged \ + kolla/centos-binary-neutron-openvswitch-agent:4.0.0 || exit 1 + +exit 0 diff --git a/install/openstack/deploy_openstack_neutron_server.sh b/install/openstack/deploy_openstack_neutron_server.sh new file mode 100644 index 0000000..d22f2e0 --- /dev/null +++ b/install/openstack/deploy_openstack_neutron_server.sh @@ -0,0 +1,133 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``OPENSTACK_ENDPOINT_IP`` +# - ``MYSQL_HOST``, ``MYSQL_ROOT_PWD`` +# - ``KEYSTONE_ADMIN_PWD`` +# - ``KEYSTONE_NEUTRON_PWD``, ``MYSQL_NEUTRON_PWD`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + + +## register - Creating the Neutron service and endpoint +for IF in 'admin' 'internal' 'public'; do + docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m kolla_keystone_service \ + -a "service_name=neutron + service_type=network + description='Openstack Networking' + endpoint_region=RegionOne + url='https://${OPENSTACK_ENDPOINT_IP}:9697/' + interface='${IF}' + region_name=RegionOne + auth='{{ openstack_keystone_auth }}' + verify=False " \ + -e "{'openstack_keystone_auth': { + 'auth_url': 'https://${OPENSTACK_ENDPOINT_IP}:35358/v3', + 'username': 'admin', + 'password': '${KEYSTONE_ADMIN_PWD}', + 'project_name': 'admin', + 'domain_name': 'default' } + }" +done + + +## register - Creating the Neutron project, user, and role +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m kolla_keystone_user \ + -a "project=service + user=neutron + password=${KEYSTONE_NEUTRON_PWD} + role=admin + region_name=RegionOne + auth='{{ openstack_keystone_auth }}' + verify=False " \ + -e "{'openstack_keystone_auth': { + 'auth_url': 'https://${OPENSTACK_ENDPOINT_IP}:35358/v3', + 'username': 'admin', + 'password': '${KEYSTONE_ADMIN_PWD}', + 'project_name': 'admin', + 'domain_name': 'default' } + }" + + +# bootstrap - Creating Neutron database +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m mysql_db \ + -a "login_host=${MYSQL_HOST} + login_port=3306 + login_user=root + login_password=${MYSQL_ROOT_PWD} + name=neutron" + +# bootstrap - Creating Neutron database user and setting permissions +docker exec stackube_openstack_kolla_toolbox /usr/bin/ansible localhost -m mysql_user \ + -a "login_host=${MYSQL_HOST} + login_port=3306 + login_user=root + login_password=${MYSQL_ROOT_PWD} + name=neutron + password=${MYSQL_NEUTRON_PWD} + host=% + priv='neutron.*:ALL' + append_privs=yes" + + + + +## log dir +mkdir -p /var/log/stackube/openstack +chmod 777 /var/log/stackube/openstack + + +# bootstrap_service - Running Neutron bootstrap container +docker run --net host \ + --name stackube_openstack_bootstrap_neutron \ + -v /etc/stackube/openstack/neutron-server/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + -e "KOLLA_BOOTSTRAP=" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + kolla/centos-binary-neutron-server:4.0.0 + +sleep 2 +docker rm stackube_openstack_bootstrap_neutron + + +## start_container - neutron-server +docker run -d --net host \ + --name stackube_openstack_neutron_server \ + -v /etc/stackube/openstack/neutron-server/:/var/lib/kolla/config_files/:ro \ + -v /var/log/stackube/openstack:/var/log/kolla/:rw \ + \ + -e "KOLLA_SERVICE_NAME=neutron-server" \ + -e "KOLLA_CONFIG_STRATEGY=COPY_ALWAYS" \ + \ + --restart unless-stopped \ + kolla/centos-binary-neutron-server:4.0.0 + + + + +exit 0 diff --git a/install/openstack/deploy_openstack_rabbitmq.sh b/install/openstack/deploy_openstack_rabbitmq.sh new file mode 100644 index 0000000..bc4c229 --- /dev/null +++ b/install/openstack/deploy_openstack_rabbitmq.sh @@ -0,0 +1,50 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# +# Dependencies: +# +# - ``RABBITMQ_PWD`` must be defined +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o nounset +set -o pipefail +set -x + +## rabbitmq +mkdir -p /var/lib/stackube/openstack/rabbitmq && \ +docker run -d \ + --name stackube_openstack_rabbitmq \ + --net host \ + -v /var/lib/stackube/openstack/rabbitmq:/var/lib/rabbitmq \ + --restart unless-stopped \ + rabbitmq:3.6 || exit 1 + +sleep 5 +for i in 1 2 3 4 5; do + docker exec stackube_openstack_rabbitmq rabbitmqctl status && break + sleep $i +done +sleep 5 + +docker exec stackube_openstack_rabbitmq rabbitmqctl add_user openstack ${RABBITMQ_PWD} || exit 1 +docker exec stackube_openstack_rabbitmq rabbitmqctl set_permissions openstack ".*" ".*" ".*" || exit 1 + +exit 0 diff --git a/install/openstack/remove_openstack_from_node.sh b/install/openstack/remove_openstack_from_node.sh new file mode 100644 index 0000000..fe6553d --- /dev/null +++ b/install/openstack/remove_openstack_from_node.sh @@ -0,0 +1,42 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -x + +## clean certificates +source ${programDir}/lib_tls.sh || exit 1 +cleanup_CA || exit 1 + + +## remove docker containers +stackubeConstaners=`docker ps -a | awk '{print $NF}' | grep '^stackube_openstack_' ` +if [ "${stackubeConstaners}" ]; then + docker rm -f $stackubeConstaners || exit 1 +fi + +## rm dirs +rm -fr /etc/stackube/openstack /var/log/stackube/openstack /var/lib/stackube/openstack || exit 1 + + + +exit 0 + diff --git a/install/remove.sh b/install/remove.sh new file mode 100644 index 0000000..b936575 --- /dev/null +++ b/install/remove.sh @@ -0,0 +1,108 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + + +function usage { + echo " +Usage: + bash $(basename $0) CONFIG_FILE +" +} + +[ "$1" ] || { usage; exit 1; } +[ -f "$1" ] || { echo "Error: $1 not exists or not a file!"; exit 1; } + +source $(readlink -f $1) || { echo "'source $(readlink -f $1)' failed!"; exit 1; } + + +[ "${CONTROL_NODE_PRIVATE_IP}" ] || { echo "Error: CONTROL_NODE_PRIVATE_IP not defined!"; exit 1; } +[ "${NETWORK_NODES_PRIVATE_IP}" ] || { echo "Error: NETWORK_NODES_PRIVATE_IP not defined!"; exit 1; } +[ "${COMPUTE_NODES_PRIVATE_IP}" ] || { echo "Error: COMPUTE_NODES_PRIVATE_IP not defined!"; exit 1; } +[ "${STORAGE_NODES_PRIVATE_IP}" ] || { echo "Error: STORAGE_NODES_PRIVATE_IP not defined!"; exit 1; } +[ "${STORAGE_NODES_CEPH_OSD_DATA_DIR}" ] || { echo "Error: STORAGE_NODES_CEPH_OSD_DATA_DIR not defined!"; exit 1; } + + +##################### + +set -x + + +## log +logDir='/var/log/stackube' +logFile="${logDir}/remove.log-$(date '+%Y-%m-%d_%H-%M-%S')" +mkdir -p ${logDir} + +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${NETWORK_NODES_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP} +${STORAGE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +{ + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') remove_kubernetes" + remove_kubernetes='' + for i in `seq 1 10`; do + bash ${programDir}/remove_kubernetes.sh $(readlink -f $1) + if [ "$?" == "0" ]; then + remove_kubernetes='done' + break + fi + done + [ "${remove_kubernetes}" == "done" ] || { echo "Error: remove_kubernetes failed !"; exit 1; } + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') remove_openstack" + remove_openstack='' + for i in `seq 1 10`; do + bash ${programDir}/remove_openstack.sh $(readlink -f $1) + if [ "$?" == "0" ]; then + remove_openstack='done' + break + fi + done + [ "${remove_openstack}" == "done" ] || { echo "Error: remove_openstack failed !"; exit 1; } + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') remove_ceph" + remove_ceph='' + for i in `seq 1 10`; do + bash ${programDir}/remove_ceph.sh $(readlink -f $1) + if [ "$?" == "0" ]; then + remove_ceph='done' + break + fi + done + [ "${remove_ceph}" == "done" ] || { echo "Error: remove_ceph failed !"; exit 1; } + + echo -e "\n$(date '+%Y-%m-%d %H:%M:%S') All done!" + +} 2>&1 | tee -a ${logFile} + + +allStats=(${PIPESTATUS[@]}) +if [ "${allStats[0]}" != "0" ]; then + exit 1 +fi + + +exit 0 + + + + diff --git a/install/remove_ceph.sh b/install/remove_ceph.sh new file mode 100644 index 0000000..56157e6 --- /dev/null +++ b/install/remove_ceph.sh @@ -0,0 +1,67 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +source $(readlink -f $1) + +[ "${CONTROL_NODE_PRIVATE_IP}" ] +[ "${STORAGE_NODES_PRIVATE_IP}" ] +[ "${STORAGE_NODES_CEPH_OSD_DATA_DIR}" ] + + +# ceph-mon +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +for IP in ${allIpList}; do + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/ceph/remove_ceph_from_node.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/remove_ceph_from_node.sh" +done + + + +# ceph-osd +storageIpList=(`echo "${STORAGE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g'`) +osdDataDirList=(`echo "${STORAGE_NODES_CEPH_OSD_DATA_DIR}" | sed -e 's/,/\n/g'`) +[ ${#storageIpList[@]} -eq ${#osdDataDirList[@]} ] + +MAX=$((${#storageIpList[@]} - 1)) +for i in `seq 0 ${MAX}`; do + IP="${storageIpList[$i]}" + dataDir="${osdDataDirList[$i]}" + echo -e "\n------ ${IP} ${dataDir} ------" + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/ceph/remove_ceph_from_node.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "export CEPH_OSD_DATA_DIR='${dataDir}' + /bin/bash /tmp/stackube_install/remove_ceph_from_node.sh" +done + + + +exit 0 + diff --git a/install/remove_kubernetes.sh b/install/remove_kubernetes.sh new file mode 100644 index 0000000..f371143 --- /dev/null +++ b/install/remove_kubernetes.sh @@ -0,0 +1,49 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +source $(readlink -f $1) + +[ "${CONTROL_NODE_PRIVATE_IP}" ] +[ "${COMPUTE_NODES_PRIVATE_IP}" ] + + + +## all nodes +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +# hyperd frakti +for IP in ${allIpList}; do + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/kubernetes/remove_kubernetes_from_node.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/remove_kubernetes_from_node.sh" +done + + +exit 0 + diff --git a/install/remove_openstack.sh b/install/remove_openstack.sh new file mode 100644 index 0000000..a9345d1 --- /dev/null +++ b/install/remove_openstack.sh @@ -0,0 +1,53 @@ +#!/bin/bash +# Copyright (c) 2017 OpenStack Foundation. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +programDir=`dirname $0` +programDir=$(readlink -f $programDir) +parentDir="$(dirname $programDir)" +programDirBaseName=$(basename $programDir) + +set -o errexit +set -o nounset +set -o pipefail +set -x + + +source $(readlink -f $1) + +[ "${CONTROL_NODE_PRIVATE_IP}" ] +[ "${NETWORK_NODES_PRIVATE_IP}" ] +[ "${COMPUTE_NODES_PRIVATE_IP}" ] +[ "${STORAGE_NODES_PRIVATE_IP}" ] + + + +allIpList=`echo " +${CONTROL_NODE_PRIVATE_IP} +${NETWORK_NODES_PRIVATE_IP} +${COMPUTE_NODES_PRIVATE_IP} +${STORAGE_NODES_PRIVATE_IP}" | sed -e 's/,/\n/g' | sort | uniq ` + +for IP in ${allIpList}; do + ssh root@${IP} 'mkdir -p /tmp/stackube_install' + scp ${programDir}/openstack/remove_openstack_from_node.sh root@${IP}:/tmp/stackube_install/ + scp ${programDir}/lib_tls.sh root@${IP}:/tmp/stackube_install/ + ssh root@${IP} "/bin/bash /tmp/stackube_install/remove_openstack_from_node.sh" +done + + + +exit 0 +