openstack/trove
Code Issues Proposed changes

Improve trove guest agent image building

Browse Source 
- Add a new element 'guest-agent' for image building. This element is
  used when dev_mode=false, so that the trove code is downloaded into
  the image during the building phase rather than during the guest
  agent initialization.
- Improve trovestack sub-command 'build-image'.

  ./trovestack build-image ${datastore_type} \
     ${guest_os} \
     ${guest_release} \
     ${dev_mode}

- Improve documentation.

Story: #2005387
Task: #30375

Change-Id: I9d7acbd6a97f8c01b48b0f2cf94398d549d89124
This commit is contained in:
Lingxian Kong 2019-04-07 00:27:15 +13:00
parent fc1c246516
commit 798af4d02a
26 changed files with 675 additions and 687 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
devstack/plugin.sh
View File

@ -631,6 +631,8 @@ function _setup_minimal_image {
if [ -d ${SSH_DIR} ]; then if [ -d ${SSH_DIR} ]; then
cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys
sort ${SSH_DIR}/authorized_keys | uniq > ${SSH_DIR}/authorized_keys.uniq
mv ${SSH_DIR}/authorized_keys.uniq ${SSH_DIR}/authorized_keys
else else
mkdir -p ${SSH_DIR} mkdir -p ${SSH_DIR}
/usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q -N "" /usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q -N ""
@ -638,6 +640,13 @@ function _setup_minimal_image {
chmod 600 ${SSH_DIR}/authorized_keys chmod 600 ${SSH_DIR}/authorized_keys
fi fi
# Make sure the guest agent has permission to ssh into the devstack host
# in order to download trove code during the service initialization.
home_keys=$HOME/.ssh/authorized_keys
cat ${SSH_DIR}/id_rsa.pub >> ${home_keys}
sort ${home_keys} | uniq > ${home_keys}.uniq
mv ${home_keys}.uniq ${home_keys}
echo "Run disk image create to actually create a new image" echo "Run disk image create to actually create a new image"
disk-image-create -a amd64 -o "${VM}" -x ${QEMU_IMG_OPTIONS} ${DISTRO} \ disk-image-create -a amd64 -o "${VM}" -x ${QEMU_IMG_OPTIONS} ${DISTRO} \
vm cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${RELEASE}-guest \ vm cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${RELEASE}-guest \
@ -723,6 +732,10 @@ if is_service_enabled trove; then
# Start the trove API and trove taskmgr components # Start the trove API and trove taskmgr components
echo_summary "Starting Trove" echo_summary "Starting Trove"
start_trove start_trove
# Guarantee the file permission in the trove code repo in order to
# download trove code from trove-guestagent.
sudo chown -R $STACK_USER:$STACK_USER "$DEST/trove"
elif [[ "$1" == "stack" && "$2" == "test-config" ]]; then elif [[ "$1" == "stack" && "$2" == "test-config" ]]; then
echo_summary "Configuring Tempest for Trove" echo_summary "Configuring Tempest for Trove"
configure_tempest_for_trove configure_tempest_for_trove

446
doc/source/admin/building_guest_images.rst
View File

@ -85,12 +85,12 @@ Injected Configuration for the Guest Agent
When TaskManager launches the guest VM it injects the specific settings When TaskManager launches the guest VM it injects the specific settings
for the guest into the VM, into the file /etc/trove/conf.d/guest_info.conf. for the guest into the VM, into the file /etc/trove/conf.d/guest_info.conf.
The file is injected one of three ways. If use_heat=True, it is injected The file is injected one of three ways.
during the heat launch process. If use_nova_server_config_drive=True
it is injected via ConfigDrive. Otherwise it is passed to the nova If use_nova_server_config_drive=True, it is injected via ConfigDrive. Otherwise
create call as the 'files' parameter and will be injected based on it is passed to the nova create call as the 'files' parameter and will be
the configuration of Nova; the Nova default is to discard the files. injected based on the configuration of Nova; the Nova default is to discard the
If the settings in guest_info.conf are not present on the guest files. If the settings in guest_info.conf are not present on the guest
Guest Agent will fail to start up. Guest Agent will fail to start up.
------------------------------ ------------------------------
@ -108,135 +108,23 @@ Building Guest Images using DIB
A Trove Guest Image can be built with any tool that produces an image A Trove Guest Image can be built with any tool that produces an image
accepted by Nova. In this document we describe how to build guest accepted by Nova. In this document we describe how to build guest
images using the 'Disk Image Builder' (DIB) tool, and we focus on images using the
building qemu images [1]_. DIB is an OpenStack tool and is available for `'Disk Image Builder' (DIB) <https://docs.openstack.org/diskimage-builder/latest/>`_
download at tool, and we focus on building qemu images.
https://git.openstack.org/cgit/openstack/diskimage-builder/tree/ or
https://pypi.org/project/diskimage-builder/0.1.38.
DIB uses a chroot'ed environment to construct the image. The goal is DIB uses a chroot'ed environment to construct the image. The goal is
to build a bare machine that has all the components required for to build a bare machine that has all the components required for
launch by Nova. launch by Nova.
---------- ----------------------------
Invocation Build image using trovestack
---------- ----------------------------
You can download the DIB tool from OpenStack's public git Trove provides a script called ``trovestack`` that could do most of the
repository. Note that DIB works with Ubuntu and Fedora (RedHat). Other management and test tasks. Refer to the "Build guest agent image" section
operating systems are not yet fully supported. in
`trovestack document <https://github.com/openstack/trove/blob/master/integration/README.md>`_
.. code-block:: bash for how to build trove guest agent images.
user@machine:/opt/stack$ git clone https://git.openstack.org/openstack/diskimage-builder
Cloning into 'diskimage-builder'...
remote: Counting objects: 8881, done.
remote: Total 8881 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (8881/8881), 1.92 MiB | 0 bytes/s, done.
Resolving deltas: 100% (4668/4668), done.
Checking connectivity... done.
user@machine:/opt/stack$ cd diskimage-builder
user@machine:/opt/stack/diskimage-builder$ sudo pip install -r requirements.txt
user@machine:/opt/stack/diskimage-builder$ sudo python setup.py install
Ensure that you have qemu-img [2]_ and kpartx installed.
The disk-image-create command is the main command in the DIB tool that
is used to build guest images for Trove. The disk-image-create command
takes the following options:
.. code-block:: bash
user@machine:/opt/stack/diskimage-builder$ disk-image-create -h
Usage: disk-image-create [OPTION]... [ELEMENT]...
Options:
-a i386|amd64|armhf -- set the architecture of the image(default amd64)
-o imagename -- set the imagename of the output image file(default image)
-t qcow2,tar -- set the image types of the output image files (default qcow2)
File types should be comma separated
-x -- turn on tracing
-u -- uncompressed; do not compress the image - larger but faster
-c -- clear environment before starting work
--image-size size -- image size in GB for the created image
--image-cache directory -- location for cached images(default ~/.cache/image-create)
--max-online-resize size -- max number of filesystem blocks to support when resizing.
Useful if you want a really large root partition when the image is deployed.
Using a very large value may run into a known bug in resize2fs.
Setting the value to 274877906944 will get you a 1PB root file system.
Making this value unnecessarily large will consume extra disk space
on the root partition with extra file system inodes.
--min-tmpfs size -- minimum size in GB needed in tmpfs to build the image
--no-tmpfs -- do not use tmpfs to speed image build
--offline -- do not update cached resources
--qemu-img-options -- option flags to be passed directly to qemu-img.
Options need to be comma separated, and follow the key=value pattern.
--root-label label -- label for the root filesystem. Defaults to 'cloudimg-rootfs'.
--ramdisk-element -- specify the main element to be used for building ramdisks.
Defaults to 'ramdisk'. Should be set to 'dracut-ramdisk' for platforms such
as RHEL and CentOS that do not package busybox.
--install-type -- specify the default installation type. Defaults to 'source'. Set
to 'package' to use package based installations by default.
-n skip the default inclusion of the 'base' element
-p package[,package,package] -- list of packages to install in the image
-h|--help -- display this help and exit
ELEMENTS_PATH will allow you to specify multiple locations for the elements.
NOTE: At least one distribution root element must be specified.
Examples:
disk-image-create -a amd64 -o ubuntu-amd64 vm ubuntu
export ELEMENTS_PATH=~/source/tripleo-image-elements/elements
disk-image-create -a amd64 -o fedora-amd64-heat-cfntools vm fedora heat-cfntools
user@machine:/opt/stack/diskimage-builder$
The example command provided above would build a perfectly functional
Nova image with the 64 bit Fedora operating system.
In addition to the -a argument which specifies to build an amd64 (64
bit) image, and the -o which specifies the output file, the command
line lists the various elements that should be used in building the
image. The next section of this document talks about image elements.
Building a Trove guest image is a little more involved and the standard
elements (more about this later) are highly configurable through the use
of environment variables.
This command will create a guest image usable by Trove:
.. code-block:: bash
# assign a suitable value for each of these environment
# variables that change the way the elements behave.
export HOST_USERNAME
export HOST_SCP_USERNAME
export GUEST_USERNAME
export CONTROLLER_IP
export TROVESTACK_SCRIPTS
export SERVICE_TYPE
export PATH_TROVE
export ESCAPED_PATH_TROVE
export SSH_DIR
export GUEST_LOGDIR
export ESCAPED_GUEST_LOGDIR
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive"
export DATASTORE_PKG_LOCATION
export BRANCH_OVERRIDE
# you typically do not have to change these variables
export ELEMENTS_PATH=$TROVESTACK_SCRIPTS/files/elements
export ELEMENTS_PATH+=:$PATH_DISKIMAGEBUILDER/elements
export ELEMENTS_PATH+=:$PATH_TRIPLEO_ELEMENTS/elements
export DIB_APT_CONF_DIR=/etc/apt/apt.conf.d
export DIB_CLOUD_INIT_ETC_HOSTS=true
local QEMU_IMG_OPTIONS="--qemu-img-options compat=1.1"
# run disk-image-create that actually causes the image to be built
$disk-image-create -a amd64 -o "${VM}" \
-x ${QEMU_IMG_OPTIONS} ${DISTRO} ${EXTRA_ELEMENTS} vm \
cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${SERVICE_TYPE}
----------------------------- -----------------------------
Disk Image Builder 'Elements' Disk Image Builder 'Elements'
@ -248,106 +136,16 @@ scripts that are executed by DIB in a specific order to generate the
image. You provide the names of the elements that you would like image. You provide the names of the elements that you would like
executed, in order, on the command line to disk-image-create. executed, in order, on the command line to disk-image-create.
Elements are executed within the chroot'ed environment while DIB is DIB comes with some
run. Elements are executed in phases and the various phases are (in `built-in elements <https://docs.openstack.org/diskimage-builder/latest/elements.html>`_.
order) root.d, extra-data.d, pre-install.d, install.d, post-install.d, In addition, projects like
block-device.d, finalise.d [3]_, and cleanup.d [4]_. The latter `TripleO <https://github.com/openstack/tripleo-image-elements>`_ provide
reference provides a very good outline on writing elements and is a elements as well.
'must read'.
Some elements use environment.d to setup environment Trove also provides a set of its own elements. In keeping with the philosophy
variables. Element dependencies can be established using the of making elements 'layered', Trove provides two sets of elements. The first
element-deps and element-provides files which are plain text files. implements the guest agent for various operating systems and the second
implements the database for these operating systems.
-----------------
Existing Elements
-----------------
DIB comes with some tools that are located in the elements directory.
.. code-block:: bash
user@machine:/opt/stack/diskimage-builder/elements$ ls
apt-conf dpkg ramdisk
apt-preferences dracut-network ramdisk-base
apt-sources dracut-ramdisk rax-nova-agent
architecture-emulation-binaries element-manifest redhat-common
baremetal enable-serial-console rhel
base epel rhel7
cache-url fedora rhel-common
centos7 hwburnin rpm-distro
cleanup-kernel-initrd hwdiscovery select-boot-kernel-initrd
cloud-init-datasources ilo selinux-permissive
cloud-init-nocloud ironic-agent serial-console
debian ironic-discoverd-ramdisk source-repositories
debian-systemd iso stable-interface-names
debian-upstart local-config svc-map
deploy manifests uboot
deploy-baremetal mellanox ubuntu
deploy-ironic modprobe-blacklist ubuntu-core
deploy-kexec opensuse vm
dhcp-all-interfaces package-installs yum
dib-run-parts pip-cache zypper
disable-selinux pkg-map
dkms pypi
In addition, projects like TripleO [5]_ provide elements as well.
Trove provides a set of elements as part of the trove [6]_
project which will be described in the next section.
Trove Reference Elements
========================
Reference elements provided by Trove are part of the trove project.
In keeping with the philosophy of making elements 'layered', Trove
provides two sets of elements. The first implements the guest agent
for various operating systems and the second implements the database
for these operating systems.
---------------------------
Provided Reference Elements
---------------------------
The Trove reference elements are located in the
trove/integration/scripts/files/elements directory. The elements
[operating-system]-guest provide the Trove Guest capabilities and the
[operating-system]-[database] elements provide support for each
database on the specified database.
.. code-block:: bash
user@machine:/opt/stack/trove/integration/scripts/files/elements$ ls -l
total 56
drwxrwxr-x 5 user group 4096 Jan 7 12:47 fedora-guest
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-mongodb
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-mysql
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-percona
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-postgresql
drwxrwxr-x 3 user group 4096 Jan 7 12:47 fedora-redis
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-cassandra
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-couchbase
drwxrwxr-x 6 user group 4096 Jan 7 12:47 ubuntu-guest
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-mongodb
drwxrwxr-x 4 user group 4096 Jan 7 12:47 ubuntu-mysql
drwxrwxr-x 4 user group 4096 Jan 7 12:47 ubuntu-percona
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-postgresql
drwxrwxr-x 3 user group 4096 Jan 7 12:47 ubuntu-redis
user@machine:/opt/stack/trove/integration/scripts/files/elements$
With this infrastructure in place, and the elements from DIB and
TripleO accessible to the DIB command, one can generate the (for
example) Ubuntu guest image for Percona Server with the command line:
.. code-block:: bash
${DIB} -a amd64 -o ${output-file} Ubuntu vm \
cloud-init-datasources ubuntu-guest ubuntu-percona
Where ${DIB} is the fully qualified path to the disk-image-create
command and ${output-file} is the name of the output file to be
created.
------------------------------------------------------------------- -------------------------------------------------------------------
Contributing Reference Elements When Implementing a New 'Datastore' Contributing Reference Elements When Implementing a New 'Datastore'
@ -357,81 +155,6 @@ When contributing a new datastore, you should contribute elements
that will allow any user of Trove to be able to build a guest image that will allow any user of Trove to be able to build a guest image
for that datastore. for that datastore.
This is typically accomplished by submitting files into the
trove project, as above.
Getting the Guest Agent Code onto a Trove Guest Instance
========================================================
The guest agent code typically runs on the guest instance alongside
the database. There are two ways in which the guest agent code can be
placed on the guest instance and we describe both of these here.
----------------------------------------
Guest Agent Code Installed at Build Time
----------------------------------------
In this option, the guest agent code is built into the guest image,
thereby ensuring that all database instances that are launched with
the image will have the exact same version of the guest image.
This can be accomplished by placing suitable code in the elements for
the image and these elements will ensure that the guest agent code is
installed on the image.
--------------------------------------
Guest Agent Code Installed at Run Time
--------------------------------------
In this option, the guest agent code is not part of the guest image
and instead the guest agent code is obtained at runtime, potentially
from some well known location.
In devstack, this is implemented in trove-guest.upstart.conf and
trove-guest.systemd.conf. Shown below is the code from
trove-guest.upstart.conf (this code may change in the future and
is shown here as an example only). See the code highlighted below:
.. code-block:: bash
description "Trove Guest"
author "Auto-Gen"
start on (filesystem and net-device-up IFACE!=lo)
stop on runlevel [016]
chdir /var/run
pre-start script
mkdir -p /var/run/trove
chown GUEST_USERNAME:root /var/run/trove/
mkdir -p /var/lock/trove
chown GUEST_USERNAME:root /var/lock/trove/
mkdir -p GUEST_LOGDIR
chown GUEST_USERNAME:root GUEST_LOGDIR
chmod +r /etc/guest_info
# If /etc/trove does not exist, copy the trove source and the
# guest agent config from the user's development environment
if [ ! -d /etc/trove ]; then
-> sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@NETWORK_GATEWAY:PATH_TROVE/ /home/GUEST_USERNAME/trove
mkdir -p /etc/trove
-> sudo -u GUEST_USERNAME rsync -e 'ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no' -avz --exclude='.*' HOST_SCP_USERNAME@NETWORK_GATEWAY:/etc/trove/trove-guestagent.conf ~GUEST_USERNAME/
mv ~GUEST_USERNAME/trove-guestagent.conf /etc/trove/trove-guestagent.conf
fi
end script
exec su -c "/home/GUEST_USERNAME/trove/contrib/trove-guestagent -config-file=/etc/guest_info --config-file=/etc/trove/trove-guestagent.conf" GUEST_USERNAME
In building an image for a production Trove deployment, it is a very
bad idea to use this mechanism. It makes sense in a development
environment where the thing that you are developing is in Trove and
part of the Guest Agent! This is because you get to merely boot a new
Trove instance and the freshly modified code gets run on the
Guest. But, in any other circumstance, it is much better to have the
guest image include the guest agent code.
Considerations in Building a Guest Image Considerations in Building a Guest Image
======================================== ========================================
@ -522,120 +245,3 @@ repository, this would equate to:
.. code-block:: bash .. code-block:: bash
DATASTORE_PKG_LOCATION=/path/to/new_db.deb ./trovestack kick-start new_db DATASTORE_PKG_LOCATION=/path/to/new_db.deb ./trovestack kick-start new_db
Building Guest Images Using Standard Elements
=============================================
A very good reference for how one builds guest images can be found by
reviewing the trovestack script (trove/integration/scripts). Lower level
routines that actually invoke Disk Image Builder can be found in
trove/integration/scripts/functions_qemu.
The following block of code illustrates the most basic invocation of
DIB to create a guest image. This code is in
trove/integration/scripts/functions_qemu as part of the function
build_vm(). We look at this section of code in detail below.
.. code-block:: bash
# assign a suitable value for each of these environment
# variables that change the way the elements behave.
export HOST_USERNAME
export HOST_SCP_USERNAME
export GUEST_USERNAME
export CONTROLLER_IP
export TROVESTACK_SCRIPTS
export SERVICE_TYPE
export PATH_TROVE
export ESCAPED_PATH_TROVE
export SSH_DIR
export GUEST_LOGDIR
export ESCAPED_GUEST_LOGDIR
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive"
export DATASTORE_PKG_LOCATION
export BRANCH_OVERRIDE
# you typically do not have to change these variables
export ELEMENTS_PATH=$TROVESTACK_SCRIPTS/files/elements
export ELEMENTS_PATH+=:$PATH_DISKIMAGEBUILDER/elements
export ELEMENTS_PATH+=:$PATH_TRIPLEO_ELEMENTS/elements
export DIB_APT_CONF_DIR=/etc/apt/apt.conf.d
export DIB_CLOUD_INIT_ETC_HOSTS=true
local QEMU_IMG_OPTIONS="--qemu-img-options compat=1.1"
# run disk-image-create that actually causes the image to be built
$disk-image-create -a amd64 -o "${VM}" \
-x ${QEMU_IMG_OPTIONS} ${DISTRO} ${EXTRA_ELEMENTS} vm \
cloud-init-datasources ${DISTRO}-guest ${DISTRO}-${SERVICE_TYPE}
Several of the environment variables referenced above are referenced
in the course of the Disk Image Building process.
For example, let's look at GUEST_LOGDIR. Looking at the element
elements/fedora-guest/extra-data.d/20-guest-systemd, we find:
.. code-block:: bash
#!/bin/bash
set -e
set -o xtrace
# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER
# PURPOSE: stages the bootstrap file and upstart conf file while replacing variables so that guest image is properly
# configured
source $_LIB/die
[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set"
[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image"
[ -n "${HOST_SCP_USERNAME}" ] || die "HOST_SCP_USERNAME needs to be set to the user for the host instance"
[ -n "${CONTROLLER_IP}" ] || die "CONTROLLER_IP needs to be set to the ip address that guests will use to contact the controller"
[ -n "${ESCAPED_PATH_TROVE}" ] || die "ESCAPED_PATH_TROVE needs to be set to the path to the trove directory on the trovestack host"
[ -n "${TROVESTACK_SCRIPTS}" ] || die "TROVESTACK_SCRIPTS needs to be set to the trove/integration/scripts dir"
[ -n "${ESCAPED_GUEST_LOGDIR}" ] || die "ESCAPED_GUEST_LOGDIR must be set to the escaped guest log dir"
sed "s/GUEST_USERNAME/${GUEST_USERNAME}/g;s/GUEST_LOGDIR/${ESCAPED_GUEST_LOGDIR}/g;s/HOST_SCP_USERNAME/${HOST_SCP_USERNAME}/g;s/CONTROLLER_IP/${CONTROLLER_IP}/g;s/PATH_TROVE/${ESCAPED_PATH_TROVE}/g" \
${TROVESTACK_SCRIPTS}/files/trove-guest.systemd.conf >
${TMP_HOOKS_PATH}/trove-guest.service
As you can see, the value of GUEST_LOGDIR is used in the extra-data.d
script to appropriately configure the trove-guest.systemd.conf file.
This pattern is one that you can expect in your own building of guest
images. The invocation of disk-image-create provides a list of
elements that are to be invoked 'in order'.
That list of elements is:
.. code-block:: bash
${DISTRO}
${EXTRA_ELEMENTS}
vm
cloud-init-datasources
${DISTRO}-guest
${DISTRO}-${SERVICE_TYPE}
When invoked to (for example) create a MySQL guest image on Ubuntu, we
can expect that DISTRO would be 'Ubuntu' and SERVICE_TYPE would be
MySQL. And therefore these would end up being the elements:
.. code-block:: bash
ubuntu From diskimage-builder/elements/ubuntu
vm From diskimage-builder/elements/vm
cloud-init-datasources From diskimage-builder/elements/cloud-init-datasources
ubuntu-guest From trove/integration/scripts/files/elements/ubuntu-guest
ubuntu-mysql From trove/integration/scripts/files/elements/ubuntu-mysql
References
==========
.. [1] For more information about QEMU, refer to http://wiki.qemu.org/Main_Page
.. [2] On Ubuntu, qemu-img is part of the package qemu-utils, on Fedora and RedHat it is part of the qemu package.
.. [3] User (especially in the USA) are cautioned about this spelling which once resulted in several sleepless nights.
.. [4] https://git.openstack.org/cgit/openstack/diskimage-builder/tree/README.rst#writing-an-element
.. [5] https://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements
.. [6] https://git.openstack.org/cgit/openstack/trove/tree/integration/scripts/files/elements

184
integration/README.md
View File

@ -1,93 +1,92 @@
## Integration dev scripts, tests and docs for Trove. # Trove integration script - trovestack
*** ## Steps to setup environment
### Steps to setup this environment: Install a fresh Ubuntu 16.04 (xenial) image. We suggest creating a development virtual machine using the image.
Install a fresh Ubuntu 16.04 (xenial) image ( _We suggest creating a development virtual machine using the image_ ) 1. Login to the machine as root
1. Make sure we have git installed
#### Login to the machine as root
#### Make sure we have git installed:
```
# apt-get update # apt-get update
# apt-get install git-core -y # apt-get install git-core -y
```
#### Add a user named ubuntu if you do not already have one: 1. Add a user named ubuntu if you do not already have one:
```
# adduser ubuntu # adduser ubuntu
# visudo # visudo
```
add this line to the file below the root user Add this line to the file below the root user
ubuntu ALL=(ALL:ALL) ALL ubuntu ALL=(ALL:ALL) ALL
**OR use this if you dont want to type your password to sudo a command** Or use this if you dont want to type your password to sudo a command:
ubuntu ALL=(ALL) NOPASSWD: ALL ubuntu ALL=(ALL) NOPASSWD: ALL
if /dev/pts/0 does not have read/write for your user if /dev/pts/0 does not have read/write for your user
# chmod 666 /dev/pts/0 # chmod 666 /dev/pts/0
*Note that this number can change and if you can not connect to the screen session then the /dev/pts/# needs modding like above.* > Note that this number can change and if you can not connect to the screen session then the /dev/pts/# needs modding like above.
#### Login with ubuntu: 1. Login with ubuntu and download the Trove code.
```shell
# su ubuntu # su ubuntu
$ mkdir -p /opt/stack $ mkdir -p /opt/stack
$ cd /opt/stack $ cd /opt/stack
```
*Note that it is important that you clone the repository > Note that it is important that you clone the repository
here. This is a change from the earlier trove-integration where here. This is a change from the earlier trove-integration where
you could clone trove-integration anywhere you wanted (like HOME) you could clone trove-integration anywhere you wanted (like HOME)
and trove would get cloned for you in the right place. Since and trove would get cloned for you in the right place. Since
trovestack is now in the trove repository, if you wish to test trovestack is now in the trove repository, if you wish to test
changes that you have made to trove, it is advisable for you to changes that you have made to trove, it is advisable for you to
have your trove repository in /opt/stack to avoid another trove have your trove repository in /opt/stack to avoid another trove
repository being cloned for you. repository being cloned for you.
#### Clone this repo:
1. Clone this repo and go into the scripts directory
```
$ git clone https://github.com/openstack/trove.git $ git clone https://github.com/openstack/trove.git
#### Go into the scripts directory:
$ cd trove/integration/scripts/ $ cd trove/integration/scripts/
```
#### Running trovestack: ## Running trovestack
*Run this to get the command list with a short description of each*
Run this to get the command list with a short description of each
$ ./trovestack $ ./trovestack
#### Install all the dependencies and then install trove via trovestack. ### Install Trove
*This brings up trove (rd-api rd-tmgr) and initializes the trove database.* *This brings up trove services and initializes the trove database.*
$ ./trovestack install $ ./trovestack install
*** ### Connecting to the screen session
#### Connecting to the screen session
$ screen -x stack $ screen -x stack
*If that command fails with the error* If that command fails with the error
Cannot open your terminal '/dev/pts/1' Cannot open your terminal '/dev/pts/1'
*If that command fails with the error chmod the corresponding /dev/pts/#* If that command fails with the error chmod the corresponding /dev/pts/#
$ chmod 660 /dev/pts/1 $ chmod 660 /dev/pts/1
#### Navigate the log screens ### Navigate the log screens
To produce the list of screens that you can scroll through and select To produce the list of screens that you can scroll through and select
ctrl+a then " ctrl+a then "
Num Name An example of screen list:
```
..... (full list ommitted) ..... (full list ommitted)
20 c-vol 20 c-vol
21 h-eng 21 h-eng
22 h-api 22 h-api
@ -96,6 +95,7 @@ Num Name
25 tr-api 25 tr-api
26 tr-tmgr 26 tr-tmgr
27 tr-cond 27 tr-cond
```
Alternatively, to go directly to a specific screen window Alternatively, to go directly to a specific screen window
@ -103,58 +103,47 @@ Alternatively, to go directly to a specific screen window
then enter a number (like 25) or name (like tr-api) then enter a number (like 25) or name (like tr-api)
#### Detach from the screen session ### Detach from the screen session
Allows the services to continue running in the background Allows the services to continue running in the background
ctrl+a then d ctrl+a then d
*** ### Kick start the build/test-init/build-image commands
#### Kick start the build/test-init/build-image commands
*Add mysql as a parameter to set build and add the mysql guest image. This will also populate /etc/trove/test.conf with appropriate values for running the integration tests.* *Add mysql as a parameter to set build and add the mysql guest image. This will also populate /etc/trove/test.conf with appropriate values for running the integration tests.*
$ ./trovestack kick-start mysql $ ./trovestack kick-start mysql
*Optional commands if you did not run kick-start* ### Initialize the test configuration and set up test users (overwrites /etc/trove/test.conf)
#### Initialize the test configuration and set up test users (overwrites /etc/trove/test.conf)
$ ./trovestack test-init $ ./trovestack test-init
#### Build the image and add it to glance ### Build guest agent image
The trove guest agent image could be created using `trovestack` script
according to the following command:
$ ./trovestack build-image mysql ```shell
PATH_DEVSTACK_OUTPUT=/opt/stack \
./trovestack build-image \
${datastore_type} \
${guest_os} \
${guest_os_release} \
${dev_mode}
```
*** - If the script is running as a part of DevStack, the viriable
`PATH_DEVSTACK_OUTPUT` is set automatically.
- if `dev_mode=false`, the trove code for guest agent is injected into the
image at the building time. Now `dev_mode=false` is still in experimental
and not considered production ready yet.
- If `dev_mode=true`, the script assumes to be running on the host of
trove-taskmanager, otherwise, `CONTROLLER_IP` needs to be specified
explicitly.
### Reset your environment For example, build a Mysql image for Ubuntu Xenial operating system:
#### Stop all the services running in the screens and refresh the environment: ```shell
$ ./trovestack build-image mysql ubuntu xenial false
$ killall -9 screen ```
$ screen -wipe
$ RECLONE=yes ./trovestack install
$ ./trovestack kick-start mysql
or
$ RECLONE=yes ./trovestack install
$ ./trovestack test-init
$ ./trovestack build-image mysql
***
### Recover after reboot
If the VM was restarted, then the process for bringing up Openstack and Trove is quite simple
$./trovestack start-deps
$./trovestack start
Use screen to ensure all modules have started without error
$screen -r stack
***
### Running Integration Tests ### Running Integration Tests
Check the values in /etc/trove/test.conf in case it has been re-initialized prior to running the tests. For example, from the previous mysql steps: Check the values in /etc/trove/test.conf in case it has been re-initialized prior to running the tests. For example, from the previous mysql steps:
@ -167,21 +156,44 @@ should be:
"dbaas_datastore": "mysql", "dbaas_datastore": "mysql",
"dbaas_datastore_version": "5.5", "dbaas_datastore_version": "5.5",
Once Trove is running on DevStack, you can use the dev scripts to run the integration tests locally. Once Trove is running on DevStack, you can run the integration tests locally.
$./trovestack int-tests $./trovestack int-tests
This will runs all of the blackbox tests by default. Use the --group option to run a different group: This will runs all of the blackbox tests by default. Use the `--group` option to run a different group:
$./trovestack int-tests --group=simple_blackbox $./trovestack int-tests --group=simple_blackbox
You can also specify the TESTS_USE_INSTANCE_ID environment variable to have the integration tests use an existing instance for the tests rather than creating a new one. You can also specify the `TESTS_USE_INSTANCE_ID` environment variable to have the integration tests use an existing instance for the tests rather than creating a new one.
$./TESTS_DO_NOT_DELETE_INSTANCE=True TESTS_USE_INSTANCE_ID=INSTANCE_UUID ./trovestack int-tests --group=simple_blackbox $./TESTS_DO_NOT_DELETE_INSTANCE=True TESTS_USE_INSTANCE_ID=INSTANCE_UUID ./trovestack int-tests --group=simple_blackbox
*** ## Reset your environment
### VMware Fusion 5 speed improvement ### Stop all the services running in the screens and refresh the environment
$ killall -9 screen
$ screen -wipe
$ RECLONE=yes ./trovestack install
$ ./trovestack kick-start mysql
or
$ RECLONE=yes ./trovestack install
$ ./trovestack test-init
$ ./trovestack build-image mysql
## Recover after reboot
If the VM was restarted, then the process for bringing up Openstack and Trove is quite simple
$./trovestack start-deps
$./trovestack start
Use screen to ensure all modules have started without error
$screen -r stack
## VMware Fusion 5 speed improvement
Running Ubuntu with KVM or Qemu can be extremely slow without certain optimizations. The following are some VMware settings that can improve performance and may also apply to other virtualization platforms. Running Ubuntu with KVM or Qemu can be extremely slow without certain optimizations. The following are some VMware settings that can improve performance and may also apply to other virtualization platforms.
1. Shutdown the Ubuntu VM. 1. Shutdown the Ubuntu VM.
@ -203,7 +215,7 @@ INFO: /dev/kvm exists
KVM acceleration can be used KVM acceleration can be used
``` ```
### VMware Workstation performance improvements ## VMware Workstation performance improvements
In recent versions of VMWare, you can get much better performance if you enable the right virtualization options. For example, in VMWare Workstation (found in version 10.0.2), click on VM->Settings->Processor. In recent versions of VMWare, you can get much better performance if you enable the right virtualization options. For example, in VMWare Workstation (found in version 10.0.2), click on VM->Settings->Processor.

6
integration/scripts/files/elements/guest-agent/README.rst Normal file
View File

@ -0,0 +1,6 @@
Element to install an Trove guest agent.
Note: this requires a system base image modified to include OpenStack
repositories
the ubuntu-guest element could be removed.

5
integration/scripts/files/elements/guest-agent/element-deps Normal file
View File

@ -0,0 +1,5 @@
dib-init-system
package-installs
pkg-map
source-repositories
svc-map

34
integration/scripts/files/elements/guest-agent/environment.d/99-reliable-apt-key-importing.bash Normal file
View File

@ -0,0 +1,34 @@
# sometimes the primary key server is unavailable and we should try an
# alternate. see
# https://bugs.launchpad.net/percona-server/+bug/907789. Disable
# shell errexit so we can interrogate the exit code and take action
# based on the exit code. We will reenable it later.
#
# NOTE(zhaochao): we still have this problem from time to time, so it's
# better use more reliable keyservers and just retry on that(for now, 3
# tries should be fine).
# According to:
# [1] https://www.gnupg.org/faq/gnupg-faq.html#new_user_default_keyserver
# [2] https://sks-keyservers.net/overview-of-pools.php
# we'll just the primary suggested pool: pool.sks-keyservers.net.
function get_key_robust() {
KEY=$1
set +e
tries=1
while [ $tries -le 3 ]; do
if [ $tries -eq 3 ]; then
set -e
fi
echo "Importing the key, try: $tries"
apt-key adv --keyserver hkp://pool.sks-keyservers.net \
--recv-keys ${KEY} && break
tries=$((tries+1))
done
set -e
}
export -f get_key_robust

28
integration/scripts/files/elements/guest-agent/extra-data.d/11-ssh-key Executable file
View File

@ -0,0 +1,28 @@
#!/bin/bash
set -e
set -o xtrace
# CONTEXT: HOST prior to IMAGE BUILD as SCRIPT USER
# PURPOSE: creates the SSH key on the host if it doesn't exist. Then this copies the keys over to a staging area where
# they will be duplicated in the guest VM.
# This process allows the host to log into the guest but more importantly the guest phones home to get the trove
# source
source $_LIB/die
HOST_USERNAME=${HOST_USERNAME:-"ubuntu"}
SSH_DIR=${SSH_DIR:-"/home/${HOST_USERNAME}/.ssh"}
[ -n "${TMP_HOOKS_PATH}" ] || die "Temp hook path not set"
# copy files over the "staging" area for the guest image (they'll later be put in the correct location by the guest user
# not these keys should not be overridden otherwise a) you won't be able to ssh in and b) the guest won't be able to
# rsync the files
if [ -f ${SSH_DIR}/id_rsa ]; then
dd if=${SSH_DIR}/authorized_keys of=${TMP_HOOKS_PATH}/ssh-authorized-keys
dd if=${SSH_DIR}/id_rsa of=${TMP_HOOKS_PATH}/ssh-id_rsa
dd if=${SSH_DIR}/id_rsa.pub of=${TMP_HOOKS_PATH}/ssh-id_rsa.pub
else
die "SSH Authorized Keys file must exist along with pub and private key"
fi

38
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/75-guest-agent-install Executable file
View File

@ -0,0 +1,38 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
SCRIPTDIR=$(dirname $0)
GUEST_VENV=/opt/guest-agent-venv
# Create a virtual environment to contain the guest agent
${DIB_PYTHON} -m virtualenv $GUEST_VENV
$GUEST_VENV/bin/pip install pip --upgrade
$GUEST_VENV/bin/pip install -U -c /opt/upper-constraints.txt /opt/guest-agent
# Link the trove-guestagent out to /usr/local/bin where the startup scripts look
ln -s $GUEST_VENV/bin/trove-guestagent /usr/local/bin/guest-agent || true
mkdir -p /var/lib/trove /etc/trove/certs /var/log/trove
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.logrotate /etc/logrotate.d/guest-agent
case "$DIB_INIT_SYSTEM" in
upstart)
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.conf /etc/init/guest-agent.conf
;;
systemd)
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.service /usr/lib/systemd/system/guest-agent.service
;;
sysv)
install -D -g root -o root -m 0644 ${SCRIPTDIR}/guest-agent.init /etc/init.d/guest-agent.init
;;
*)
echo "Unsupported init system"
exit 1
;;
esac

18
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.conf Normal file
View File

@ -0,0 +1,18 @@
description "Start up the Trove Guest Agent"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 2 2
exec guest-agent --config-file /etc/trove/guest-agent.conf
post-start script
PID=`status guest-agent | egrep -oi '([0-9]+)$' | head -n1`
echo $PID > /var/run/guest-agent.pid
end script
post-stop script
rm -f /var/run/guest-agent.pid
end script

67
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.init Normal file
View File

@ -0,0 +1,67 @@
### BEGIN INIT INFO
# Provides: guest-agent
# Required-Start: $remote_fs $syslog $network
# Required-Stop: $remote_fs $syslog $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Runs the Trove Guest Agent processes
# Description: This script runs Trove Guest Agent processes.
# This script will start the Guest Agent services
# and kill them.
### END INIT INFO
# Using the lsb functions to perform the operations.
. /lib/lsb/init-functions
# Process name ( For display )
NAME=guest-agent
# Daemon name, where is the actual executable
DAEMON=/usr/local/bin/guest-agent
# pid file for the daemon
PIDFILE=/var/run/guest-agent.pid
# If the daemon is not there, then exit.
test -x $DAEMON || exit 5
case $1 in
start)
# Checked the PID file exists and check the actual status of process
if [ -e $PIDFILE ]; then
status_of_proc -p $PIDFILE $DAEMON "$NAME process" && status="0" || status="$?"
# If the status is SUCCESS then don't need to start again.
if [ $status = "0" ]; then
exit # Exit
fi
fi
# Start the daemon.
log_daemon_msg "Starting the process" "$NAME"
# Start the daemon with the help of start-stop-daemon
# Log the message appropriately
if start-stop-daemon --start -m --quiet --oknodo --pidfile $PIDFILE --startas $DAEMON -- --config-file /etc/octavia/guest-agent.conf ; then
log_end_msg 0
else
log_end_msg 1
fi
;;
stop)
# Stop the daemon.
if [ -e $PIDFILE ]; then
status_of_proc -p $PIDFILE $DAEMON "Stopping the $NAME process" && status="0" || status="$?"
if [ "$status" = 0 ]; then
start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
/bin/rm -rf $PIDFILE
fi
else
log_daemon_msg "$NAME process is not running"
log_end_msg 0
fi
;;
restart)
# Restart the daemon.
$0 stop && sleep 2 && $0 start
;;
*)
# For invalid arguments, print the usage message.
echo "Usage: $0 {start|stop|restart|reload|status}"
exit 2
;;
esac

14
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.logrotate Normal file
View File

@ -0,0 +1,14 @@
/var/log/guest-agent.log {
daily
rotate 10
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
# Signal name shall not have the SIG prefix in kill command
# http://pubs.opengroup.org/onlinepubs/9699919799/utilities/kill.html
kill -s USR1 $(cat /var/run/guest-agent.pid)
endscript
}

14
integration/scripts/files/elements/guest-agent/install.d/guest-agent-source-install/guest-agent.service Normal file
View File

@ -0,0 +1,14 @@
[Unit]
Description=OpenStack Trove Guest Agent
After=network.target syslog.service
Wants=syslog.service
[Service]
ExecStart=/usr/local/bin/guest-agent --config-dir=/etc/trove/conf.d
KillMode=mixed
Restart=always
ExecStartPost=/bin/sh -c "echo $MAINPID > /var/run/guest-agent.pid"
PIDFile=/var/run/guest-agent.pid
[Install]
WantedBy=multi-user.target

62
integration/scripts/files/elements/guest-agent/package-installs.yaml Normal file
View File

@ -0,0 +1,62 @@
guest-agent:
installtype: package
build-essential:
installtype: source
libffi-dev:
installtype: source
libssl-dev:
installtype: source
python-dev:
installtype: source
python-sqlalchemy:
python-lxml:
python-eventlet:
python-webob:
python-httplib2:
python-iso8601:
python-pexpect:
python-mysqldb:
python-migrate:
acl:
acpid:
apparmor:
apparmor-utils:
apt-transport-https:
at:
bash-completion:
cloud-guest-utils:
cloud-init:
cron:
curl:
dbus:
dkms:
dmeventd:
ethtool:
gawk:
ifenslave:
ifupdown:
iptables:
iputils-tracepath:
irqbalance:
isc-dhcp-client:
less:
logrotate:
lsof:
net-tools:
netbase:
netcat-openbsd:
open-vm-tools:
openssh-client:
openssh-server:
pollinate:
psmisc:
rsyslog:
screen:
socat:
tcpdump:
ubuntu-cloudimage-keyring:
ureadahead:
uuid-runtime:
vim-tiny:
vlan:

26
integration/scripts/files/elements/guest-agent/pkg-map Normal file
View File

@ -0,0 +1,26 @@
{
"family": {
"redhat": {
"guest-agent": "openstack-trove-guest-agent",
"netcat-openbsd": "nmap-ncat",
"netbase": "",
"cron": "",
"ifenslave": "",
"iputils-tracepath": "",
"cloud-guest-utils": "",
"apparmor": "",
"dmeventd": "",
"isc-dhcp-client": "",
"uuid-runtime": "",
"ubuntu-cloudimage-keyring": "",
"vim-tiny": "",
"ureadahead": "",
"apt-transport-https": "",
"pollinate": "",
"ifupdown": ""
}
},
"default": {
"guest-agent": "guest-agent"
}
}

11
integration/scripts/files/elements/guest-agent/post-install.d/11-enable-guest-agent-systemd Executable file
View File

@ -0,0 +1,11 @@
#!/bin/bash
if [ ${DIB_DEBUG_TRACE:-0} -gt 0 ]; then
set -x
fi
set -eu
set -o pipefail
if [ "$DIB_INIT_SYSTEM" == "systemd" ]; then
systemctl enable $(svc-map guest-agent)
fi

39
integration/scripts/files/elements/guest-agent/post-install.d/12-ssh-key Executable file
View File

@ -0,0 +1,39 @@
#!/bin/bash
# CONTEXT: GUEST during CONSTRUCTION as ROOT
# PURPOSE: take "staged" ssh keys (see extra-data.d/62-ssh-key) and put them in the GUEST_USERS home directory
set -e
set -o xtrace
SSH_DIR="/home/${GUEST_USERNAME}/.ssh"
TMP_HOOKS_DIR="/tmp/in_target.d"
if ! id -u ${GUEST_USERNAME} >/dev/null 2>&1; then
echo "Adding ${GUEST_USERNAME} user"
useradd -G sudo -m ${GUEST_USERNAME} -s /bin/bash
chown ${GUEST_USERNAME}:${GUEST_USERNAME} /home/${GUEST_USERNAME}
passwd ${GUEST_USERNAME} <<_EOF_
${GUEST_USERNAME}
${GUEST_USERNAME}
_EOF_
fi
if [ -f "${TMP_HOOKS_DIR}/ssh-authorized-keys" ]; then
if [ ! -d ${SSH_DIR} ]; then
# this method worked more reliable in vmware fusion over doing sudo -Hiu ${GUEST_USERNAME}
mkdir ${SSH_DIR}
chown ${GUEST_USERNAME}:${GUEST_USERNAME} ${SSH_DIR}
fi
sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/authorized_keys conv=notrunc if=${TMP_HOOKS_DIR}/ssh-authorized-keys
if [ ! -f "${SSH_DIR}/id_rsa" ]; then
sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa if=${TMP_HOOKS_DIR}/ssh-id_rsa
# perms have to be right on this file for ssh to work
sudo -Hiu ${GUEST_USERNAME} chmod 600 ${SSH_DIR}/id_rsa
sudo -Hiu ${GUEST_USERNAME} dd of=${SSH_DIR}/id_rsa.pub if=${TMP_HOOKS_DIR}/ssh-id_rsa.pub
fi
else
echo "SSH Keys were not staged by host"
exit -1
fi

11
integration/scripts/files/elements/guest-agent/post-install.d/99-clean-apt Executable file
View File

@ -0,0 +1,11 @@
#!/bin/sh
# CONTEXT: GUEST during CONSTRUCTION as ROOT
# PURPOSE: Delete contents of apt cache on guest (saves image disk space)
set -e
set -o xtrace
apt-get clean

3
integration/scripts/files/elements/guest-agent/source-repository-guest-agent Normal file
View File

@ -0,0 +1,3 @@
# This is used for source-based builds
guest-agent git /opt/guest-agent https://git.openstack.org/openstack/trove
upper-constraints file /opt/upper-constraints.txt https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt

3
integration/scripts/files/elements/guest-agent/svc-map Normal file
View File

@ -0,0 +1,3 @@
guest-agent:
default: guest-agent
redhat: trove-guest-agent

11
integration/scripts/files/elements/no-resolvconf/README.rst Normal file
View File

@ -0,0 +1,11 @@
This element clears out /etc/resolv.conf and prevents dhclient from populating
it with data from DHCP. This means that DNS resolution will not work from the
amphora. This is OK because all outbound connections from the guest will
be based using raw IP addresses.
In addition we remove dns from the nsswitch.conf hosts setting.
This has the real benefit of speeding up host boot and configutation times.
This is especially helpful when running tempest tests in a devstack environment
where DNS resolution from the guest usually doesn't work anyway. This means
that the guest never waits for DNS timeouts to occur.

19
integration/scripts/files/elements/no-resolvconf/finalise.d/99-disable-resolv-conf Executable file
View File

@ -0,0 +1,19 @@
#!/bin/bash
echo "" > /etc/resolv.conf
echo "" > /etc/resolv.conf.ORIG
if [ -d /etc/dhcp/dhclient-enter-hooks.d ]; then
# Debian/Ubuntu
echo "#!/bin/sh
make_resolv_conf() { : ; }" > /etc/dhcp/dhclient-enter-hooks.d/noresolvconf
chmod +x /etc/dhcp/dhclient-enter-hooks.d/noresolvconf
rm -f /etc/dhcp/dhclient-enter-hooks.d/resolvconf
else
# RHEL/CentOS/Fedora
echo "#!/bin/sh
make_resolv_conf() { : ; }" > /etc/dhclient-enter-hooks
chmod +x /etc/dhclient-enter-hooks
fi
if [ -e /etc/nsswitch.conf ]; then
sed -i -e "/hosts:/ s/dns//g" /etc/nsswitch.conf
fi

2
integration/scripts/files/elements/ubuntu-guest/extra-data.d/15-trove-dep
View File

@ -10,7 +10,7 @@ source $_LIB/die
BRANCH_OVERRIDE=${BRANCH_OVERRIDE:-default} BRANCH_OVERRIDE=${BRANCH_OVERRIDE:-default}
ADD_BRANCH=$(basename ${BRANCH_OVERRIDE}) ADD_BRANCH=$(basename ${BRANCH_OVERRIDE})
REQUIREMENTS_FILE=${TROVESTACK_SCRIPTS}/files/requirements/ubuntu-requirements.txt REQUIREMENTS_FILE=${TROVESTACK_SCRIPTS}/../../requirements.txt
[ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set" [ -n "$TMP_HOOKS_PATH" ] || die "Temp hook path not set"
[ -e ${REQUIREMENTS_FILE} ] || die "Requirements not found" [ -e ${REQUIREMENTS_FILE} ] || die "Requirements not found"

2
integration/scripts/files/elements/ubuntu-guest/install.d/62-ssh-key
View File

@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
# CONTEXT: GUEST during CONSTRUCTION as ROOT
# PURPOSE: take "staged" ssh keys (see extra-data.d/62-ssh-key) and put them in the GUEST_USERS home directory # PURPOSE: take "staged" ssh keys (see extra-data.d/62-ssh-key) and put them in the GUEST_USERS home directory
# In future, this should be removed and use Nova keypair to inject ssh keys.
set -e set -e
set -o xtrace set -o xtrace

4
integration/scripts/files/elements/ubuntu-mysql/pre-install.d/10-percona-apt-key
View File

@ -6,8 +6,8 @@
set -e set -e
set -o xtrace set -o xtrace
[ -n "${GUEST_USERNAME}" ] || die "GUEST_USERNAME needs to be set to the user for the guest image" GUEST_USERNAME=${GUEST_USERNAME:-"ubuntu"}
[ -n "${RELEASE}" ] || die "RELEASE must be set to a valid Ubuntu release (e.g. trusty)" RELEASE=${DIB_RELEASE:-"xenial"}
# Add Percona GPG key # Add Percona GPG key
mkdir -p /home/${GUEST_USERNAME}/.gnupg mkdir -p /home/${GUEST_USERNAME}/.gnupg

246
integration/scripts/functions_qemu
View File

@ -3,147 +3,118 @@
# Additional functions that would mostly just pertain to a Ubuntu + Qemu setup # Additional functions that would mostly just pertain to a Ubuntu + Qemu setup
# #
# tmpfs dedicates half the available RAM to an internal cache to speed up
# image building. This can lead to failures when the data doesn't fit into
# the cache, for example when compiling large programs. This flag allows
# tmpfs to be turned off by the caller by setting USE_TMPF=false
USE_TMPFS=${USE_TMPFS:-true}
# Use the latest distro provided hardware enablement stack which is required
# by some platforms (ppc64el) for nested virtualization. Trovestack runs a
# VM (db guest image) in a VM (OpenStack+Trove)
DIB_USE_HWE_KERNEL=${DIB_USE_HWE_KERNEL:-true}
function build_vm() { function build_vm() {
exclaim "Actually building the image, this can take up to 15 minutes" exclaim "Actually building the image, this can take up to 15 minutes"
# set variables here and ensure they are not changed during the duration of this script datastore_type=$1
readonly HOMEDIR=$1 guest_os=$2
readonly HOST_USERNAME=$2 guest_release=$3
GUEST_USERNAME=${GUEST_USERNAME:-$2} dev_mode=$4
HOST_SCP_USERNAME=${HOST_SCP_USERNAME:-$2} guest_username=$5
VM=$3 image_output=$6
DISTRO=$4
SERVICE_TYPE=$5
readonly SSH_DIR=${KEY_DIR:-${HOMEDIR}/.ssh} elementes="base vm"
trove_elements_path=${PATH_TROVE}/integration/scripts/files/elements
GUEST_IMAGETYPE=${GUEST_IMAGETYPE:-"qcow2"}
GUEST_IMAGESIZE=${GUEST_IMAGESIZE:-3}
GUEST_CACHEDIR=${GUEST_CACHEDIR:-"$HOME/.cache/image-create"}
GUEST_WORKING_DIR=${GUEST_WORKING_DIR:-"$HOME/images"}
export SSH_DIR=${SSH_DIR:-"$HOME/.ssh"}
export GUEST_USERNAME=${guest_username}
export HOST_SCP_USERNAME=$(whoami)
manage_ssh_keys manage_ssh_keys
ARCH=amd64 # In dev mode, the trove guest agent needs to download trove code from
if [ $DISTRO == 'ubuntu' ]; then # trove-taskmanager host.
export DIB_RELEASE=$RELEASE if [[ "${dev_mode}" == "true" ]]; then
export DIB_CLOUD_IMAGES=cloud-images.ubuntu.com/$DIB_RELEASE/current host_ip=$(ip route get 8.8.8.8 | head -1 | awk '{print $7}')
export DIB_USE_HWE_KERNEL export CONTROLLER_IP=${CONTROLLER_IP:-${host_ip}}
ARCH=$(dpkg --print-architecture) export PATH_TROVE=${PATH_TROVE}
elif [ $DISTRO == 'fedora' ]; then export ESCAPED_PATH_TROVE=$(echo ${PATH_TROVE} | sed 's/\//\\\//g')
EXTRA_ELEMENTS=selinux-permissive export GUEST_LOGDIR=${GUEST_LOGDIR:-"/var/log/trove/"}
export ESCAPED_GUEST_LOGDIR=$(echo ${GUEST_LOGDIR} | sed 's/\//\\\//g')
export TROVESTACK_SCRIPTS=${TROVESTACK_SCRIPTS}
export HOST_USERNAME=${HOST_SCP_USERNAME}
fi fi
case "$USE_TMPFS" in # For system-wide installs, DIB will automatically find the elements, so we only check local path
false|FALSE|False|no|NO|n|N) if [ "${DIB_LOCAL_ELEMENTS_PATH}" ]; then
TMPFS_ARGS='--no-tmpfs' export ELEMENTS_PATH=${trove_elements_path}:${DIB_LOCAL_ELEMENTS_PATH}
;; else
*) export ELEMENTS_PATH=${trove_elements_path}
TMPFS_ARGS='' fi
;;
esac
export HOST_USERNAME export DIB_RELEASE=${guest_release}
export HOST_SCP_USERNAME
export GUEST_USERNAME
export CONTROLLER_IP
export TROVESTACK_SCRIPTS
export SERVICE_TYPE
export PATH_TROVE
export ESCAPED_PATH_TROVE
export SSH_DIR
export GUEST_LOGDIR
export ESCAPED_GUEST_LOGDIR
export ELEMENTS_PATH=$TROVESTACK_SCRIPTS/files/elements
export ELEMENTS_PATH+=:$PATH_DISKIMAGEBUILDER_ELEMENTS
export ELEMENTS_PATH+=:$PATH_TRIPLEO_ELEMENTS/elements
export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive" export DIB_CLOUD_INIT_DATASOURCES="ConfigDrive"
export DATASTORE_PKG_LOCATION
export BRANCH_OVERRIDE if [ "${GUEST_WORKING_DIR}" ]; then
export DIB_APT_CONF_DIR=/etc/apt/apt.conf.d mkdir -p ${GUEST_WORKING_DIR}
export DIB_CLOUD_INIT_ETC_HOSTS=true TEMP=$(mktemp -d ${GUEST_WORKING_DIR}/diskimage-create.XXXXXX)
local QEMU_IMG_OPTIONS="--qemu-img-options compat=1.1" else
disk-image-create ${TMPFS_ARGS} -a ${ARCH} -o "${VM}" \ TEMP=$(mktemp -d diskimage-create.XXXXXX)
-x ${QEMU_IMG_OPTIONS} ${DISTRO} ${EXTRA_ELEMENTS} vm \ fi
cloud-init-datasources ${DISTRO}-${RELEASE}-guest ${DISTRO}-${RELEASE}-${SERVICE_TYPE} pushd $TEMP > /dev/null
elementes="$elementes ${guest_os}"
if [[ "${dev_mode}" != "true" ]]; then
elementes="$elementes pip-and-virtualenv"
elementes="$elementes pip-cache"
elementes="$elementes no-resolvconf"
elementes="$elementes guest-agent"
else
elementes="$elementes ${guest_os}-guest"
elementes="$elementes ${guest_os}-${guest_release}-guest"
fi
elementes="$elementes ${guest_os}-${datastore_type}"
elementes="$elementes ${guest_os}-${guest_release}-${datastore_type}"
# Build the image
disk-image-create -x \
-a amd64 \
-o ${image_output} \
-t ${GUEST_IMAGETYPE} \
--image-size ${GUEST_IMAGESIZE} \
--image-cache ${GUEST_CACHEDIR} \
$elementes
# out of $TEMP
popd > /dev/null
rm -rf $TEMP
exclaim "Image ${image_output}.${GUEST_IMAGETYPE} was built successfully."
} }
function build_guest_image() { function build_guest_image() {
exclaim "Building an image for use with development and integration tests." exclaim "Building an image for Trove guest agent."
if [ -z "$1" ] datastore_type=$1
guest_os=${2:-"ubuntu"}
guest_release=${3:-"xenial"}
dev_mode=${4:-"true"}
guest_username=${5:-"ubuntu"}
if [ -z "$datastore_type" ]
then then
echo "You must pass an image type to build, like mysql" echo "You must pass an image type to build, like mysql"
exit 1 exit 1
fi fi
SERVICE_TYPE=$1
VALID_SERVICES='mysql percona mariadb redis cassandra couchbase mongodb postgresql couchdb vertica db2 pxc' VALID_SERVICES='mysql percona mariadb redis cassandra couchbase mongodb postgresql couchdb vertica db2 pxc'
if ! [[ " $VALID_SERVICES " =~ " $SERVICE_TYPE " ]]; then if ! [[ " $VALID_SERVICES " =~ " $datastore_type " ]]; then
exclaim "You did not pass in a valid image type. Valid types are:" $VALID_SERVICES exclaim "You did not pass in a valid image type. Valid types are:" $VALID_SERVICES
exit 1 exit 1
fi fi
GUEST_LOGDIR=$(iniget $PATH_TROVE/etc/trove/trove-guestagent.conf.sample DEFAULT log_dir) image_name=${guest_os}_${datastore_type}
GUEST_LOGFILE=$(iniget $PATH_TROVE/etc/trove/trove-guestagent.conf.sample DEFAULT log_file) image_folder=$HOME/images
mkdir -p $image_folder
image_path=${image_folder}/${image_name}
if [ -z $GUEST_LOGDIR ] || [ -z $GUEST_LOGFILE ] # Always rebuild the image.
then rm -rf ${image_folder}/*
exclaim "error: log_dir and log_file are required in: " $PATH_TROVE/etc/trove/trove-guestagent.conf.sample
exit 1
fi
ESCAPED_GUEST_LOGDIR=`echo $GUEST_LOGDIR | sed 's/\//\\\\\//g'` build_vm ${datastore_type} ${guest_os} ${guest_release} ${dev_mode} ${guest_username} ${image_path}
USERNAME=`whoami`
# To change the distro, edit the trovestack.rc file
readonly IMAGENAME=${DISTRO}_${SERVICE_TYPE}
readonly VM_PATH=$USERHOME/images/${IMAGENAME}
readonly VM_PATH_NAME=${VM_PATH}/${IMAGENAME}
mkdir -p $VM_PATH
# If the path does not exist, build it, otherwise just upload it
# (unless we're explicitly told to rebuild it)
REBUILD_IMAGE=$(echo "${REBUILD_IMAGE}" | tr '[:upper:]' '[:lower:]')
if [ "${REBUILD_IMAGE}" = "true" ] || [ ! -d $VM_PATH ] || [ `ls -1 $VM_PATH | wc -l` -eq '0' ]
then
if [ "${REBUILD_IMAGE}" = "true" ]
then
exclaim "Rebuilding image"
rm -rf "${VM_PATH}"
fi
build_vm $USERHOME $USERNAME $VM_PATH_NAME $DISTRO $SERVICE_TYPE
touch -c "${VM_PATH}"
else
exclaim "Found image in $VM_PATH - using the qcow2 image found here..."
ELEMENTS_DIR="files/elements/${DISTRO}-${SERVICE_TYPE}"
ELEMENTS_DIR_GUEST="files/elements/${DISTRO}-guest"
# Print out a warning on all the elements files that are newer than the image.
# Directories are not excluded as that is the only way to determine if a file
# has been removed.
# The rebuild is not automatically triggered as there are valid reasons for a
# new file to be present (rollback of change, inadvertent .log files present,
# feature half implemented, etc.).
IMAGE_OLD=
while IFS= read -r -d '' ELEMENT_FILE
do
if [ "${ELEMENT_FILE}" -nt "${VM_PATH}" ]
then
IMAGE_OLD=true
exclaim "${COLOR_RED}WARNING: Element file '${ELEMENT_FILE}' is newer than cached image${COLOR_NONE}"
fi
done < <(find "${ELEMENTS_DIR}" "${ELEMENTS_DIR_GUEST}" -depth -print0)
if [ "${IMAGE_OLD}" = "true" ]
then
exclaim "${COLOR_RED}Use ${COLOR_NONE}REBUILD_IMAGE=True${COLOR_RED} to rebuild image${COLOR_NONE}"
fi
fi
} }
function clean_instances() { function clean_instances() {
@ -151,42 +122,23 @@ function clean_instances() {
for i in $LIST; do sudo virsh destroy $i; done for i in $LIST; do sudo virsh destroy $i; done
} }
# Trove doesn't support to specify keypair when creating the db instance, the
# ssh keys are injected when the image is built. This could be removed when
# we support keypair in the future.
function manage_ssh_keys() { function manage_ssh_keys() {
if [ -e ${SSH_DIR} ]; then if [ -d ${SSH_DIR} ]; then
echo "${SSH_DIR} already exists" echo "${SSH_DIR} already exists"
else else
echo "Creating ${SSH_DIR} for ${HOST_USERNAME}" echo "Creating ${SSH_DIR} for ${HOST_SCP_USERNAME}"
sudo -Hiu ${HOST_USERNAME} mkdir -m go-w -p ${SSH_DIR} sudo -Hiu ${HOST_SCP_USERNAME} mkdir -m go-w -p ${SSH_DIR}
fi fi
if [ ! -f ${SSH_DIR}/id_rsa.pub ]; then if [ ! -f ${SSH_DIR}/id_rsa.pub ]; then
sudo ${PKG_MGR} ${PKG_GET_ARGS} install expect /usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q -N ""
generate_empty_passphrase_ssh_key ${HOST_USERNAME}
fi fi
add_host_key_to_authorizedkeys cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys
} sort ${SSH_DIR}/authorized_keys | uniq > ${SSH_DIR}/authorized_keys.uniq
mv ${SSH_DIR}/authorized_keys.uniq ${SSH_DIR}/authorized_keys
function generate_empty_passphrase_ssh_key () { chmod 600 ${SSH_DIR}/authorized_keys
echo "generating a empty passphrase DEV ONLY rsa key"
expect -c "
spawn sudo -Hiu ${HOST_USERNAME} /usr/bin/ssh-keygen -f ${SSH_DIR}/id_rsa -q
expect \"empty for no passphrase\"
send \n
expect assphrase
send \n
expect eof"
}
function add_host_key_to_authorizedkeys () {
# test to see if the host key is already in its own authorized_keys file - if not then add it. This is then later copied
# to the guest image
is_in_keyfile=`cat ${SSH_DIR}/id_rsa.pub | grep -f - ${SSH_DIR}/authorized_keys | wc -l`
if [ $is_in_keyfile == 0 ]; then
echo "Adding keyfile to authorized_keys, it does not yet exist"
cat ${SSH_DIR}/id_rsa.pub >> ${SSH_DIR}/authorized_keys
chmod 600 ${SSH_DIR}/authorized_keys
else
echo "Keyfile already exists in authorized_keys - skipping"
fi
} }

56
integration/scripts/trovestack
View File

@ -97,10 +97,6 @@ KEYSTONE_AUTH_PORT=${KEYSTONE_AUTH_PORT:-35357}
GLANCE_HOSTPORT=${GLANCE_HOSTPORT:-$SERVICE_HOST:9292} GLANCE_HOSTPORT=${GLANCE_HOSTPORT:-$SERVICE_HOST:9292}
GLANCE_SERVICE_PROTOCOL=${GLANCE_SERVICE_PROTOCOL:-http} GLANCE_SERVICE_PROTOCOL=${GLANCE_SERVICE_PROTOCOL:-http}
# The following depends on whether neutron is used or nova-network
# neutron uses a bridge, nova-network does not
[[ $ENABLE_NEUTRON = true ]] && CONTROLLER_IP=$BRIDGE_IP || CONTROLLER_IP=$NETWORK_GATEWAY
# PATH_TROVE more than likely has file separators, which sed does not like # PATH_TROVE more than likely has file separators, which sed does not like
# This will escape them # This will escape them
ESCAPED_PATH_TROVE=$(echo $PATH_TROVE | sed 's/\//\\\//g') ESCAPED_PATH_TROVE=$(echo $PATH_TROVE | sed 's/\//\\\//g')
@ -828,51 +824,52 @@ function cmd_test_init() {
function cmd_build_image() { function cmd_build_image() {
local IMAGE_DATASTORE_TYPE=${1:-'mysql'} local IMAGE_DATASTORE_TYPE=${1:-'mysql'}
local ESCAPED_PATH_TROVE=${2:-'\/opt\/stack\/trove'} local IMAGE_GUEST_OS=${2:-'ubuntu'}
local HOST_SCP_USERNAME=${3:-'ubuntu'} local IMAGE_GUEST_RELEASE=${3:-'xenial'}
local GUEST_USERNAME=${4:-'ubuntu'} local DEV_MODE=${4:-'true'}
exclaim "Ensuring we have all packages needed to build image." exclaim "Ensuring we have all packages needed to build image."
sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS update sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS update
sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS install qemu sudo $HTTP_PROXY $PKG_MGR $PKG_GET_ARGS install qemu git kpartx debootstrap
sudo -H $HTTP_PROXY pip install --upgrade pip dib-utils sudo -H $HTTP_PROXY pip install diskimage-builder
pkg_install python-yaml
install_devstack_code install_devstack_code
cmd_clone_projects do_not_force_update $TROVESTACK_SCRIPTS/image-projects-list exclaim "Use diskimage-builder to actually build the Trove Guest Agent Image."
build_guest_image $IMAGE_DATASTORE_TYPE $IMAGE_GUEST_OS $IMAGE_GUEST_RELEASE $DEV_MODE
exclaim "Use tripleo-diskimagebuilder to actually build the Trove Guest Agent Image."
build_guest_image $IMAGE_DATASTORE_TYPE
} }
function cmd_build_and_upload_image() { function cmd_build_and_upload_image() {
local DATASTORE_TYPE=$1 local datastore_type=$1
local RESTART_TROVE=${2:-$(get_bool RESTART_TROVE "true")} local restart_trove=${2:-$(get_bool RESTART_TROVE "true")}
local guest_os=${3:-"ubuntu"}
local guest_release=${4:-"xenial"}
local dev_mode=${5:-"true"}
local guest_username=${6:-"ubuntu"}
if [ -z "${DATASTORE_TYPE}" ]; then if [ -z "${datastore_type}" ]; then
exclaim "${COLOR_RED}Datastore argument was not specified.${COLOR_NONE}" exclaim "${COLOR_RED}Datastore argument was not specified.${COLOR_NONE}"
exit 1 exit 1
fi fi
glance_imageid=$(openstack $CLOUD_ADMIN_ARG image list | grep "$datastore_type" | get_field 1)
echo "IMAGEID: $glance_imageid"
if [[ -z $glance_imageid ]]; then
build_guest_image ${datastore_type} ${guest_os} ${guest_release} ${dev_mode} ${guest_username}
GLANCE_IMAGEID=$(openstack $CLOUD_ADMIN_ARG image list | grep "$DATASTORE_TYPE" | get_field 1) image_folder=$HOME/images
echo "IMAGEID: $GLANCE_IMAGEID" qcow_image=`find $image_folder -name '*.qcow2'`
if [[ -z $GLANCE_IMAGEID ]]; then image_url="file://$qcow_image"
build_guest_image "${DATASTORE_TYPE}" glance_imageid=`get_glance_id upload_image $image_url`
QCOW_IMAGE=`find $VM_PATH -name '*.qcow2'` [[ -z "$glance_imageid" ]] && echo "Glance upload failed!" && exit 1
IMAGE_URL="file://$QCOW_IMAGE" echo "IMAGE ID: $glance_imageid"
GLANCE_IMAGEID=`get_glance_id upload_image $IMAGE_URL`
[[ -z "$GLANCE_IMAGEID" ]] && echo "Glance upload failed!" && exit 1
echo "IMAGE ID: $GLANCE_IMAGEID"
if [[ -f /etc/trove/cloudinit/mysql.cloudinit ]]; then if [[ -f /etc/trove/cloudinit/mysql.cloudinit ]]; then
sudo cp /etc/trove/cloudinit/mysql.cloudinit /etc/trove/cloudinit/${DATASTORE_TYPE}.cloudinit sudo cp /etc/trove/cloudinit/mysql.cloudinit /etc/trove/cloudinit/${datastore_type}.cloudinit
fi fi
fi fi
exclaim "Updating Datastores" exclaim "Updating Datastores"
cmd_set_datastore "${GLANCE_IMAGEID}" "${DATASTORE_TYPE}" "${RESTART_TROVE}" cmd_set_datastore "${glance_imageid}" "${datastore_type}" "${restart_trove}"
} }
@ -1158,7 +1155,6 @@ function cmd_run_ci() {
set -e set -e
cmd_install cmd_install
cmd_test_init "${DATASTORE_TYPE}" cmd_test_init "${DATASTORE_TYPE}"
# The arg will be the image type
cmd_build_and_upload_image "${DATASTORE_TYPE}" "${RESTART_TROVE}" cmd_build_and_upload_image "${DATASTORE_TYPE}" "${RESTART_TROVE}"
# Test in fake mode. # Test in fake mode.