x/browbeat
Code Issues Proposed changes

Adding CI scripts

Browse Source 
This commit adds basic CI scripts and CI integration
into Browbeat, this script is sourced from the
Triple O quickstart full-deploy.sh CI script

This script currently works with Rally and Shaker
once my other commit is in, perfkit support depends
on a bugfix in Centos or upstream DNF support in perfkit

Change-Id: I022f05e4408fe5466c3b1032cbbca7adc96d0527
This commit is contained in:
jkilpatr 2016-06-22 09:23:45 -04:00
parent b3d818a22e
commit df03cb36a9
5 changed files with 630 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
README.rst
View File

@ -32,6 +32,9 @@ Table of Contents
- `Running PerfKitBenchmarker <#running-perfkitbenchmarker>`__
- `Contributing <#contributing>`__
- `Adding Functionality <#adding-functionality>`__
- `Links and Resources <#resources>`__
Browbeat
@ -368,6 +371,17 @@ Finally, push the patch for review using,
$ git review
Adding functionality
--------------------
If you are adding new functionality to Browbeat please add testing for that functionality in.
::
$ ci-scripts/install-and-check.sh
See the README.rst in the ci-scripts folder for more details on the structure of the script and how to add additional tests.
Resources
=========

81
ci-scripts/README.rst Normal file
View File

@ -0,0 +1,81 @@
Table of Contents
=================
- `CI Structure <#ci-structure>`__
- `Script Documentation <#script-documentation>`__
- `Install and Check <#install-and-check>`__
- `Invoking Locally <#invoking-locally>`__
CI Structure
============
For an example Jenkins configuration see `this job <https://ci.centos.org/view/rdo/view/POC/job/poc-browbeat-tripleo-quickstart-mitaka-delorean-full-deploy-minimal/>`_
If you would like to make your own CI job add your CI script to this directory and invoke it as minimally as possible on the Jenkins end, this will help us keep script changes in the repository and better test them before merging.
Script Documentation
====================
Install and Check
-----------------
Currently the main CI script that is run against every commit submittied to the Openstack Gerrit. For each test a fresh Openstack instance is deployed using `TripleO Quickstart <https:github.com/openstack/tripleo-quickstart>`_, Browbeat is then installed. Both of these happen regardless of what was included in the commit. Workload tests are run only if a file diff between the commit and Browbeat master contains the workload name. Success is defined as all processes in the script exiting with exit code 0, note Browbeat will return zero if a test fails its SLA or otherwise fails in a manner that's not total.
To add an additional workload to the script add the workload name to the tools loop near the bottom of the file.
::
for tool in rally perfkit shaker <tool name>; do
Then add configuration details that run all functions of the added task or plugin to the browbeat-ci.yaml file in ci-scripts/config.
You can view the output of this job `here <https://ci.centos.org/view/rdo/view/POC/job/poc-browbeat-tripleo-quickstart-mitaka-delorean-full-deploy-minimal/>`_
Invoking Locally
~~~~~~~~~~~~~~~~
To run tripleo/install-and-check.sh using your local machine as the driver for a TripleO Quickstart / Browbeat deployment create an empty directory to use as your workspace and point virthost at a machine running CentOS 7+ or RHEL 7+ with at least 32gb of ram.
::
$ export WORKSPACE=<your empty directory>
$ export VIRTHOST=<deployment machine hostname>
Navigate to the workspace directory
::
$ cd $WORKSPACE
Clone the required repositories
::
$ git clone https://github.com/openstack/browbeat
$ git clone https://github.com/openstack/tripleo-quickstart/
$ git clone https://github.com/redhat-openstack/ansible-role-tripleo-inventory
Install the Ansible roles from Github into the virtual environment, as well as a few Python packages
::
$ virtualenv --no-site-packages $WORKSPACE
$ source $WORKSPACE/bin/activate
$ cd $WORKSPACE/ansible-role-tripleo-inventory/
$ python setup.py install
$ cd $WORKSPACE/tripleo-quickstart
$ python setup.py install
$ pip install --upgrade ansible netaddr
Install the package dependencies, if you're nervous about using root just look inside of quickstart.sh, these are very generic and you might already have all of them installed.
::
$ sudo bash $WORKSPACE/tripleo-quickstart/quickstart.sh --install-deps
Finally invoke the script and settle in, this command will take about two hours to complete and will place all the relevant ssh credentials and other information to access your instance once the run is complete in the workspace directory.
::
$ bash $WORKSPACE/browbeat/ci-scripts/tripleo/install-and-check.sh mitaka delorean minimal periodic

241
ci-scripts/config/browbeat-ci.yaml Normal file
View File

@ -0,0 +1,241 @@
# Tests to be compleated for the install-and-check.sh script minimal and short workloads are performed
# to confirm functionality.
browbeat:
results : results/
rerun: 1
cloud_name: openstack
elasticsearch:
enabled: false
host: 1.1.1.1
port: 9200
metadata_files:
- name: hardware-metadata
file: metadata/hardware-metadata.json
- name: environment-metadata
file: metadata/environment-metadata.json
- name: software-metadata
file: metadata/software-metadata.json
ansible:
hosts: ansible/hosts
adjust:
keystone_token: ansible/browbeat/adjustment-keystone-token.yml
neutron_l3: ansible/browbeat/adjustment-l3.yml
nova_db: ansible/browbeat/adjustment-db.yml
workers: ansible/browbeat/adjustment-workers.yml
grafana_snapshot: ansible/browbeat/snapshot-general-performance-dashboard.yml
connmon:
enabled: false
sudo: true
grafana:
enabled: true
grafana_ip: 1.1.1.1
grafana_port: 3000
dashboards:
- openstack-general-system-performance
snapshot:
enabled: false
snapshot_compute: false
perfkit:
enabled: true
sleep_before: 0
sleep_after: 0
venv: /home/stack/perfkit-venv/bin/activate
default:
image: centos7
machine_type: m1.small
os_type: rhel
openstack_image_username: centos
openstack_floating_ip_pool: browbeat_public
openstack_network: browbeat_private
benchmarks:
- name: fio-centos-m1-small
enabled: false
benchmarks: fio
data_disk_size: 4
rally:
enabled: true
sleep_before: 5
sleep_after: 5
venv: /home/stack/rally-venv/bin/activate
plugins:
- netcreate-boot: rally/rally-plugins/netcreate-boot
- netcreate-boot-ping: rally/rally-plugins/netcreate-boot-ping
- subnet-router-create: rally/rally-plugins/subnet-router-create
benchmarks:
- name: authenticate
enabled: true
concurrency:
- 8
- 16
times: 50
scenarios:
- name: authentic-keystone
enabled: true
file: rally/authenticate/keystone-cc.yml
sla_max_avg_duration: 6
sla_max_seconds: 30
sla_max_failure: 0
- name: cinder
enabled: true
concurrency:
- 1
- 2
times: 4
scenarios:
- name: create-attach-volume-centos
enabled: true
file: rally/cinder/cinder-create-and-attach-volume-cc.yml
sla_max_seconds: 30
sla_max_failure: 0
- name: keystonebasic
enabled: false
concurrency:
- 8
times: 50
scenarios:
- name: create-and-list-tenants
enabled: true
file: rally/keystonebasic/create_and_list_tenants-cc.yml
sla_max_seconds: 30
sla_max_failure: 0
times: 50
- name: neutron
enabled: true
concurrency:
- 2
- 4
times: 10
scenarios:
- name: create-list-network
enabled: true
file: rally/neutron/neutron-create-list-network-cc.yml
sla_max_seconds: 30
sla_max_failure: 0
- name: nova
enabled: true
concurrency:
- 1
- 2
times: 16
scenarios:
- name: boot-snapshot-delete
enabled: true
file: rally/nova/nova-boot-snapshot-cc.yml
image_name: cirros
flavor_name: m1.tiny
- name: plugins
enabled: true
concurrency:
- 8
- 16
times: 16
scenarios:
- name: netcreate-boot
enabled: true
image_name: cirros
flavor_name: m1.tiny
file: rally/rally-plugins/netcreate-boot/netcreate_boot.yml
sla_max_avg_duration: 12
sla_max_seconds: 30
sla_max_failure: 0
#shaker scenarios require atleast 2 compute nodes
shaker:
enabled: true
server: localhost
port: 5555
flavor: m1.small
join_timeout: 600
sleep_before: 5
sleep_after: 5
venv: /home/stack/shaker-venv
dns_nameserver: 192.168.23.1
shaker_region: regionOne
scenarios:
- name: l2-4-1
enabled: true
density: 4
compute: 1
progression: linear
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l2.yaml
- name: l2-8-1
enabled: true
density: 8
compute: 1
progression: linear
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l2.yaml
- name: l2-4-2
enabled: true
density: 4
compute: 2
progression: linear
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l2.yaml
- name: l2-4-8
enabled: true
density: 8
compute: 2
progression: linear
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l2.yaml
- name: l3-north-south-4-1
enabled: true
placement: single_room
density: 4
compute: 1
progression: null
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_north_south.yaml
- name: l3-north-south-8-1
enabled: false
placement: single_room
density: 8
compute: 1
progression: null
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_north_south.yaml
- name: l3-north-south-4-2
enabled: true
placement: single_room
density: 4
compute: 2
progression: null
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_north_south.yaml
- name: l3-north-south-8-2
enabled: true
placement: single_room
density: 8
compute: 2
progression: null
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_north_south.yaml
- name: l3-east-west-4-1
enabled: true
density: 4
compute: 1
placement: single_room
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_east_west.yaml
- name: l3-east-west-8-1
enabled: true
density: 8
compute: 1
placement: single_room
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_east_west.yaml
- name: l3-east-west-4-2
enabled: true
density: 4
compute: 2
placement: single_room
time: 60
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_east_west.yaml
- name: l3-east-west-8-2
enabled: true
density: 8
compute: 2
time: 60
placement: single_room
file: lib/python2.7/site-packages/shaker/scenarios/openstack/dense_l3_east_west.yaml

167
ci-scripts/config/tripleo/install-and-check/all.yml Normal file
View File

@ -0,0 +1,167 @@
---
# OpenStack Installer
tripleo: true
browbeat_path: /home/stack/browbeat
home_dir: /home/stack
supported_distro: ((ansible_distribution == "CentOS" && ansible_distribution_major_version >= "7") or
(ansible_distribution == "RedHat" && ansible_distribution_major_version >= "7"))
# Login user for the remote hosts
host_remote_user: heat-admin
# Login user for the local/jump machine
local_remote_user: stack
# The Overcloud RC file
overcloudrc: /home/stack/overcloudrc
# The default Browbeat venv
browbeat_venv: /home/stack/browbeat-venv
# The default Rally venv
rally_venv: /home/stack/rally-venv
# The default Shaker venv
shaker_venv: /home/stack/shaker-venv
# The default PerfKit venv:
perfkit_venv: /home/stack/perfkit-venv
# Guest images for the Overcloud
images:
centos7:
url: http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2
cirros:
url: http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img
# DNS Server to add
dns_server: 192.168.23.1
# epel7 rpm for collectd packages
epel7_rpm: https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# Host where connmond will be running
connmon_host: 192.0.2.1
########################################
# Collectd Configuration
########################################
# Interval in seconds
collectd_interval: 10
# Run collectd on specific openstack nodes:
collectd_undercloud: true
collectd_controller: true
collectd_ceph: true
collectd_compute: false
########################################
# Docker related
# (use these if deploying graphite/carbon/grafana as containers)
########################################
persistent_carbon_data_path: /data/carbon/whisper
persistent_grafana_data_path: /data/grafana
docker_carbon_cache_port: 2003
docker_graphite_port: 8888
docker_grafana_port: 3000
carbon_cache_docker_image: kambiz/carbon-cache:0.9.15
graphite_web_docker_image: kambiz/graphite-web:0.9.15
grafana_docker_image: grafana/grafana:2.6.0
########################################
# Graphite Configuration
########################################
# Graphite Server ip address (Collectd -> Graphite server)
# you must fill out graphite_host prior to playbook execution
graphite_host:
graphite_port: 80
# Graphite prefix / Cloud name used both with graphite and grafana dashboards
graphite_prefix: openstack
# Graphite username and password for login on the dashboard
graphite_username: root
graphite_password: calvin
########################################
# Grafana Dashboarding Configuration
########################################
# Grafana Server IP Address/Port (Can be hosted on the Graphite server)
# you must fill out grafana_host prior to playbook execution
grafana_host:
grafana_port: 3000
grafana_username: admin
grafana_password: admin
# Batch number of hosts per row for all-{cpu, memory, disk, network} openstack dashboards
dashboards_batch: 20
# For use with all-{cpu, memory, disk, network} openstack dashboards, uses the graphite prefix to create dashboards for specific openstack cloud
dashboard_cloud_name: "{{graphite_prefix}}"
########################################
# Shaker Configuration
########################################
# Port for Shaker (5555 should suffice)
shaker_port: 5555
# Should choose m1.small or larger
shaker_flavor: m1.small
# Shaker centos image builder template, change this back to centos once outside of nested virt (jkilpatr)
shaker_centos: "{{shaker_venv}}/lib/python2.7/site-packages/shaker/resources/image_builder_templates/debian.yaml"
shaker_region: regionOne
#######################################
# Connman Configuration
#######################################
# Port for Connman
connmon_port: 5800
########################################
# Browbeat Network Configuration
########################################
# Public network that perfkit and shaker utilize
browbeat_pub_net_name: browbeat_public
browbeat_pub_subnet: 192.0.2.0/24
browbeat_pub_pool_start: 192.0.2.100
browbeat_pub_pool_end: 192.0.2.200
browbeat_pub_pool_gw: 192.0.2.1
# Private subnet
browbeat_pri_net_name: browbeat_private
browbeat_pri_subnet: 172.16.10.0/24
browbeat_pri_pool_start: 172.16.10.2
browbeat_pri_pool_end: 172.16.10.100
browbeat_pri_pool_gw: 172.16.10.1
browbeat_pri_pool_dns: 192.168.23.1
browbeat_router_name: browbeat_router
########################################
# ELK Server Variables
########################################
### nginx ###
# add nonstandard port here for undercloud usage
# usage: port nginx listens to reverse-proxy Kibana
# e.g. 8888
nginx_kibana_port: 80
#
# usage: port filebeat client grabs the client SSL certificate
# e.g. 9999
elk_server_ssl_cert_port: 8080
#
### logging backend ###
# you can pick between logstash or fluentd
# if left empty logstash will be used
### accepted options ###
# logging_backend:
# logging_backend: logstash
# logging_backend: fluentd
logging_backend:
#
### logstash options ###
logstash_syslog_port: 5044
### fluentd options ###
fluentd_syslog_port: 42185
fluentd_http_port: 9919
fluentd_debug_port: 24230
### install curator tool ###
# curator is the recommended tool for managing elasticsearch indexes
# https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html
# default is no (set to blank) or false
# set the below variable to 'true' to activate
install_curator_tool: false

127
ci-scripts/tripleo/install-and-check.sh Executable file
View File

@ -0,0 +1,127 @@
#!/bin/bash
# CI test based on Triple O's full deploy CI test, will deploy
# a Triple O instance, install Browbeat and then perform various checks
# Usage:
# export VIRTHOST=<virthost>
# export WORKSPACE=<workspace>
# install-and-test.sh <release> <build_system> <config> <job_type>
#
# The workspace should be a blank folder with the browbeat and TripleO-Quickstart
# repositories cloned into that folder with their default folder names. The env vars
# must be exported outside of install-and-test.sh not inside, otherwise they don't get
# passed to some components of TripleO. You also must have your ssh key copied to root
# on virthost for ansible to work.
#
set -eux
RELEASE=$1
BUILD_SYS=$2
CONFIG=$3
JOB_TYPE=$4
export SSH_CMD="ssh -tt -F $WORKSPACE/ssh.config.ansible undercloud"
export SCP_CMD="scp -F $WORKSPACE/ssh.config.ansible"
export DIFF_CMD="git diff origin/master --name-only"
if [ "$JOB_TYPE" = "gate" ] || [ "$JOB_TYPE" = "periodic" ]; then
LOCATION='stable'
elif [ "$JOB_TYPE" = "promote" ]; then
LOCATION='testing'
else
echo "Job type must be one of gate, periodic, or promote"
exit 1
fi
# (trown) This is so that we ensure separate ssh sockets for
# concurrent jobs. Without this, two jobs running in parallel
# would try to use the same undercloud-stack socket.
socketdir=$(mktemp -d /tmp/sockXXXXXX)
export ANSIBLE_SSH_CONTROL_PATH=$socketdir/%%h-%%r
deployCloud()
{
pushd $WORKSPACE/tripleo-quickstart
bash quickstart.sh \
--tags all \
-e undercloud_image_url="http://artifacts.ci.centos.org/artifacts/rdo/images/$RELEASE/$BUILD_SYS/$LOCATION/undercloud.qcow2" \
--config $WORKSPACE/config/general_config/$CONFIG.yml \
--working-dir $WORKSPACE/ \
--teardown all \
--no-clone \
$VIRTHOST $RELEASE
popd
}
deployBrowbeat()
{
#The vitrualenv script is delicate to enviromental changes
#this restores the default terminal mode temporarily (jkilpatr)
set +eux
. $WORKSPACE/bin/activate
set -eux
pip install ansible
pushd $WORKSPACE/browbeat/ansible
cp $WORKSPACE/browbeat/ci-scripts/config/tripleo/install-and-check/all.yml $WORKSPACE/browbeat/ansible/install/group_vars/all.yml
#Clone repo from ci (with current changes) to $VIRTHOST so that we test the current commit
$SCP_CMD -r $WORKSPACE/browbeat undercloud:
ansible-playbook --ssh-common-args="-F $WORKSPACE/ssh.config.ansible" -i $WORKSPACE/hosts install/browbeat.yml
ansible-playbook --ssh-common-args="-F $WORKSPACE/ssh.config.ansible" -i $WORKSPACE/hosts install/browbeat_network.yml
#Non functional until upstream fix released (jkipatr)
#ansible-playbook --ssh-common-args="-F $WORKSPACE/ssh.config.ansible" -i $WORKSPACE/hosts install/shaker_build.yml
#Required to change the configs on the Undercloud since we must run from there
$SCP_CMD $WORKSPACE/browbeat/ci-scripts/config/browbeat-ci.yaml undercloud:/home/stack/browbeat/browbeat-config.yaml
popd
}
runGather()
{
pushd $WORKSPACE/browbeat/ansible
ansible-playbook --ssh-common-args="-F $WORKSPACE/ssh.config.ansible" -i $WORKSPACE/hosts gather/site.yml
popd
}
runBrowbeat()
{
$SSH_CMD ". /home/stack/browbeat-venv/bin/activate; cd /home/stack/browbeat; python browbeat.py $1"
}
#minimum check
time deployCloud
time deployBrowbeat
pushd $WORKSPACE/browbeat
if [[ $($DIFF_CMD | grep -i metadata) != "" ]]
then
time runGather
fi
popd
pushd $WORKSPACE/browbeat
#re-add shaker to tools list when fixed upstream
#re-add perfkit to this list after investigation
for tool in rally; do
# remove comments after further testing
if [[ $($DIFF_CMD | grep -i $tool) != "" ]]
then
time runBrowbeat $tool
fi
done
popd
exit 0