x/fuel-ccp-ci-config
Code Issues Proposed changes

Retire repository

Browse Source 
Fuel (from openstack namespace) and fuel-ccp (in x namespace)
repositories are unused and ready to retire.

This change removes all content from the repository and adds the usual
README file to point out that the repository is retired following the
process from
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project

See also
http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011647.html

Depends-On: https://review.opendev.org/699362
Change-Id: I86573e6a50a146b8920779ee4bfa9d4c75729ea2
This commit is contained in:
Andreas Jaeger 2019-12-18 09:40:21 +01:00
parent 282921041d
commit 7f1d7a878a
49 changed files with 10 additions and 2656 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -1,2 +0,0 @@
.tox
output

176
LICENSE
View File

@ -1,176 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

10
README.rst Normal file
View File

@ -0,0 +1,10 @@
This project is no longer maintained.
The contents of this repository are still available in the Git
source code management system. To see the contents of this
repository before it reached its end of life, please check out the
previous commit with "git checkout HEAD^1".
For any further questions, please email
openstack-discuss@lists.openstack.org or join #openstack-dev on
Freenode.

9
jenkins-tp/conf/jenkins_job.ini.example
View File

@ -1,9 +0,0 @@
[jenkins]
user=USER
password=APIKEY
url=https://jenkins-tp.ng.mirantis.net
query_plugins_info=False
[job_builder]
ignore_cache=True
keep_descriptions=False

16
jenkins-tp/jobs/build-branch-all-template.yaml
View File

@ -1,16 +0,0 @@
- job-template:
name: test-{os_version}-build-all
description: |
Fuel CCP test build job
node: standard
builders:
- inject:
properties-content: |
OS_VER: '{os_version}'
- get-from-zuul
- shell: !include-raw-escape builders/build-all-images-branch.sh
concurrent: true
wrappers:
- ng-cleanup
triggers:
- zuul

23
jenkins-tp/jobs/build-branch-job-template.yaml
View File

@ -1,23 +0,0 @@
#
# Template used to execute tox tests
#
- job-template:
name: test-{os_version}-build-{name}
builders:
- zuul-clone
- build-image-branch:
name: '{name}'
repos: '{repos}'
GIT_BRANCH: '{os_version}'
concurrent: true
description: |
Test build {name} image job
node: standard
wrappers:
- ng-cleanup
- fuel-ccp-ci-jenkins
triggers:
- zuul
properties:
- least-load:
disabled: False

19
jenkins-tp/jobs/build-job-template.yaml
View File

@ -1,19 +0,0 @@
#
# Template used to execute tox tests
#
- job-template:
name: test-build-{name}
builders:
- get-from-zuul-container
- build-image-branch:
name: '{name}'
repos: '{repos}'
GIT_BRANCH: 'master'
concurrent: true
description: |
Test build {name} image job
node: standard
wrappers:
- ng-cleanup
triggers:
- zuul

20
jenkins-tp/jobs/build-job-with-tox-template.yaml
View File

@ -1,20 +0,0 @@
#
# Template used to execute tox tests
#
- job-template:
name: test-build-with-tox-{component}
builders:
- get-from-zuul-container
- build-image:
component: '{component}'
repos: '{repos}'
- check-tox:
tox-test: '{tox-test}'
concurrent: true
description: |
Test build {component} image job
node: standard
wrappers:
- ng-cleanup
triggers:
- zuul

86
jenkins-tp/jobs/builders/build-all-images-branch.sh
View File

@ -1,86 +0,0 @@
#!/bin/bash -xe
virtualenv microenv
source microenv/bin/activate
pip install --upgrade pip
pip install .
export PATH="$HOME/.local/bin:$PATH"
export APT_CACHE_SERVER_IP="`getent hosts cache-scc.ng.mirantis.net| awk '{{print $1}}'`"
export APT_CACHE_SERVER_PORT="3142"
export APT_CACHE_SERVER="${APT_CACHE_SERVER_IP}:${APT_CACHE_SERVER_PORT}"
if [ $OS_VER == "master" ]; then
GIT_BRANCH="master"
else
GIT_BRANCH="stable/newton"
fi
tee ccp-test.yaml << EOF
debug: True
builder:
no_cache: True
images:
namespace: build-all
tag: latest
maintainer: mirantis@mirantis.com
repositories:
path: ${WORKSPACE}/containers/openstack
skip_empty: True
sources:
openstack/cinder:
git_url: https://github.com/openstack/cinder.git
git_ref: ${GIT_BRANCH}
openstack/ironic:
git_url: https://github.com/openstack/ironic.git
git_ref: ${GIT_BRANCH}
openstack/murano:
git_url: https://github.com/openstack/murano.git
git_ref: ${GIT_BRANCH}
openstack/murano-dashboard:
git_url: https://github.com/openstack/murano-dashboard.git
git_ref: ${GIT_BRANCH}
openstack/sahara:
git_url: https://github.com/openstack/sahara.git
git_ref: ${GIT_BRANCH}
openstack/sahara-dashboard:
git_url: https://github.com/openstack/sahara-dashboard.git
git_ref: ${GIT_BRANCH}
openstack/keystone:
git_url: https://github.com/openstack/keystone.git
git_ref: ${GIT_BRANCH}
openstack/horizon:
git_url: https://github.com/openstack/horizon.git
git_ref: ${GIT_BRANCH}
openstack/nova:
git_url: https://github.com/openstack/nova.git
git_ref: ${GIT_BRANCH}
openstack/neutron:
git_url: https://github.com/openstack/neutron.git
git_ref: ${GIT_BRANCH}
openstack/heat:
git_url: https://github.com/openstack/heat.git
git_ref: ${GIT_BRANCH}
openstack/glance:
git_url: https://github.com/openstack/glance.git
git_ref: ${GIT_BRANCH}
openstack/requirements:
git_url: https://github.com/openstack/requirements.git
git_ref: ${GIT_BRANCH}
url:
debian: http://${APT_CACHE_SERVER}/debian
security: http://${APT_CACHE_SERVER}/security
ceph:
debian:
repo: http://${APT_CACHE_SERVER}/ceph
mariadb:
debian:
repo: http://${APT_CACHE_SERVER}/mariadb
EOF
ccp --config-file ccp-test.yaml build
deactivate

93
jenkins-tp/jobs/builders/build-image-branch.sh
View File

@ -1,93 +0,0 @@
#!/bin/bash -xe
JOB_NAME={name}
OS_VER={GIT_BRANCH}
REPO_LIST="{repos}"
export PATH="$HOME/.local/bin:$PATH"
export APT_CACHE_SERVER_IP="`getent hosts cache-scc.ng.mirantis.net| awk '{{print $1}}'`"
export APT_CACHE_SERVER_PORT="3142"
export APT_CACHE_SERVER="$APT_CACHE_SERVER_IP:$APT_CACHE_SERVER_PORT"
if [ $OS_VER == "master" ]; then
GIT_BRANCH=master
else
GIT_BRANCH=stable/newton
fi
tee fuel-ccp/ccp-test.yaml << EOF
debug: True
builder:
no_cache: True
images:
namespace: $JOB_NAME-build
tag: latest
maintainer: mirantis@mirantis.com
repositories:
repos:
EOF
for r in $REPO_LIST;do
tee -a fuel-ccp/ccp-test.yaml << EOF
- git_url: https://git.openstack.org/openstack/$r
name: ${{r/fuel-ccp-/}}
EOF
done
tee -a fuel-ccp/ccp-test.yaml << EOF
path: $WORKSPACE/containers/openstack
skip_empty: True
sources:
openstack/cinder:
git_url: https://github.com/openstack/cinder.git
git_ref: $GIT_BRANCH
openstack/ironic:
git_url: https://github.com/openstack/ironic.git
git_ref: $GIT_BRANCH
openstack/murano:
git_url: https://github.com/openstack/murano.git
git_ref: $GIT_BRANCH
openstack/murano-dashboard:
git_url: https://github.com/openstack/murano-dashboard.git
git_ref: $GIT_BRANCH
openstack/sahara:
git_url: https://github.com/openstack/sahara.git
git_ref: $GIT_BRANCH
openstack/sahara-dashboard:
git_url: https://github.com/openstack/sahara-dashboard.git
git_ref: $GIT_BRANCH
openstack/keystone:
git_url: https://github.com/openstack/keystone.git
git_ref: $GIT_BRANCH
openstack/horizon:
git_url: https://github.com/openstack/horizon.git
git_ref: $GIT_BRANCH
openstack/nova:
git_url: https://github.com/openstack/nova.git
git_ref: $GIT_BRANCH
openstack/neutron:
git_url: https://github.com/openstack/neutron.git
git_ref: $GIT_BRANCH
openstack/heat:
git_url: https://github.com/openstack/heat.git
git_ref: $GIT_BRANCH
openstack/glance:
git_url: https://github.com/openstack/glance.git
git_ref: $GIT_BRANCH
openstack/requirements:
git_url: https://github.com/openstack/requirements.git
git_ref: $GIT_BRANCH
url:
debian: http://$APT_CACHE_SERVER/debian
security: http://$APT_CACHE_SERVER/security
ceph:
debian:
repo: http://$APT_CACHE_SERVER/ceph
mariadb:
debian:
repo: http://$APT_CACHE_SERVER/mariadb
EOF
cd fuel-ccp
tox -e venv -- ccp --config-file ccp-test.yaml build

20
jenkins-tp/jobs/builders/clone-with-zuul.sh
View File

@ -1,20 +0,0 @@
#!/bin/bash -ex
rep=`ssh -o StrictHostKeyChecking=no -p 29418 mirantis-fuel-ccp-ci@review.openstack.org gerrit ls-projects | grep "fuel-ccp-"| tr '\n' ' '`
tee -a clonemap.yaml << EOF
clonemap:
- name: openstack/fuel-ccp
dest: fuel-ccp
EOF
for f in ${rep}; do
tee -a clonemap.yaml << EOF
- name: ${f}
dest: containers/${f/fuel-ccp-/}
EOF
done
source /home/jenkins/venv-zuul/bin/activate
zuul-cloner -m clonemap.yaml ssh://mirantis-fuel-ccp-ci@review.openstack.org:29418 openstack/fuel-ccp ${rep}
deactivate

139
jenkins-tp/jobs/builders/fuel-ccp-debug-deployment.sh
View File

@ -1,139 +0,0 @@
#!/bin/bash -ex
# Deploys CCP and runs simple, built-in OpenStack tests.
# Kubernetes cluster is expected to be pre-deployed and snapshoted - if not
# this script will ensure this.
# The script expects fuel-ccp cloned into fuel-ccp/ and fuel-ccp-installer
# cloned into fuel-ccp-installer/ directory (e.g. by Jenkins SCM plugin).
# CONFIGURATION:
######################################################
: ${FUEL_DEVOPS_ENV_NAME:="fuel-ccp-debug-${VERSION}-deployment"}
FUEL_DEVOPS_SNAPSHOT_NAME="fresh"
FUEL_DEVOPS_INSTALLATION_DIR="/home/jenkins/venv-fuel-devops-3.0"
HYPERKUBE_REPO="quay.io/coreos/hyperkube"
HYPERKUBE_TAG="v1.5.1_coreos.0"
HYPERKUBE_VERSION="v1.5.1"
export APT_CACHE_SERVER_IP="`getent hosts cache-scc.ng.mirantis.net| awk '{print $1}'`"
export APT_CACHE_SERVER_PORT="3142"
export REGISTRY_IP=`ip -4 addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}'`
export REGISTRY_PORT=5000
JENKINS_GID=`getent group jenkins | cut -d":" -f3`
REGISTRY_HASH=`docker inspect --format "{{.Id}}" registry`
# Prepare K8s env:
source "${FUEL_DEVOPS_INSTALLATION_DIR}"/bin/activate
NEED_TO_SETUP_ENV=false
dos.py revert "${FUEL_DEVOPS_ENV_NAME}" "${FUEL_DEVOPS_SNAPSHOT_NAME}" || NEED_TO_SETUP_ENV=true
dos.py resume "${FUEL_DEVOPS_ENV_NAME}" || NEED_TO_SETUP_ENV=true
if [ "${NEED_TO_SETUP_ENV}" = "true" ]; then
echo "Couldn't revert fuel-devops snapshot, recreating environment."
# Try to delete old envs to ensure clean host:
dos.py destroy "${FUEL_DEVOPS_ENV_NAME}" || true
dos.py erase "${FUEL_DEVOPS_ENV_NAME}" || true
# Create kargo deployment script:
cat > k8s_deploy.sh << EOF
#!/bin/bash -ex
export ENV_NAME=${FUEL_DEVOPS_ENV_NAME}
export IMAGE_PATH="/home/jenkins/workspace/cloud-images/default-image.qcow2"
export DONT_DESTROY_ON_SUCCESS=1
export DEPLOY_METHOD="kargo"
export SLAVES_COUNT="3"
export WORKSPACE="/home/jenkins/workspace"
export FUEL_DEVOPS_INSTALLATION_DIR=${FUEL_DEVOPS_INSTALLATION_DIR}
export CUSTOM_YAML='hyperkube_image_repo: "${HYPERKUBE_REPO}"
hyperkube_image_tag: "${HYPERKUBE_TAG}"
kube_version: "${HYPERKUBE_VERSION}"
docker_options: "--insecure-registry=${REGISTRY_IP}:${REGISTRY_PORT}"'
echo "Running on \${NODE_NAME}: \${ENV_NAME}"
source \${FUEL_DEVOPS_INSTALLATION_DIR}/bin/activate
echo "Use image:"
ls -la \${IMAGE_PATH}
env
pushd fuel-ccp-installer
bash -ex "./utils/jenkins/run_k8s_deploy_test.sh"
popd
EOF
chmod +x k8s_deploy.sh
# Now deploy the cluster:
./k8s_deploy.sh
# Create fresh snapshot:
dos.py suspend "${FUEL_DEVOPS_ENV_NAME}"
dos.py snapshot "${FUEL_DEVOPS_ENV_NAME}" "${FUEL_DEVOPS_SNAPSHOT_NAME}"
# Resume from snapshot to deploy CCP later on in the script:
dos.py resume "${FUEL_DEVOPS_ENV_NAME}"
fi
# Get IP address of first node in the cluster:
ADMIN_IP=$(ENV_NAME=${FUEL_DEVOPS_ENV_NAME} python fuel-ccp-installer/utils/jenkins/env.py get_slaves_ips | grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" | head -1)
SSH_COMMAND="sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@${ADMIN_IP}"
SCP_COMMAND="sshpass -p vagrant scp -o StrictHostKeyChecking=no"
# remove old key:
ssh-keygen -R "${ADMIN_IP}"
# Store info about Jenkins job on VM:
echo "${BUILD_TAG}" | ${SSH_COMMAND} "tee -a JENKINS_INFO.TXT"
# After restore snapshot ntp service doesn't work and need restart on all nodes
${SSH_COMMAND} "sudo service ntp restart"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node2 sudo service ntp restart"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node3 sudo service ntp restart"
# Dirty hack for workaround network problems on CI envs.
# When we deploy env some time after (few minutes) it change resolv.conf into broken one
# From this reason after bring up env we restart network and and kill dhclient,
# we also restart docker and kubelet to make sure that all net-host containers are in good shape
cat > fix_dns.sh << EOF
sudo service networking restart
sudo pkill -9 dhclient
sudo service kubelet restart
sudo service docker restart
EOF
chmod +x fix_dns.sh
${SCP_COMMAND} fix_dns.sh vagrant@"${ADMIN_IP}":~/
${SSH_COMMAND} "scp -o StrictHostKeyChecking=no fix_dns.sh vagrant@node2:~/"
${SSH_COMMAND} "scp -o StrictHostKeyChecking=no fix_dns.sh vagrant@node3:~/"
${SSH_COMMAND} "sudo ./fix_dns.sh"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node2 sudo ./fix_dns.sh"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node3 sudo ./fix_dns.sh"
sed -i 's/127.0.0.1:31500/'${REGISTRY_IP}':'${REGISTRY_PORT}'/g' fuel-ccp/tools/ccp-multi-deploy/config/ccp-configs-common.yaml
cat >> fuel-ccp/tools/ccp-multi-deploy/config/ccp-configs-common.yaml << EOF
images:
tag: "${BUILD_ID}"
EOF
${SCP_COMMAND} -r fuel-ccp/ vagrant@"${ADMIN_IP}":~/
# Run CCP deployment and OpenStack tests:
${SSH_COMMAND} "pushd fuel-ccp && APT_CACHE_SERVER=http://${APT_CACHE_SERVER_IP}:${APT_CACHE_SERVER_PORT} tox -e multi-deploy -- --openstack-version ${VERSION} --number-of-envs 1 -d"
docker exec "${REGISTRY_HASH}" chgrp -R "${JENKINS_GID}" /var/lib/registry
docker exec "${REGISTRY_HASH}" chmod -R g+w /var/lib/registry
for i in `curl http://${REGISTRY_IP}:${REGISTRY_PORT}/v2/_catalog | jq -r '.repositories[]'`; do
REGISTRY_DATA_DIR=/home/jenkins/registry/data/docker/registry/v2/ /home/jenkins/registry/delete_docker_registry_image.py --image "${i}":"${BUILD_ID}"
done
# Revert to fresh to decrease image size
dos.py revert "${FUEL_DEVOPS_ENV_NAME}" "${FUEL_DEVOPS_SNAPSHOT_NAME}"
# Clean-up (snapshot should remain for next jobs):
dos.py destroy "${FUEL_DEVOPS_ENV_NAME}"

4
jenkins-tp/jobs/builders/fuel-ccp-ext-config-gate.sh
View File

@ -1,4 +0,0 @@
#!/bin/bash -xe
# run pep8, py27 and py34 checks
tox

23
jenkins-tp/jobs/builders/fuel-ccp-installer-gate.sh
View File

@ -1,23 +0,0 @@
#!/bin/bash -x
set -e
env
export ENV_NAME="env-k8s-kargo-$BUILD_TAG"
export IMAGE_PATH="/home/jenkins/workspace/cloud-images/default-image.qcow2"
export DONT_DESTROY_ON_SUCCESS=1
#export VLAN_BRIDGE="" # custom bridge connected to vlan
export DEPLOY_METHOD="kargo"
export SLAVES_COUNT="3"
export WORKSPACE="/home/jenkins/workspace"
export FUEL_DEVOPS_INSTALLATION_DIR="/home/jenkins/venv-fuel-devops-3.0"
export CUSTOM_YAML='hyperkube_image_repo: "quay.io/coreos/hyperkube"
hyperkube_image_tag: "v1.5.1_coreos.0"'
echo "Running on $NODE_NAME: $ENV_NAME"
source ${FUEL_DEVOPS_INSTALLATION_DIR}/bin/activate
echo "Use image:"
ls -la $IMAGE_PATH
bash -ex "utils/jenkins/run_k8s_deploy_test.sh"
echo "[TODO] We need some check of K8s deployment here."
echo "Cleaning up:"
dos.py erase $ENV_NAME

24
jenkins-tp/jobs/builders/fuel-ccp-qa-gate.sh
View File

@ -1,24 +0,0 @@
#!/bin/bash -xe
export PATH=/bin:/usr/bin:/sbin:/usr/sbin:${PATH}
function check_venv_exists {
VIRTUAL_ENV=mcp-qa-venv
if [ -f ${VIRTUAL_ENV}/bin/activate ]; then
source ${VIRTUAL_ENV}/bin/activate
echo "Python virtual env exist"
else
rm -rf ${VIRTUAL_ENV}
virtualenv --no-site-packages ${VIRTUAL_ENV}
source ${VIRTUAL_ENV}/bin/activate
fi
}
check_venv_exists
pip install -r fuel_ccp_tests/requirements.txt
py.test -k mysql_is_running
deactivate

464
jenkins-tp/jobs/builders/fuel-ccp-simple-deployment.sh
View File

@ -1,464 +0,0 @@
#!/bin/bash -ex
# Deploys CCP and runs simple, built-in OpenStack tests.
# Kubernetes cluster is expected to be pre-deployed and snapshoted - if not
# this script will ensure this.
# The script expects fuel-ccp cloned into fuel-ccp/ and fuel-ccp-installer
# cloned into fuel-ccp-installer/ directory (e.g. by Jenkins SCM plugin).
# CONFIGURATION:
######################################################
: ${FUEL_DEVOPS_ENV_NAME:="fuel-ccp-${COMPONENT}-${VERSION}-deployment"}
FUEL_DEVOPS_SNAPSHOT_NAME="fresh"
FUEL_DEVOPS_INSTALLATION_DIR="/home/jenkins/venv-fuel-devops-3.0"
HYPERKUBE_REPO="quay.io/coreos/hyperkube"
HYPERKUBE_TAG="v1.5.1_coreos.0"
HYPERKUBE_VERSION="v1.5.1"
export APT_CACHE_SERVER_IP="`getent hosts cache-scc.ng.mirantis.net| awk '{print $1}'`"
export APT_CACHE_SERVER_PORT="3142"
export REGISTRY_IP=`ip -4 addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}'`
export REGISTRY_PORT=5000
JENKINS_GID=`getent group jenkins | cut -d":" -f3`
REGISTRY_HASH=`docker inspect --format "{{.Id}}" registry`
export DOCKER_REGISTRY_HOST="registry.mcp.fuel-infra.org"
export DOCKER_REGISTRY="${DOCKER_REGISTRY_HOST}:6000"
export DOCKER_REGISTRY_IP="`getent hosts ${DOCKER_REGISTRY_HOST} | awk '{print $1}'`"
export IMAGES_NAMESPACE="ccp"
export REGISTRY_NAMESPACE="mcp"
export SHARE_HOST="share01-scc.ng.mirantis.net"
export SHARE_IP="`getent hosts ${SHARE_HOST} | awk '{print $1}'`"
function prepare_k8s_env {
# Prepare K8s env:
source "${FUEL_DEVOPS_INSTALLATION_DIR}"/bin/activate
NEED_TO_SETUP_ENV=false
dos.py revert "${FUEL_DEVOPS_ENV_NAME}" "${FUEL_DEVOPS_SNAPSHOT_NAME}" || NEED_TO_SETUP_ENV=true
dos.py resume "${FUEL_DEVOPS_ENV_NAME}" || NEED_TO_SETUP_ENV=true
if [ "${NEED_TO_SETUP_ENV}" = "true" ]; then
echo "Couldn't revert fuel-devops snapshot, recreating environment."
# Try to delete old envs to ensure clean host:
dos.py destroy "${FUEL_DEVOPS_ENV_NAME}" || true
dos.py erase "${FUEL_DEVOPS_ENV_NAME}" || true
# Create kargo deployment script:
cat > k8s_deploy.sh << EOF
#!/bin/bash -ex
export ENV_NAME=${FUEL_DEVOPS_ENV_NAME}
export IMAGE_PATH="/home/jenkins/workspace/cloud-images/default-image.qcow2"
export DONT_DESTROY_ON_SUCCESS=1
export DEPLOY_METHOD="kargo"
export SLAVES_COUNT="3"
export ADMIN_NODE_CPU=5
export ADMIN_NODE_MEMORY=14336
export SLAVE_NODE_CPU=3
export SLAVE_NODE_MEMORY=8192
export WORKSPACE="/home/jenkins/workspace"
export FUEL_DEVOPS_INSTALLATION_DIR=${FUEL_DEVOPS_INSTALLATION_DIR}
export CUSTOM_YAML='hyperkube_image_repo: "${HYPERKUBE_REPO}"
hyperkube_image_tag: "${HYPERKUBE_TAG}"
kube_version: "${HYPERKUBE_VERSION}"
docker_options: "--insecure-registry=${REGISTRY_IP}:${REGISTRY_PORT} --insecure-registry=DOCKER_REGISTRY"'
echo "Running on \${NODE_NAME}: \${ENV_NAME}"
source \${FUEL_DEVOPS_INSTALLATION_DIR}/bin/activate
echo "Use image:"
ls -la \${IMAGE_PATH}
env
pushd fuel-ccp-installer
bash -ex "./utils/jenkins/run_k8s_deploy_test.sh"
popd
EOF
chmod +x k8s_deploy.sh
# Now deploy the cluster:
./k8s_deploy.sh
# Create fresh snapshot:
dos.py suspend "${FUEL_DEVOPS_ENV_NAME}"
dos.py snapshot "${FUEL_DEVOPS_ENV_NAME}" "${FUEL_DEVOPS_SNAPSHOT_NAME}"
# Resume from snapshot to deploy CCP later on in the script:
dos.py resume "${FUEL_DEVOPS_ENV_NAME}"
fi
}
function fix_restored_env {
# After restore snapshot ntp service doesn't work and need restart on all nodes
${SSH_COMMAND} "sudo service ntp restart"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node2 sudo service ntp restart"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node3 sudo service ntp restart"
# Dirty hack for workaround network problems on CI envs.
# When we deploy env some time after (few minutes) it change resolv.conf into broken one
# From this reason after bring up env we restart network and and kill dhclient,
# we also restart docker and kubelet to make sure that all net-host containers are in good shape
cat > fix_dns.sh << EOF
sudo service networking restart
sudo pkill -9 dhclient
sudo service kubelet restart
sudo service docker restart
EOF
chmod +x fix_dns.sh
${SCP_COMMAND} fix_dns.sh vagrant@"${ADMIN_IP}":~/
${SSH_COMMAND} "scp -o StrictHostKeyChecking=no fix_dns.sh vagrant@node2:~/"
${SSH_COMMAND} "scp -o StrictHostKeyChecking=no fix_dns.sh vagrant@node3:~/"
${SSH_COMMAND} "sudo ./fix_dns.sh"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node2 sudo ./fix_dns.sh"
${SSH_COMMAND} "ssh -o StrictHostKeyChecking=no node3 sudo ./fix_dns.sh"
}
function copy_code_to_vm {
# Prepare env on "admin" VM:
if [ ${COMPONENT} == "full" ]; then
${SCP_COMMAND} -r fuel-ccp/ vagrant@"${ADMIN_IP}":~/
elif [ ${COMPONENT} == "smoke" ]; then
${SCP_COMMAND} -r fuel-ccp/ vagrant@"${ADMIN_IP}":~/
else
${SCP_COMMAND} -r fuel-ccp/ vagrant@"${ADMIN_IP}":~/
# set +x is just for security reasons to avoid publishing internal IP
set +x
getent hosts zuul.mcp.fuel-infra.org | ${SSH_COMMAND} "sudo tee -a /etc/hosts"
set -x
${SCP_COMMAND} -r containers/openstack/ vagrant@"${ADMIN_IP}":/tmp/ccp-repos
fi
}
function prepare_ccp_config {
cat > ccp.yml << EOF
builder:
push: True
workers: 1
registry:
address: "${REGISTRY_IP}:${REGISTRY_PORT}"
timeout: 1500
replicas:
etcd: 3
database: 3
rpc: 3
notifications: 1
repositories:
path: /tmp/ccp-repos
skip_empty: True
entrypoint_repo_name: entrypoint
repos:
- git_url: https://git.openstack.org/openstack/fuel-ccp-debian-base
name: debian-base
- git_url: https://git.openstack.org/openstack/fuel-ccp-entrypoint
name: entrypoint
- git_url: https://git.openstack.org/openstack/fuel-ccp-etcd
name: etcd
- git_url: https://git.openstack.org/openstack/fuel-ccp-galera
name: galera
- git_url: https://git.openstack.org/openstack/fuel-ccp-glance
name: glance
- git_url: https://git.openstack.org/openstack/fuel-ccp-heat
name: heat
- git_url: https://git.openstack.org/openstack/fuel-ccp-horizon
name: horizon
- git_url: https://git.openstack.org/openstack/fuel-ccp-keystone
name: keystone
- git_url: https://git.openstack.org/openstack/fuel-ccp-memcached
name: memcached
- git_url: https://git.openstack.org/openstack/fuel-ccp-neutron
name: neutron
- git_url: https://git.openstack.org/openstack/fuel-ccp-nova
name: nova
- git_url: https://git.openstack.org/openstack/fuel-ccp-nginx
name: nginx
- git_url: https://git.openstack.org/openstack/fuel-ccp-openstack-base
name: openstack-base
- git_url: https://git.openstack.org/openstack/fuel-ccp-rabbitmq
name: rabbitmq
- git_url: https://git.openstack.org/openstack/fuel-ccp-ceph
name: ceph
- git_url: https://git.openstack.org/openstack/fuel-ccp-ironic
name: ironic
- git_url: https://git.openstack.org/openstack/fuel-ccp-rally
name: rally
configs:
private_interface: ens3
public_interface: ens4
neutron:
physnets:
- name: "physnet1"
bridge_name: "br-ex"
interface: "ens4"
flat: true
vlan_range: "1001:1030"
dpdk: false
etcd:
tls:
enabled: false
rally:
tempest:
openstack_release: ${VERSION}
nodes:
node[1-3]:
roles:
- controller-net-bridge
node1:
roles:
- db
- messaging
- openvswitch
- controller-net-host
node[2-3]:
roles:
- db
- messaging
- openvswitch
- compute
sources:
openstack/cinder:
git_url: https://git.openstack.org/openstack/cinder.git
git_ref: ${BRANCH}
openstack/ironic:
git_url: https://git.openstack.org/openstack/ironic.git
git_ref: ${BRANCH}
openstack/murano:
git_url: https://git.openstack.org/openstack/murano.git
git_ref: ${BRANCH}
openstack/murano-dashboard:
git_url: https://git.openstack.org/openstack/murano-dashboard.git
git_ref: ${BRANCH}
openstack/sahara:
git_url: https://git.openstack.org/openstack/sahara.git
git_ref: ${BRANCH}
openstack/sahara-dashboard:
git_url: https://git.openstack.org/openstack/sahara-dashboard.git
git_ref: ${BRANCH}
openstack/keystone:
git_url: https://git.openstack.org/openstack/keystone.git
git_ref: ${BRANCH}
openstack/horizon:
git_url: https://git.openstack.org/openstack/horizon.git
git_ref: ${BRANCH}
openstack/nova:
git_url: https://git.openstack.org/openstack/nova.git
git_ref: ${BRANCH}
openstack/neutron:
git_url: https://git.openstack.org/openstack/neutron.git
git_ref: ${BRANCH}
openstack/networking-odl:
git_url: https://git.openstack.org/openstack/networking-odl.git
git_ref: ${BRANCH}
openstack/heat:
git_url: https://git.openstack.org/openstack/heat.git
git_ref: ${BRANCH}
openstack/glance:
git_url: https://git.openstack.org/openstack/glance.git
git_ref: ${BRANCH}
openstack/requirements:
git_url: https://git.openstack.org/openstack/requirements.git
git_ref: ${BRANCH}
services:
database:
service_def: galera
rpc:
service_def: rabbitmq
notifications:
service_def: rabbitmq
roles:
db:
- database
messaging:
- rpc
- notifications
controller-net-host:
- neutron-dhcp-agent
- neutron-l3-agent
- neutron-metadata-agent
controller-net-bridge:
- etcd
- glance-api
- glance-registry
- heat-api-cfn
- heat-api
- heat-engine
- horizon
- keystone
- memcached
- neutron-server
- nova-api
- nova-conductor
- nova-consoleauth
- nova-novncproxy
- nova-scheduler
compute:
- nova-compute
- nova-libvirt
openvswitch:
- neutron-openvswitch-agent
- openvswitch-db
- openvswitch-vswitchd
EOF
}
ccp_wait_for_deployment_to_finish () {
cnt=0
until [[ `${SSH_COMMAND} ccp status -s -f value -c status` == "ok" ]]; do
echo "Waiting for OpenStack deployment to finish..."
sleep 5
cnt=$((cnt + 1))
if [ ${cnt} -eq $1 ]; then
echo "Max time exceeded"
${SSH_COMMAND} ccp status
${SSH_COMMAND} fuel-ccp/tools/diagnostic-snapshot.sh -n ccp -c ccp.yml
return 1
fi
done
echo "...................................."
echo "Jobs and pods in namespace: ccp"
${SSH_COMMAND} kubectl --namespace ccp get jobs
${SSH_COMMAND} kubectl --namespace ccp get pods
echo "openrc file: openrc-ccp"
${SSH_COMMAND} cat openrc-ccp
echo "...................................."
}
function ccp_install {
${SSH_COMMAND} "sudo -H pip install -r fuel-ccp/requirements.txt"
${SSH_COMMAND} "sudo -H pip install fuel-ccp/"
}
function deploy_ccp {
pwd
${SSH_COMMAND} "ccp -vvv --debug --config-file ~/ccp.yml build -c etcd memcached rabbitmq galera percona rabbitmq"
${SSH_COMMAND} "ccp -vvv --debug --config-file ~/ccp.yml deploy -c etcd memcached database rpc notifications"
ccp_wait_for_deployment_to_finish 70
if [ $? -ne 0 ]; then
return 1
fi
${SSH_COMMAND} "ccp -vvv --debug --config-file ~/ccp.yml build"
${SSH_COMMAND} "ccp -vvv --debug --config-file ~/ccp.yml deploy"
ccp_wait_for_deployment_to_finish 200
if [ $? -ne 0 ]; then
return 1
fi
}
prepare_k8s_env
# Get IP address of first node in the cluster:
ADMIN_IP=$(ENV_NAME=${FUEL_DEVOPS_ENV_NAME} python fuel-ccp-installer/utils/jenkins/env.py get_slaves_ips | grep -o "[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+" | head -1)
SSH_COMMAND="sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@${ADMIN_IP}"
SCP_COMMAND="sshpass -p vagrant scp -o StrictHostKeyChecking=no"
# remove old key:
ssh-keygen -R "${ADMIN_IP}"
# Let's wait 5 sec to all vms up
sleep 5
# Store info about Jenkins job on VM:
echo "${BUILD_TAG}" | ${SSH_COMMAND} "tee -a JENKINS_INFO.TXT"
fix_restored_env
copy_code_to_vm
# set version of openstack
if [ "${VERSION}" == "master" ];then
export IMAGES_TAG="latest"
export BRANCH="master"
elif [ "${VERSION}" == "ocata" ];then
export IMAGES_TAG="ocata"
export BRANCH="stable/ocata"
elif [ "${VERSION}" == "newton" ]; then
export IMAGES_TAG="newton"
export BRANCH="stable/newton"
else
export IMAGES_TAG="mitaka"
export BRANCH="stable/mitaka"
fi
prepare_ccp_config
ccp_install
if [ ${COMPONENT} == "smoke" ]; then
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${ADMIN_IP}" "echo ${SHARE_IP} ${SHARE_HOST} |sudo tee -a /etc/hosts"
set +e
ssh -i ~/.ssh/jenkins_storage share@share01-scc.ng.mirantis.net rm /srv/static/share/tests/tests/result-${VERSION}.xml
set -e
${SCP_COMMAND} ccp.yml vagrant@"${ADMIN_IP}":~/
${SSH_COMMAND} "ccp -vvv --debug --config-file ~/ccp.yml fetch"
${SCP_COMMAND} -r ~/skel/* vagrant@"${ADMIN_IP}":/tmp/ccp-repos/rally/service/files
set +e
# Run CCP deployment and OpenStack tests:
deploy_ccp
DEPLOY_STATUS=$?
#set tag dependent from test result
if [[ "${DEPLOY_STATUS}" != 0 ]]; then
echo "Deployment failed! Check diagnostic snapshot."
${SCP_COMMAND} vagrant@"${ADMIN_IP}":~/tmp/ccp-diag/*.tar.gz .
scp -i ~/.ssh/jenkins_storage *.tar.gz share@share01-scc.ng.mirantis.net:/srv/static/share/tests/diagnostic/
exit 1
fi
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${ADMIN_IP}" "echo ${DOCKER_REGISTRY_IP} ${DOCKER_REGISTRY_HOST} |sudo tee -a /etc/hosts"
TEMPEST_NAME=`sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${ADMIN_IP}" "ccp -vvv --debug --config-file ~/ccp.yml action run tempest -f value -c name"`
stop = 0
until [ ${stop} -eq 1 ]; do
test_status=`${SSH_COMMAND} ccp -vvv --debug action status ${TEMPEST_NAME} -f value -c status`
if [ "${test_status}" == "fail" ]; then
stop=1
DOCKER_TAG="${IMAGES_TAG}-unstable"
elif [ "${test_status}" == "ok" ]; then
stop=1
DOCKER_TAG="${IMAGES_TAG}"
else
sleep 60
fi
done
${SSH_COMMAND} kubectl -n ccp logs ${TEMPEST_NAME} | grep -A 9 Totals
${SSH_COMMAND} kubectl -n ccp logs ${TEMPEST_NAME} -p | grep -A 9 Totals
wget http://share01-scc.ng.mirantis.net/tests/tests/result-${VERSION}.xml
set -e
IMG=`sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@${ADMIN_IP} docker images --format "{{.Repository}}" | awk -F'/' -v search=/${IMAGES_NAMESPACE}/ '$0 ~ search {print $3}'`
# we need docker config file to authentication in remote repository
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${ADMIN_IP}" mkdir -p /home/vagrant/.docker/
sshpass -p vagrant scp -o StrictHostKeyChecking=no /home/jenkins/.docker/config.json vagrant@"${ADMIN_IP}":~/.docker/
for f in ${IMG}; do
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${ADMIN_IP}" \
"docker tag ${REGISTRY_IP}:${REGISTRY_PORT}/${IMAGES_NAMESPACE}/$f:latest ${DOCKER_REGISTRY}/${REGISTRY_NAMESPACE}/${f}:${DOCKER_TAG} && docker push ${DOCKER_REGISTRY}/${REGISTRY_NAMESPACE}/${f}:${DOCKER_TAG}"
if [ "${IMAGES_TAG}" == "ocata" ]; then
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${ADMIN_IP}" \
"docker tag ${REGISTRY_IP}:${REGISTRY_PORT}/${IMAGES_NAMESPACE}/${f}:latest ${DOCKER_REGISTRY}/${REGISTRY_NAMESPACE}/${f}:latest \
&& docker push ${DOCKER_REGISTRY}/${REGISTRY_NAMESPACE}/${f}:latest"
fi
done
else
set +e
# Run CCP deployment and OpenStack tests:
deploy_ccp
DEPLOY_STATUS=$?
echo "Deploy status: ${DEPLOY_STATUS}"
set -e
fi
# Revert to fresh to decrease image size
dos.py revert "${FUEL_DEVOPS_ENV_NAME}" "${FUEL_DEVOPS_SNAPSHOT_NAME}"
# Clean-up (snapshot should remain for next jobs):
dos.py destroy "${FUEL_DEVOPS_ENV_NAME}"

99
jenkins-tp/jobs/builders/mcp-build-images-smoke.sh
View File

@ -1,99 +0,0 @@
#!/bin/bash
set -ex
# This script builds images and use them to run a deployment test.
# If test passes it will push images to registry with "latest" tag.
# Otherwise it will push them with "unstable" tag.
# Logs from building images are downloaded to workspace in ccp.* files
#
# Configuration
#
# fuel devops environment name
export ENV_PREFIX="fuel-ccp-tests"
export ENV_NAME="${ENV_PREFIX}.${BUILD_NUMBER}"
# vm image
export IMAGE_PATH="${IMAGE_PATH}"
# workspace directory
export WORKSPACE="${WORKSPACE}"
# build images or not
export BUILD_IMAGES=True
# kargo script to deploy k8s
export DEPLOY_SCRIPT="${WORKSPACE}/fuel-ccp-installer/utils/jenkins/kargo_deploy.sh"
# tests configuration
export CONF_PATH="fuel_ccp_tests/templates/default.yaml"
# images namespace
export IMAGES_NAMESPACE="mcp"
# leave env after finishing test
export SHUTDOWN_ENV_ON_TEARDOWN=false
# docker registry
export DOCKER_REGISTRY="registry.mcp.fuel-infra.org"
# hypercube image
export HYPERKUBE_IMAGE_NAME="quay.io/coreos/hyperkube"
# hypercube version
export HYPERKUBE_IMAGE_TAG="v1.5.1_coreos.0"
# etcd version
export ETCD_IMAGE_TAG="v3.0.10"
# set version of openstack
if [ "${OS_VER}" == "ocata" ];then
export OS_RELEASE="master"
export IMAGES_TAG="ocata"
else
export OS_RELEASE="stable/newton"
export IMAGES_TAG="newton"
fi
if [[ -z ${IMAGE_PATH} ]]; then
echo "The IMAGE_PATH variable is not set!"
exit 1
fi
echo STARTED_TIME="$(date -u +'%Y-%m-%dT%H:%M:%S')" > ci_status_params.txt
virtualenv "${WORKSPACE}"/venv-fuel-ccp-tests
source "${WORKSPACE}"/venv-fuel-ccp-tests/bin/activate
pushd fuel-ccp-tests
pip install -r fuel_ccp_tests/requirements.txt
pip install -U .
popd
echo "----==== RUN TEST ====----"
cd fuel-ccp-tests
# we want to run next steps even if test fail
set +e
py.test -vvv -s fuel_ccp_tests/tests/system/test_deploy.py::TestDeployOpenstack::test_fuel_ccp_deploy_microservices
SMOKE_STATUS=$?
set -e
#set tag dependent from test result
if [[ "${SMOKE_STATUS}" == 0 ]]; then
DOCKER_TAG="${OS_VER}"
else
DOCKER_TAG="${OS_VER}-unstable"
fi
MASTER_IP=`awk '/kube_host/ {print $3}' ${WORKSPACE}/fuel-ccp-tests/${ENV_NAME}_k8s_deployed.ini`
ssh-keygen -R "${MASTER_IP}"
sshpass -p vagrant scp -o StrictHostKeyChecking=no vagrant@"${MASTER_IP}":ccp.* .
IMG=`sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@${MASTER_IP} docker images --format "{{.Repository}}" | awk -F'/' -v search=/${IMAGES_NAMESPACE}/ '$0 ~ search {print $3}'`
# we need docker config file to authentication in remote repository
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${MASTER_IP}" mkdir -p /home/vagrant/.docker/
sshpass -p vagrant scp -o StrictHostKeyChecking=no /home/jenkins/.docker/config.json vagrant@"${MASTER_IP}":~/.docker/
for f in ${IMG}; do
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${MASTER_IP}" \
"docker tag 127.0.0.1:31500/${IMAGES_NAMESPACE}/$f:${IMAGES_TAG} ${DOCKER_REGISTRY}/${IMAGES_NAMESPACE}/${f}:${DOCKER_TAG} && docker push ${DOCKER_REGISTRY}/${IMAGES_NAMESPACE}/${f}:${DOCKER_TAG}"
if [ "${DOCKER_TAG}" == "ocata" ]; then
sshpass -p vagrant ssh -o StrictHostKeyChecking=no vagrant@"${MASTER_IP}" \
"docker tag 127.0.0.1:31500/${IMAGES_NAMESPACE}/${f}:${IMAGES_TAG} ${DOCKER_REGISTRY}/${IMAGES_NAMESPACE}/${f}:latest \
&& docker push ${DOCKER_REGISTRY}/${IMAGES_NAMESPACE}/${f}:latest"
fi
done
deactivate

76
jenkins-tp/jobs/builders/mcp-build-images.sh
View File

@ -1,76 +0,0 @@
#!/bin/bash -ex
# Builds MCP images and pushes them to Docker registry with configurable
# tag.
# Optionally it can purge n older images (works correctly if tag is
# numerical).
# This script assumes that user is already authenticated to Docker
# registry (e.g. by running 'docker login').
# Parameters must be passed via env variables.
# CONFIGURATION:
######################################################
DOCKER_REGISTRY="registry.mcp.fuel-infra.org"
DOCKER_NAMESPACE="mcp"
IMAGES_MAINTAINER="mos-microservices@mirantis.com"
REPOSITORIES_PATH="microservices-repos"
: ${TAG:="latest"}
# Install fuel-ccp:
virtualenv mcp
source mcp/bin/activate
pip install .
if [ $TAG == "newton" ]; then
GIT_BRANCH="stable/newton"
else
GIT_BRANCH="master"
fi
# Build images:
cat > ccp-test.yaml << EOF
debug: True
builder:
no_cache: True
push: True
images:
namespace: ${DOCKER_NAMESPACE}
tag: "${TAG}"
maintainer: ${IMAGES_MAINTAINER}
repositories:
path: ${REPOSITORIES_PATH}
skip_empty: True
registry:
address: ${DOCKER_REGISTRY}
sources:
openstack/keystone:
git_url: https://github.com/openstack/keystone.git
git_ref: ${GIT_BRANCH}
openstack/horizon:
git_url: https://github.com/openstack/horizon.git
git_ref: ${GIT_BRANCH}
openstack/nova:
git_url: https://github.com/openstack/nova.git
git_ref: ${GIT_BRANCH}
openstack/neutron:
git_url: https://github.com/openstack/neutron.git
git_ref: ${GIT_BRANCH}
openstack/heat:
git_url: https://github.com/openstack/heat.git
git_ref: ${GIT_BRANCH}
openstack/keystone:
git_url: https://github.com/openstack/keystone.git
git_ref: ${GIT_BRANCH}
openstack/glance:
git_url: https://github.com/openstack/glance.git
git_ref: ${GIT_BRANCH}
openstack/horizon:
git_url: https://github.com/openstack/horizon.git
git_ref: ${GIT_BRANCH}
EOF
ccp --config-file ccp-test.yaml build
deactivate

37
jenkins-tp/jobs/builders/mcp-clean-docker-registry.sh
View File

@ -1,37 +0,0 @@
#!/bin/bash -ex
# Removes unwanted tags from Docker registry.
# This scripts expects that authenitcation to Docker registry is configured
# beforehand (e.g. by running docker login).
# (c) mzawadzki@mirantis.com
# CONFIGURATION:
######################################################
: ${DOCKER_NAMESPACE:="mcp"}
: ${DOCKER_REGISTRY:="registry.mcp.fuel-infra.org"}
REGISTRY_MANAGER="registry-manage --host ${DOCKER_REGISTRY}"
TAG_REGEXP=${PARAM_TAG_REGEXP}
# List all tags for all images, select and delete tags matching TAG_REGEXP:
#
# Sample output from registry-manage tool:
# "list" subcommand:
# mcp/base
# mcp/base-tools
# mcp/cron
# mcp/elasticsearch
# mcp/etcd
# "list-tags mcp/base" subcommand:
# 21
# 22
# latest
# 16
# new
for image in `${REGISTRY_MANAGER} list | grep "^${DOCKER_NAMESPACE}/"`; do
${REGISTRY_MANAGER} list-tags ${image} | \
grep "${TAG_REGEXP}" | \
awk -v REGISTRY_MANAGER="${REGISTRY_MANAGER}" -v image="${image}" \
-v TAG_REGEXP="${TAG_REGEXP}" \
'$1 ~ TAG_REGEXP {exit system(REGISTRY_MANAGER" delete "image":"$1);}'
done

10
jenkins-tp/jobs/builders/ng-build-specs.sh
View File

@ -1,10 +0,0 @@
#!/bin/bash
set -ex
echo "** Building docs from rst files:"
tox -e docs
ls -al doc/build/html/
echo "** Uploading HTML docs to the server:"
rsync -v -r --delete -e ssh doc/build/html docs-publisher@docs.ng.mirantis.net:~/

95
jenkins-tp/jobs/builders/prepare_env.sh
View File

@ -1,95 +0,0 @@
#!/bin/bash
set -ex
export PATH=/bin:/usr/bin:/sbin:/usr/sbin:${PATH}
ACT=0
function update_devops {
ACT=1
VIRTUAL_ENV=/home/jenkins/venv-fuel-devops${1}
REPO_NAME=${2}
BRANCH=${3}
if [[ -d "${VIRTUAL_ENV}" ]] && [[ "${FORCE_DELETE_DEVOPS}" == "true" ]]; then
echo "Delete venv from ${VIRTUAL_ENV}"
rm -rf ${VIRTUAL_ENV}
fi
if [ -f ${VIRTUAL_ENV}/bin/activate ]; then
source ${VIRTUAL_ENV}/bin/activate
echo "Python virtual env exist"
else
rm -rf ${VIRTUAL_ENV}
virtualenv --no-site-packages ${VIRTUAL_ENV}
source ${VIRTUAL_ENV}/bin/activate
fi
#
# fuel-devops use ~/.devops directory to store log configuration
# we need to delete log.yaml befeore update to get it in current
# version
#
test -f ~/.devops/log.yaml && rm ~/.devops/log.yaml
# Upgrade pip inside virtualenv
pip install pip --upgrade
if [[ -n "${VENV_REQUIREMENTS}" ]]; then
echo "Install with custom requirements"
echo "${VENV_REQUIREMENTS}" >"${WORKSPACE}/venv-requirements.txt"
pip install -r "${WORKSPACE}/venv-requirements.txt" --upgrade
fi
pip install git+https://git.openstack.org/openstack/fuel-devops@3.0.1 --upgrade
echo "=============================="
pip freeze
echo "=============================="
django-admin.py syncdb --settings=devops.settings --noinput
django-admin.py migrate devops --settings=devops.settings --noinput
deactivate
}
function download_images {
ACT=1
TARGET_CLOUD_DIR=/home/jenkins/workspace/cloud-images
VM_IMAGE="http://share01-scc.ng.mirantis.net/packer-ubuntu-1604-server-new.qcow2"
mkdir -p ${TARGET_CLOUD_DIR}
pushd ${TARGET_CLOUD_DIR}
wget ${VM_IMAGE}
popd
}
function install_zuul_env {
ACT=1
VIRTUAL_ENV="/home/jenkins/venv-zuul"
virtualenv "${VIRTUAL_ENV}"
source "${VIRTUAL_ENV}"/bin/activate
# Upgrade pip inside virtualenv
pip install pip --upgrade
pip install zuul
}
# DevOps 3.0.x
if [[ ${update_devops_3_0_x} == "true" ]]; then
update_devops "-3.0" "fuel-ccp-tests" "master"
fi
if [[ ${download_images} == "true" ]]; then
download_images
fi
if [[ "${install_zuul}" == "true" ]]; then
install_zuul_env
fi
if [ ${ACT} -eq 0 ]; then
echo "No action selected!"
exit 1
fi

14
jenkins-tp/jobs/builders/remove-simple-deployment-envs.sh
View File

@ -1,14 +0,0 @@
#!/bin/bash -ex
# Removes all fuel-devops deployment envs.
# CONFIGURATION:
######################################################
FUEL_DEVOPS_INSTALLATION_DIR="/home/jenkins/venv-fuel-devops-3.0"
source "${FUEL_DEVOPS_INSTALLATION_DIR}"/bin/activate
for f in `dos.py list | grep deployment`; do
dos.py erase "${f}";
done

20
jenkins-tp/jobs/builders/update-jenkins-post-job.sh
View File

@ -1,20 +0,0 @@
#!/bin/bash -xe
echo "This is job for update jenkins jobs after merge"
git clone https://git.openstack.org/$ZUUL_PROJECT .
cat > jenkins_jobs.ini << EOF
[jenkins]
user=${JJB_USER}
password=${JJB_PASS}
url=https://jenkins-tp.ng.mirantis.net/
query_plugins_info=False
[job_builder]
ignore_cache=True
recursive=True
EOF
tox -e venv -- jenkins-jobs --flush-cache --conf jenkins_jobs.ini update \
--delete-old jenkins-tp/jobs

13
jenkins-tp/jobs/builders/update-vm-ubuntu-image.sh
View File

@ -1,13 +0,0 @@
#!/bin/bash
set -ex
mkdir -p $IMAGE_SLAVE_PATH
echo "** Updating Ubuntu image"
rsync --delete-before -v -r -e 'ssh -o StrictHostKeyChecking=no' $STORAGE_USER@$STORAGE_HOST:. $IMAGE_SLAVE_PATH
cd $IMAGE_SLAVE_PATH
lastfile=`ls -t1 *.qcow2 | head -n 1`
ln -sf $lastfile default-image.qcow2

48
jenkins-tp/jobs/fuel-ccp-debug-deploy.yaml
View File

@ -1,48 +0,0 @@
- job-template:
name: fuel-ccp-debug-{version}-deployment
description: |
Deploy CCP and run basic OpenStack tests (uses snapshot for K8s cluster)
node: standard
builders:
- inject:
properties-content: |
COMPONENT=full
VERSION={version}
- shell:
!include-raw-escape 'builders/fuel-ccp-debug-deployment.sh'
concurrent: true
properties:
- heavy-job:
weight: 2
- throttle:
option: project
max-per-node: 1
- least-load:
disabled: False
scm:
- openstack:
scm-basedir: 'fuel-ccp'
scm-branch: 'master'
scm-repo: 'fuel-ccp'
- openstack:
scm-basedir: 'fuel-ccp-installer'
scm-branch: 'master'
scm-repo: 'fuel-ccp-installer'
wrappers:
- ng-cleanup
- fuel-ccp-ci-jenkins
- timeout:
timeout: 120
fail: true
parameters:
- node:
name: NODE
allowed-multiselect: true
- project:
name: Deploy CCP in debug mode without remove snapshots and with selected node
version:
- master
- newton
jobs:
- 'fuel-ccp-debug-{version}-deployment'

50
jenkins-tp/jobs/fuel-ccp-docs-build-jobs.yaml
View File

@ -1,50 +0,0 @@
#
# Template for job executed manually
#
- job-template:
name: build-all-docs-manual-job-{repo}
description: |
Docs build job executed manually
node: docs
builders:
- shell:
!include-raw 'builders/ng-build-specs.sh'
concurrent: true
wrappers:
- ng-cleanup
- fuel-ccp-ci-docs
scm:
- fuel-infra:
scm-basedir: ''
scm-branch: '{branch}'
scm-repo: 'nextgen/{repo}'
#
# Template for triggered job
#
- job-template:
name: build-all-docs-triggered-job-{repo}
description: |
Docs build job triggered
node: docs
builders:
- shell: |
git clone https://git.openstack.org/openstack/$ZUUL_PROJECT .
git fetch $ZUUL_URL/$ZUUL_PROJECT $ZUUL_REF
git checkout FETCH_HEAD
- shell:
!include-raw 'builders/ng-build-specs.sh'
concurrent: true
wrappers:
- ng-cleanup
- fuel-ccp-ci-docs
triggers:
- zuul
- project:
name: fuel-ccp-docs-build-jobs
jobs:
- 'build-all-docs-manual-job-{repo}'
- 'build-all-docs-triggered-job-{repo}'
repo:
- 'fuel-ccp-specs':
branch: 'master'

63
jenkins-tp/jobs/fuel-ccp-gate-jobs.yaml
View File

@ -1,63 +0,0 @@
- job:
name: fuel-ccp-ms-ext-config-gate-job
description: |
Some gate fuel-ccp-ms-ext-config test job
node: standard
builders:
- get-from-zuul
- shell: !include-raw builders/fuel-ccp-ext-config-gate.sh
concurrent: true
wrappers:
- ng-cleanup
triggers:
- zuul
- job:
name: fuel-ccp-installer
description: |
Fuel CCP Installer gate job
node: standard
builders:
- get-from-zuul
- shell: !include-raw builders/fuel-ccp-installer-gate.sh
concurrent: true
wrappers:
- ng-cleanup
triggers:
- zuul
publishers:
- archive:
allow-empty: true
artifacts: '*'
latest-only: false
- post-tasks:
- matches:
- log-text: ''
escalate-status: true
run-if-job-successful: false
script: |
#!/bin/bash
set -ex
export ENV_NAME="env-k8s-kargo-$BUILD_TAG"
source /home/jenkins/venv-fuel-devops-3.0/bin/activate
echo "Running on $NODE_NAME: $ENV_NAME"
dos.py erase $ENV_NAME || true
deactivate
- job:
name: fuel-ccp-mcp-qa
description: |
Gate mcp-qa test job
node: standard
builders:
- get-from-zuul
- shell: !include-raw builders/fuel-ccp-qa-gate.sh
concurrent: true
wrappers:
- ng-cleanup
triggers:
- zuul

14
jenkins-tp/jobs/fuel-ccp-post-jobs.yaml
View File

@ -1,14 +0,0 @@
- job:
name: update-jenkins-jobs
description: |
Update jenkins jobs after merge
node: standard
builders:
- shell: !include-raw builders/update-jenkins-post-job.sh
concurrent: true
wrappers:
- ng-cleanup
- inject-passwords:
global: true
triggers:
- zuul

90
jenkins-tp/jobs/fuel-ccp-simple-deployment.yaml
View File

@ -1,90 +0,0 @@
- job-template:
name: fuel-ccp-{component}-{version}-deployment
description: |
Deploy CCP and run basic OpenStack tests (uses snapshot for K8s cluster)
node: standard
builders:
- zuul-clone
- inject:
properties-content: |
COMPONENT={component}
VERSION={version}
- shell:
!include-raw-escape 'builders/fuel-ccp-simple-deployment.sh'
publishers:
- junit:
results: result-{version}.xml
allow-empty-results: true
- post-tasks:
- matches:
- log-text: ''
escalate-status: false
run-if-job-successful: false
script: |
#!/bin/bash
set -ex
ENV_NAME="fuel-ccp-{component}-{version}-deployment"
FUEL_DEVOPS_SNAPSHOT_NAME="fresh"
FUEL_DEVOPS_INSTALLATION_DIR="/home/jenkins/venv-fuel-devops-3.0"
source "$FUEL_DEVOPS_INSTALLATION_DIR"/bin/activate
echo "Running on $NODE_NAME: $ENV_NAME"
virsh list --all
dos.py revert "$ENV_NAME" "$FUEL_DEVOPS_SNAPSHOT_NAME"
dos.py destroy "$ENV_NAME"
deactivate
concurrent: true
properties:
- heavy-job:
weight: 3
- throttle:
option: project
max-per-node: 1
- least-load:
disabled: False
scm:
- openstack:
scm-basedir: 'fuel-ccp'
scm-branch: 'master'
scm-repo: 'fuel-ccp'
- openstack:
scm-basedir: 'fuel-ccp-installer'
scm-branch: 'master'
scm-repo: 'fuel-ccp-installer'
wrappers:
- ng-cleanup
- fuel-ccp-ci-jenkins
- timeout:
timeout: 120
fail: true
- project:
name: Deploy CCP and run basic OpenStack tests (uses snapshot for K8s cluster)
version:
- master
- newton
- mitaka
- ocata
component:
- smoke
- full
- debian-base
- etcd
- mariadb
- memcached
- rabbitmq
- stacklight
- cinder
- galera
- glance
- heat
- horizon
- ironic
- keystone
- murano
- neutron
- nova
- openstack-base
- sahara
jobs:
- 'fuel-ccp-{component}-{version}-deployment'

151
jenkins-tp/jobs/macros.yaml
View File

@ -1,151 +0,0 @@
#
# Configurable repository sources
#
- scm:
name: mcpinstaller
scm:
- git:
url: 'ssh://nextgen-ci@review.fuel-infra.org:29418/{scm-repo}'
branches:
- '{scm-branch}'
basedir: '{scm-basedir}'
- scm:
name: openstack
scm:
- git:
url: 'https://git.openstack.org/openstack/{scm-repo}.git'
branches:
- '{scm-branch}'
basedir: '{scm-basedir}'
- scm:
name: fuel-infra
scm:
- git:
url: 'ssh://nextgen-ci@review.fuel-infra.org:29418/{scm-repo}.git'
basedir: '{scm-basedir}'
skip-tag: true
branches:
- '{scm-branch}'
- scm:
name: zuul_repo
scm:
- git:
url: $ZUUL_URL/$ZUUL_PROJECT
branches:
- $ZUUL_BRANCH
#
# Repository source with gerrit changes
#
- scm:
name: zuul_repo_trigger
scm:
- git:
basedir: '{scm-basedir}'
branches:
- $ZUUL_BRANCH
remotes:
- gerrit:
refspec: $ZUUL_REF
url: $ZUUL_URL/$ZUUL_PROJECT
#
# Static repositories
#
#
# SSH credentials
#
- wrapper:
name: fuel-ccp-ci-jenkins
wrappers:
- ssh-agent-credentials:
users:
- '623272f1-bc9a-4c75-93f3-b7ec6f242581'
- wrapper:
name: fuel-ccp-ci-docs
wrappers:
- ssh-agent-credentials:
users:
- '04a7dbe6-f3cd-4cc1-afc5-b8f2bd6cbb38'
- wrapper:
name: 'ng-cleanup'
wrappers:
- workspace-cleanup
#
# Publishers
#
# TODO: check target fuel-ccp-ci
- publisher:
name: logs-public
publishers:
- ssh:
site: 'ci-logs.fuel-infra.org'
target: 'fuel-ccp-ci/$JOB_NAME/$BUILD_ID'
source: 'logs/*'
flatten: true
- publisher:
name: logs-private
publishers:
- archive:
allow-empty: true
artifacts: 'logs/*'
latest-only: false
- publisher:
name: email-image-build
publishers:
- email-ext:
body: '{body-failed}'
aborted: true
failure: true
still-failing: true
fixed: false
send-to:
- recipients
recipients: '{mail-to}'
subject: '[Jenkins] $JOB_NAME - #$BUILD_ID'
#
# Builders
#
- builder:
name: check-tox
builders:
- shell: 'tox -v -e {tox-test}'
- builder:
name: build-image-branch
builders:
- shell: !include-raw: builders/build-image-branch.sh
- builder:
name: zuul-clone
builders:
- shell: !include-raw: builders/clone-with-zuul.sh
- builder:
name: get-from-zuul-container
builders:
- shell: |
git clone https://git.openstack.org/openstack/fuel-ccp
mkdir -p containers/${ZUUL_PROJECT/fuel-ccp-/}
cd containers/${ZUUL_PROJECT/fuel-ccp-/}
git clone https://git.openstack.org/$ZUUL_PROJECT .
git fetch $ZUUL_URL/$ZUUL_PROJECT $ZUUL_REF
git checkout FETCH_HEAD
- builder:
name: get-from-zuul
builders:
- shell: |
git clone https://git.openstack.org/$ZUUL_PROJECT .
git fetch $ZUUL_URL/$ZUUL_PROJECT $ZUUL_REF
git checkout FETCH_HEAD
git status

60
jenkins-tp/jobs/mcp-build-images-smoke.yaml
View File

@ -1,60 +0,0 @@
- job-template:
name: 'build-{version}-images-smoke'
builders:
- inject:
properties-content: |
IMAGE_PATH=/home/jenkins/workspace/cloud-images/default-image.qcow2
OS_VER={version}
- shell:
!include-raw-escape builders/mcp-build-images-smoke.sh
concurrent: false
node: standard
description: |
Run tests.
<p>
Job for build images and run deploy with them.<BR/>
It's based on QA Team smoke test job.<BR/>
</p>
logrotate:
daysToKeep: 30
numToKeep: 50
scm:
- openstack:
scm-branch: 'master'
scm-repo: 'fuel-ccp-tests'
scm-basedir: 'fuel-ccp-tests'
- openstack:
scm-branch: 'master'
scm-repo: 'fuel-ccp-installer'
scm-basedir: 'fuel-ccp-installer'
wrappers:
- timeout:
fail: true
timeout: 235
publishers:
- post-tasks:
- matches:
- log-text: ''
escalate-status: true
run-if-job-successful: false
script: |
#!/bin/bash
set -xe
export ENV_PREFIX="fuel-ccp-tests"
export ENV_NAME=$ENV_PREFIX.$BUILD_NUMBER
source /home/jenkins/venv-fuel-devops-3.0/bin/activate
dos.py erase $ENV_NAME
deactivate
- project:
name: smoke-build
version:
- newton
- ocata
jobs:
- build-{version}-images-smoke

74
jenkins-tp/jobs/mcp-build-images.yaml
View File

@ -1,74 +0,0 @@
- job-template:
name: mcp-build-images-{version}
description: |
Build MCP Docker images and store them into Docker registry with latest tag.
node: standard
builders:
- inject:
properties-content: |
TAG={version}
- shell:
!include-raw-escape 'builders/mcp-build-images.sh'
concurrent: false
scm:
- openstack:
scm-basedir: ''
scm-branch: 'master'
scm-repo: 'fuel-ccp'
wrappers:
- ng-cleanup
- job-template:
name: mcp-build-images-build-number
description: |
Build MCP Docker images and store them into Docker registry with BUILD_NUMBER as a tag.
node: standard
builders:
- inject:
properties-content: |
TAG=$BUILD_NUMBER
- shell:
!include-raw-escape 'builders/mcp-build-images.sh'
concurrent: true
scm:
- openstack:
scm-basedir: ''
scm-branch: 'master'
scm-repo: 'fuel-ccp'
wrappers:
- ng-cleanup
publishers:
- email-image-build:
body-failed:
!include-raw notifications/image_build.failed
mail-to: mos-microservices@mirantis.com
- job-template:
name: mcp-clean-docker-registry
description: |
Manual job for removing selected tags from MCP Docker registry.
node: standard
parameters:
- text:
name: PARAM_TAG_REGEXP
description: "Tag (regexp) to delete for all images in mcp namespace"
builders:
- inject:
properties-content: |
DOCKER_REGISTRY=registry.mcp.fuel-infra.org
DOCKER_NAMESPACE=mcp
- shell:
!include-raw-escape 'builders/mcp-clean-docker-registry.sh'
concurrent: false
wrappers:
- ng-cleanup
- project:
name: Build MCP Docker images and store them into Docker registry.
jobs:
- 'mcp-build-images-{version}':
version:
- ocata
- newton
- 'mcp-build-images-build-number'
- 'mcp-clean-docker-registry'

5
jenkins-tp/jobs/notifications/image_build.failed
View File

@ -1,5 +0,0 @@
Image build job failed: $BUILD_URL
Please raise this issue on MCP Slack channel (#mcp-engineering) to check if
someone is fixing this already, or start fixing it yourself.

35
jenkins-tp/jobs/prepare_env.yaml
View File

@ -1,35 +0,0 @@
- job:
builders:
- shell:
!include-raw 'builders/prepare_env.sh'
concurrent: true
description: |
Creates a python virtual environment for system tests. Stores additional images.<br><br>
For current and above environments choose 3.0.x DevOps version from fuel-qa repository.
logrotate:
daysToKeep: 14
name: prepare_env
parameters:
- bool:
name: 'update_devops_3_0_x'
default: false
- bool:
name: 'download_images'
default: false
- bool:
name: 'install_zuul'
default: false
- bool:
name: FORCE_DELETE_DEVOPS
default: false
description: 'Force to delete devops venv before update'
- node:
name: NODE
allowed-multiselect: true
- text:
name: VENV_REQUIREMENTS
default: 'psycopg2'
description: 'Use custom requirements for venv'
wrappers:
- timeout:
timeout: 60

169
jenkins-tp/jobs/projects.yaml
View File

@ -1,169 +0,0 @@
- project:
name: cinder
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-ceph fuel-ccp-cinder
- project:
name: debian-base
jobs:
- test-build-{name}:
repos: fuel-ccp-debian-base
- project:
name: etcd
jobs:
- test-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-etcd
- project:
name: galera
jobs:
- test-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-galera
tox-test: pep8
- project:
name: glance
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-ceph fuel-ccp-glance
- project:
name: grafana
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-grafana
- project:
name: heat
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-heat
- project:
name: horizon
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-horizon
- project:
name: ironic
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-ironic
- project:
name: keystone
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-keystone
- project:
name: memcached
jobs:
- test-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-memcached
tox-test: py27
- project:
name: mariadb
jobs:
- test-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-mariadb
tox-test: py27
- project:
name: murano
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-murano
- project:
name: neutron
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-neutron
- project:
name: nova
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-ceph fuel-ccp-nova
- project:
name: openstack-base
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base
- project:
name: rabbitmq
jobs:
- test-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-rabbitmq
- project:
name: sahara
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-sahara
- project:
name: searchlight
os_version:
- master
- newton
jobs:
- test-{os_version}-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-openstack-base fuel-ccp-searchlight
- project:
name: stacklight
jobs:
- test-build-{name}:
repos: fuel-ccp-debian-base fuel-ccp-stacklight
- project:
name: fuelccp
os_version:
- master
- newton
jobs:
- test-{os_version}-build-all

16
jenkins-tp/jobs/remove-all-envs.yaml
View File

@ -1,16 +0,0 @@
- job:
name: remove-all-simple-deployment-envs
description: |
This job will trigger jobs for removing all simple deployment envs on all slaves.
builders:
- shell: |
echo "Removing simple deployment envs from all slaves"
- trigger-builds:
- project: "remove-simple-deployment-envs"
parameter-factories:
- factory: allnodesforlabel
name: NODE
node-label: standard
concurrent: true
wrappers:
- ng-cleanup

24
jenkins-tp/jobs/remove-envs.yaml
View File

@ -1,24 +0,0 @@
- job:
name: remove-simple-deployment-envs
builders:
- inject:
properties-content: |
FUEL_DEVOPS_ENV_NAME=ccp-simple-deployment-env
FUEL_DEVOPS_SNAPSHOT_NAME=fresh
- shell:
!include-raw 'builders/remove-simple-deployment-envs.sh'
concurrent: true
properties:
- heavy-job:
weight: 6
description: |
Removes envs for simple deployment tests.
logrotate:
daysToKeep: 14
parameters:
- node:
name: NODE
allowed-multiselect: true
wrappers:
- timeout:
timeout: 60

30
jenkins-tp/jobs/set-default-vm-image.yaml
View File

@ -1,30 +0,0 @@
- job:
name: set-default-vm-image
builders:
- shell: |
#!/bin/bash -ex
cd /home/jenkins/workspace/cloud-images/
if [ -L default-image.qcow2 ]; then
rm default-image.qcow2
fi
ln -s $IMAGE default-image.qcow2
concurrent: true
description: |
Job set default Ubuntu images on Jenkins slaves
logrotate:
daysToKeep: 14
parameters:
- extended-choice:
name: IMAGE
description: "VM image file for k8s hosts"
property-file: http://share01-scc.ng.mirantis.net/images.lst
property-key: images
- node:
name: NODE
allowed-multiselect: true
wrappers:
- ssh-agent-credentials:
users:
- '0859086d-e10a-4ad5-af71-8abec4b6b6ec'
- timeout:
timeout: 60

21
jenkins-tp/jobs/tox-job-template.yaml
View File

@ -1,21 +0,0 @@
#
# Template used to execute tox tests
#
- job-template:
name: verify-tox-{tox-test}
builders:
- shell: |
env | grep ZUUL
git clone https://git.openstack.org/$ZUUL_PROJECT .
git fetch $ZUUL_URL/$ZUUL_PROJECT $ZUUL_REF
git checkout FETCH_HEAD
- check-tox:
tox-test: '{tox-test}'
concurrent: true
description: |
Tox {tox-test} syntax test
node: standard
wrappers:
- ng-cleanup
triggers:
- zuul

17
jenkins-tp/jobs/update-all-slaves.yaml
View File

@ -1,17 +0,0 @@
- job:
name: update-all-slaves
description: |
This job will trigger jobs for update all slave
builders:
- shell: |
echo "Update slaves"
- trigger-builds:
- project: "update-vm-ubuntu-image"
parameter-factories:
- factory: allnodesforlabel
name: NODE
node-label: standard
concurrent: true
wrappers:
- ng-cleanup
auth-token: 42e7ac191

29
jenkins-tp/jobs/update-vm-ubuntu-image.yaml
View File

@ -1,29 +0,0 @@
- job:
name: update-vm-ubuntu-image
builders:
- inject:
properties-content: |
STORAGE_USER=share
STORAGE_HOST=static01-scc.mcp.mirantis.net
IMAGE_SLAVE_PATH=/home/jenkins/workspace/cloud-images/
- shell:
!include-raw 'builders/update-vm-ubuntu-image.sh'
- trigger-builds:
- project: "remove-simple-deployment-envs"
same-node: true
current-parameters: true
concurrent: true
description: |
Job updates Ubuntu images on Jenkins slaves
logrotate:
daysToKeep: 14
parameters:
- node:
name: NODE
allowed-multiselect: true
wrappers:
- ssh-agent-credentials:
users:
- '0859086d-e10a-4ad5-af71-8abec4b6b6ec'
- timeout:
timeout: 60

45
jenkins-tp/jobs/verify-ci.yaml
View File

@ -1,45 +0,0 @@
#
# Jobs used to verify syntax in CI repository
#
# Shell
- job-template:
name: verify-bashate-{repo}
builders:
- get-from-zuul
- check-tox:
tox-test: bashate
concurrent: true
description: |
Bashate syntax test
node: standard
wrappers:
- ng-cleanup
# JJB
- job-template:
name: verify-jjb-{repo}
builders:
- get-from-zuul
- check-tox:
tox-test: fuel-ccp-ci
concurrent: true
description: |
JJB syntax test
node: standard
wrappers:
- ng-cleanup
- project:
name: verify-ci-bashate
jobs:
- 'verify-bashate-{repo}'
repo:
- 'project-config'
- project:
name: verify-ci-jjb
jobs:
- 'verify-jjb-{repo}'
repo:
- 'project-config'

9
jenkins-tp/jobs/verify-tox.yaml
View File

@ -1,9 +0,0 @@
- project:
name: verify-tox-tests
jobs:
- 'verify-tox-{tox-test}':
tox-test:
- pep8
- py27
- bashate
- py34

1
requirements.txt
View File

@ -1 +0,0 @@
jenkins-job-builder==1.6.1

2
test-requirements.txt
View File

@ -1,2 +0,0 @@
bashate>=0.2
yamllint

5
tools/yamllint.sh
View File

@ -1,5 +0,0 @@
#!/bin/bash
set -ex
workdir=$(dirname $0)
yamllint -c $workdir/yamllint.yaml $(find . -not -path '*/\.*' -type f -name '*.yaml')

21
tools/yamllint.yaml
View File

@ -1,21 +0,0 @@
extends: default
rules:
braces:
max-spaces-inside: 1
comments:
level: error
comments-indentation:
level: warning
document-end:
present: no
document-start:
level: error
present: no
empty-lines:
max: 1
max-start: 0
max-end: 0
line-length:
level: warning
max: 120

25
tox.ini
View File

@ -1,25 +0,0 @@
[tox]
minversion = 1.6
skipsdist = True
envlist = bashate,fuel-ccp-ci
[testenv]
deps =
-r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
usedevelop = False
[testenv:linters]
whitelist_externals =
bash
commands =
{toxinidir}/tools/yamllint.sh
# bashate check
bash -c "find {toxinidir} -type f -name '*.sh' -not -path '*/.tox/*' -print0 | xargs -0 bashate -v -iE006"
[testenv:venv]
commands = {posargs}
[testenv:fuel-ccp-ci]
commands =
jenkins-jobs --conf conf/jenkins_job.ini.example test jenkins-tp/jobs -o {toxinidir}/output/{envname}

170
zuul-tp/layout.yaml
View File

@ -1,170 +0,0 @@
pipelines:
- name: check
description: Newly uploaded patchsets enter this pipeline to receive an initial +/-1 Verified vote from Jenkins.
success-message: Build succeeded (check pipeline).
failure-message: Build failed (check pipeline). For information on how to proceed, see http://docs.openstack.org/infra/manual/developers.html#automated-testing
manager: IndependentPipelineManager
source: gerrit
precedence: normal
require:
open: True
current-patchset: True
trigger:
gerrit:
- event: patchset-created
- event: change-restored
- event: comment-added
comment: (?i)^(Patch Set [0-9]+:)?( [\w\\+-]*)*(\n\n)?\s*(recheck|reverify|ccp-recheck)
success:
gerrit:
verified: 1
failure:
gerrit:
verified: -1
- name: post
description: This pipeline runs jobs that operate after each change is merged.
manager: IndependentPipelineManager
source: gerrit
precedence: low
trigger:
gerrit:
- event: ref-updated
ref: ^(?!refs/).*$
- name: periodic
description: Jobs in this queue are triggered on a timer.
manager: IndependentPipelineManager
source: gerrit
precedence: low
trigger:
timer:
- time: '5 10,14,18 * * 1-5'
project-templates:
- name: verify-tox
check:
- 'verify-tox-pep8'
- 'verify-tox-py27'
projects:
- name: openstack-dev/ci-sandbox
check:
- verify-tox-py27
- verify-tox-pep8
- name: openstack/fuel-ccp-nextgen-specs
check:
- verify-tox-pep8
- name: openstack/fuel-ccp
check:
- test-master-build-all
- test-newton-build-all
- name: openstack/fuel-ccp-cinder
check:
- test-master-build-cinder
- test-newton-build-cinder
- name: openstack/fuel-ccp-debian-base
check:
- test-build-debian-base
- name: openstack/fuel-ccp-etcd
check:
- test-build-etcd
- name: openstack/fuel-ccp-galera
check:
- test-build-galera
- name: openstack/fuel-ccp-glance
check:
- test-master-build-glance
- test-newton-build-glance
- name: openstack/fuel-ccp-grafana
check:
- test-master-build-grafana
- test-newton-build-grafana
- name: openstack/fuel-ccp-heat
check:
- test-newton-build-heat
- test-master-build-heat
- name: openstack/fuel-ccp-horizon
check:
- test-newton-build-horizon
- test-master-build-horizon
- name: openstack/fuel-ccp-ironic
check:
- test-newton-build-ironic
- test-master-build-ironic
- name: openstack/fuel-ccp-keystone
check:
- test-newton-build-keystone
- test-master-build-keystone
- name: openstack/fuel-ccp-mariadb
check:
- test-build-mariadb
- name: openstack/fuel-ccp-memcached
check:
- test-build-memcached
- name: openstack/fuel-ccp-murano
check:
- test-newton-build-murano
- test-master-build-murano
- name: openstack/fuel-ccp-neutron
check:
- test-newton-build-neutron
- test-master-build-neutron
- name: openstack/fuel-ccp-nova
check:
- test-newton-build-nova
- test-master-build-nova
- name: openstack/fuel-ccp-openstack-base
check:
- test-newton-build-openstack-base
- test-master-build-openstack-base
- name: openstack/fuel-ccp-rabbitmq
check:
- test-build-rabbitmq
- name: openstack/fuel-ccp-sahara
check:
- test-newton-build-sahara
- test-master-build-sahara
- name: openstack/fuel-ccp-searchlight
check:
- test-newton-build-searchlight
- test-master-build-searchlight
- name: openstack/fuel-ccp-stacklight
check:
- test-build-stacklight
- name: openstack/fuel-ccp-installer
check:
- fuel-ccp-installer
- name: openstack/fuel-ccp-tests
check:
- fuel-ccp-mcp-qa
- name: openstack/fuel-ccp-ci-config
check:
- verify-jjb-project-config
post:
- update-jenkins-jobs