Manage a pool of nodes for a distributed test infrastructure

Go to file

James E. Blair b48e8ad4ec Fix test_node_assignment_at_quota There is a bug in the request handler at quota where if: the request handler runs but must pause due to quota while still needing more than one node, and then a single node becomes available and the handler runs again and causes a node to be launched but then must wait for another node to become available, the handler will never unpause. This is because nodes that it launches are not added to the handler's nodeset until after the entire request is handled (they are added by the poll method). However, nodes that are allocated to the request from ready node stock are added to the nodeset. The current nodeset is used to determine whether more nodes are needed. Because the nodes from the recent launches are not part of the nodeset, they are still counted as being "needed", and so the request handler continues to wait for more slots to become available. The fix is to add the newly requested node to the node set immediately when it is requested rather than when it becomes READY in the poll() method. This should be safe since any node failures causes the entire request to be failed. Co-Authored-By: David Shrewsbury <shrewsbury.dave@gmail.com> Change-Id: I88c682807b395fc549f7c698d0c42c888dab2bc2		2017-03-21 12:55:47 +00:00
devstack	Merge "Remove MySQL" into feature/zuulv3	2017-03-13 19:31:09 +00:00
doc	Stop writing nodepool bash variable on nodes	2017-03-14 12:40:44 -04:00
nodepool	Fix test_node_assignment_at_quota	2017-03-21 12:55:47 +00:00
tools	Remove ready-script support	2017-03-14 12:34:56 -04:00
.gitignore	Add files for zuul-nodepool integration test	2017-01-24 09:46:08 -08:00
.gitreview	Set default branch to feature/zuulv3	2016-08-18 09:08:23 -07:00
.testr.conf	Add a test for subnodes	2014-03-31 09:22:00 -07:00
bindep.txt	Add libffi development headers to bindep	2017-03-21 08:53:12 -04:00
LICENSE	Add LICENSE file	2015-12-10 12:54:23 -05:00
README.rst	Remove MySQL	2017-03-13 13:42:43 -04:00
requirements.txt	Remove MySQL	2017-03-13 13:42:43 -04:00
setup.cfg	Builders can be run on their own	2016-01-12 15:33:01 -08:00
setup.py	Bump pbr requirements to >=1.3	2015-09-14 16:19:13 -04:00
test-requirements.txt	Merge "Remove discover from test-requirements"	2016-11-16 14:33:35 +00:00
tox.ini	Initial support for bindep	2016-11-25 09:51:26 +00:00

README.rst

Nodepool

Nodepool is a service used by the OpenStack CI team to deploy and manage a pool of devstack images on a cloud server for use in OpenStack project testing.

Developer setup

Make sure you have pip installed:

wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py

Install dependencies:

sudo pip install bindep
sudo apt-get install $(bindep -b nodepool)

mkdir src
cd ~/src
git clone git://git.openstack.org/openstack-infra/system-config
git clone git://git.openstack.org/openstack-infra/nodepool
cd nodepool
sudo pip install -U -r requirements.txt
sudo pip install -e .

If you're testing a specific patch that is already in gerrit, you will also want to install git-review and apply that patch while in the nodepool directory, ie:

git review -x XXXXX

Create or adapt a nodepool yaml file. You can adapt an infra/system-config one, or fake.yaml as desired. Note that fake.yaml's settings won't Just Work - consult ./modules/openstack_project/templates/nodepool/nodepool.yaml.erb in the infra/system-config tree to see a production config.

If the cloud being used has no default_floating_pool defined in nova.conf, you will need to define a pool name using the nodepool yaml file to use floating ips.

Export variable for your ssh key so you can log into the created instances:

export NODEPOOL_SSH_KEY=`cat ~/.ssh/id_rsa.pub | awk '{print $2}'`

Start nodepool with a demo config file (copy or edit fake.yaml to contain your data):

export STATSD_HOST=127.0.0.1
export STATSD_PORT=8125
nodepoold -d -c tools/fake.yaml

All logging ends up in stdout.

Use the following tool to check on progress:

nodepool image-list