d5b0dee642
The default AWS rate limit is 2 instances/sec, but in practice, we can achieve something like 0.6 instances/sec with the current code. That's because the create instance REST API call itself takes more than a second to return. To achieve even the default AWS rate (much less a potentially faster one which may be obtainable via support request), we need to alter the approach. This change does the following: * Paralellizes create API calls. We create a threadpool with (typically) 8 workers to execute create instance calls in the background. 2 or 3 workers should be sufficient to meet the 2/sec rate, more allows for the occasional longer execution time as well as a customized higher rate. We max out at 8 to protect nodepool from too many threads. * The state machine uses the new background create calls instead of synchronously creating instances. This allows other state machines to progress further (ie, advance to ssh keyscan faster in the case of a rush of requests). * Delete calls are batched. They don't take as long as create calls, yet their existence at all uses up rate limiting slots which could be used for creating instances. By batching deletes, we make more room for creates. * A bug in the RateLimiter could cause it not to record the initial time and therefore avoid actually rate limiting. This is fixed. * The RateLimiter is now thread-safe. * The default rate limit for AWS is changed to 2 requests/sec. * Documentation for the 'rate' parameter for the AWS driver is added. * Documentation for the 'rate' parameter for the Azure driver is corrected to describe the rate as requests/sec instead of delay between requests. Change-Id: Ida2cbc59928e183eb7da275ff26d152eae784cfe
Nodepool
Nodepool is a system for managing test node resources. It supports launching single-use test nodes from cloud providers as well as managing access to pre-defined pre-existing nodes. Nodepool is part of a suite of tools that form a comprehensive test system, including Zuul.
The latest documentation for Nodepool is published at: https://zuul-ci.org/docs/nodepool/
The latest documentation for Zuul is published at: https://zuul-ci.org/docs/zuul/
Getting Help
There are two Zuul-related mailing lists:
- zuul-announce
-
A low-traffic announcement-only list to which every Zuul operator or power-user should subscribe.
- zuul-discuss
-
General discussion about Zuul, including questions about how to use it, and future development.
You will also find Zuul developers in the #zuul channel on Freenode IRC.
Contributing
To browse the latest code, see: https://opendev.org/zuul/nodepool To clone the latest code, use git clone https://opendev.org/zuul/nodepool
Bugs are handled at: https://storyboard.openstack.org/#!/project/zuul/nodepool
Code reviews are handled by gerrit at https://review.opendev.org
After creating a Gerrit account, use git review to submit patches. Example:
# Do your commits
$ git review
# Enter your username if promptedJoin #zuul on Freenode to discuss development or usage.
License
Nodepool is free software, licensed under the Apache License, version 2.0.
Python Version Support
Nodepool requires Python 3. It does not support Python 2.