adab6eeb0d
GCE is the only "simple" driver since it turns out every other cloud is not simple enough. The state machine driver interface is just as simple (perhaps simpler) and can accomodate a variety of clouds. This ports the GCE driver to the statemachine interface and drops the simple interface entirely. Change-Id: Icfc298d83750ca31503211f920900d9207488bc2
185 lines
6.6 KiB
ReStructuredText
185 lines
6.6 KiB
ReStructuredText
.. _devguide:
|
|
|
|
Developer's Guide
|
|
=================
|
|
|
|
The following guide is intended for those interested in the inner workings
|
|
of nodepool and its various processes.
|
|
|
|
Operation
|
|
---------
|
|
|
|
If you send a SIGUSR2 to one of the daemon processes, Nodepool will
|
|
dump a stack trace for each running thread into its debug log. It is
|
|
written under the log bucket ``nodepool.stack_dump``. This is useful
|
|
for tracking down deadlock or otherwise slow threads.
|
|
|
|
Nodepool Builder
|
|
----------------
|
|
|
|
The following is the overall diagram for the `nodepool-builder` process and
|
|
its most important pieces::
|
|
|
|
+-----------------+
|
|
| ZooKeeper |
|
|
+-----------------+
|
|
^ |
|
|
bld | | watch
|
|
+------------+ req | | trigger
|
|
| client +---------+ | +--------------------+
|
|
+------------+ | | NodepoolBuilderApp |
|
|
| +---+----------------+
|
|
| |
|
|
| | start/stop
|
|
| |
|
|
+-------v-------+ |
|
|
| <-------+
|
|
+---------> NodePool- <----------+
|
|
| +---+ Builder +---+ |
|
|
| | | | | |
|
|
| | +---------------+ | |
|
|
| | | |
|
|
done | | start start | | done
|
|
| | bld upld | |
|
|
| | | |
|
|
| | | |
|
|
+---------v---+ +---v----------+
|
|
| BuildWorker | | UploadWorker |
|
|
+-+-------------+ +-+--------------+
|
|
| BuildWorker | | UploadWorker |
|
|
+-+-------------+ +-+--------------+
|
|
| BuildWorker | | UploadWorker |
|
|
+-------------+ +--------------+
|
|
|
|
Drivers
|
|
-------
|
|
|
|
.. autoclass:: nodepool.driver.Driver
|
|
:members:
|
|
.. autoclass:: nodepool.driver.Provider
|
|
:members:
|
|
.. autoclass:: nodepool.driver.ProviderNotifications
|
|
:members:
|
|
.. autoclass:: nodepool.driver.NodeRequestHandler
|
|
:members:
|
|
.. autoclass:: nodepool.driver.NodeRequestHandlerNotifications
|
|
:members:
|
|
.. autoclass:: nodepool.driver.ProviderConfig
|
|
:members:
|
|
|
|
|
|
Writing A New Provider Driver
|
|
-----------------------------
|
|
|
|
Nodepool drivers are loaded from the nodepool/drivers directory. A driver
|
|
is composed of three main objects:
|
|
|
|
- A ProviderConfig to manage validation and loading of the provider.
|
|
- A Provider to manage resource allocations.
|
|
- A NodeRequestHandler to manage nodeset (collection of resource) allocations.
|
|
|
|
Those objects are referenced from the Driver main interface that needs to
|
|
be implemented in the __init__.py file of the driver directory.
|
|
|
|
|
|
.. _provider_config:
|
|
|
|
ProviderConfig
|
|
~~~~~~~~~~~~~~
|
|
|
|
The ProviderConfig is constructed with the driver object and the provider
|
|
configuration dictionary.
|
|
|
|
The main procedures of the ProviderConfig are:
|
|
|
|
- getSchema() exposes a voluptuous schema of the provider configuration.
|
|
- load(config) parses the provider configuration. Note that the config
|
|
argument is the global Nodepool.yaml configuration. Each provided labels
|
|
need to be referenced back to the global config.labels dictionary so
|
|
that the launcher service know which provider provide which labels.
|
|
|
|
|
|
Provider
|
|
~~~~~~~~
|
|
|
|
The Provider is constructed with the ProviderConfig.
|
|
|
|
The main procedures of the Provider are:
|
|
|
|
- cleanupNode(external_id) terminates a resource
|
|
|
|
- listNodes() returns the list of existing resources. This procedure needs to
|
|
map the nodepool_node_id with each resource. If the provider doesn't support
|
|
resource metadata, the driver needs to implement a storage facility to
|
|
associate resource created by Nodepool with the internal nodepool_node_id.
|
|
The launcher periodically look for non-existent node_id in listNodes() to
|
|
delete any leaked resources.
|
|
|
|
- getRequestHandler(pool, request) returns a NodeRequestHandler object to manage
|
|
the creation of resources. The contract between the handler and the provider is
|
|
free form. As a rule of thumb, the handler should be in charge of interfacing
|
|
with Nodepool's database while the provider should provides primitive to create
|
|
resources. For example the Provider is likely to implement a
|
|
createResource(pool, label) procedure that will be used by the handler.
|
|
|
|
|
|
NodeRequestHandler
|
|
~~~~~~~~~~~~~~~~~~
|
|
|
|
The NodeRequestHandler is constructed with the assigned pool and the request object.
|
|
Before the handler is used, the following attributes are set:
|
|
|
|
* self.provider : the provider configuration.
|
|
* self.pool : the pool configuration.
|
|
* self.zk : the database client.
|
|
* self.manager : the Provider object.
|
|
|
|
The main procedures of the NodeRequestHandler are:
|
|
|
|
- launch(node) starts the creation of a new resource.
|
|
- launchesComplete() returns True if all the node of the nodesets self
|
|
attributes are READY.
|
|
|
|
An Handler may not have to launch each node of the nodesets as Nodepool will
|
|
re-use existing nodes.
|
|
|
|
The launch procedure usually consists of the following operations:
|
|
|
|
- Use the provider to create the resources associated with the node label.
|
|
Once an external_id is obtained, it should be stored to the node.external_id.
|
|
- Once the resource is created, READY should be stored to the node.state.
|
|
Otherwise raise an exception to restart the launch attempt.
|
|
|
|
|
|
State Machine Drivers
|
|
---------------------
|
|
|
|
To use this system, you will need to implement a few subclasses.
|
|
First, create a :ref:`provider_config` subclass as you would for any
|
|
driver.
|
|
|
|
Then, subclass :py:class:`~nodepool.driver.statemachine.Instance` to
|
|
map remote instance data into a format the driver can understand.
|
|
|
|
Next, create two subclasses of
|
|
:py:class:`~nodepool.driver.statemachine.StateMachine` to
|
|
implement creating and deleting instances.
|
|
|
|
Subclass :py:class:`~nodepool.driver.statemachine.Adapter` to
|
|
implement the main methods that interact with the cloud.
|
|
|
|
Finally, subclass
|
|
:py:class:`~nodepool.driver.statemachine.StateMachineDriver` to tie
|
|
them all together.
|
|
|
|
See the ``example`` provider for an example.
|
|
|
|
.. autoclass:: nodepool.driver.statemachine.Instance
|
|
:members:
|
|
.. autoclass:: nodepool.driver.statemachine.StateMachine
|
|
:members:
|
|
.. autoclass:: nodepool.driver.statemachine.Adapter
|
|
:members:
|
|
.. autoclass:: nodepool.driver.statemachine.StateMachineDriver
|
|
:members:
|