Zun container sandbox spec
Zun needs to manage containers as well as their associated IaaS resources, such as IP addresses, security groups, ports, volumes, etc.. To decouple the management of containers from their associated resources, we proposed to introduce a new concept called ``sandbox``. Partial-Implement: blueprint neutron-integration Change-Id: Ibdf9e4842d81b5b88c73531b9031c692aa2e24e5
This commit is contained in:
Hongbin Lu
parent
ed4f85890b
commit
19e3d9ce43
212
specs/container-sandbox.rst
Normal file
212
specs/container-sandbox.rst
Normal file
@ -0,0 +1,212 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
=================
|
||||
Container Sandbox
|
||||
=================
|
||||
Related Launchpad Blueprint:
|
||||
|
||||
https://blueprints.launchpad.net/zun/+spec/neutron-integration
|
||||
|
||||
Zun needs to manage containers as well as their associated IaaS resources,
|
||||
such as IP addresses, security groups, ports, volumes, etc.. To decouple the
|
||||
management of containers from their associated resources, we proposed to
|
||||
introduce a new concept called ``sandbox``.
|
||||
|
||||
A sandbox represents an isolated environment for one or multiple containers.
|
||||
The primary responsibility of sandbox is to provision and manage IaaS
|
||||
resources associated with a container or a group of containers. In this model,
|
||||
each container must have a sandbox, and resources (i.e. Neutron ports) are
|
||||
attached to sandboxes (instead of directly being attached to containers).
|
||||
|
||||
The implementation of sandbox is driver-specific. Each driver needs to
|
||||
implement the sandbox interface (as well as the container interface).
|
||||
For docker driver, sandbox can be implemented by using docker container itself.
|
||||
In this case, creating a container in Zun might create two docker containers
|
||||
internally: a sandbox container and a 'real' container. The real container
|
||||
might use the sandbox container as a proxy for networking, storage, or others.
|
||||
Alternatively, users might create a container in an existing sandbox if they
|
||||
don't want to create an extra sandbox.
|
||||
|
||||
Problem description
|
||||
===================
|
||||
Zun containers and Nova instances share the common needs for networking,
|
||||
storage, scheduling, host management, quota management, and many others.
|
||||
On the one hand, it is better to consolidate the management containers
|
||||
and Nova instances to minimize the duplication, but on the other hand,
|
||||
Zun needs to expose container-specific features that might go beyond the
|
||||
Nova APIs.
|
||||
|
||||
To provide a full-featured container experience with minimized duplication
|
||||
with Nova, an approach is to decouple the management of containers (implemented
|
||||
in Zun) from management of other resources (implemented in Nova). This
|
||||
motivates the introduction of sandbox that can be implemented as a docker
|
||||
container provisioned by Nova. As a result, we obtain flexibility to offload
|
||||
complex tasks to Nova.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
1. Introduce a new abstraction called ``sandbox``. Sandbox represents an
|
||||
isolated environment for one or multiple containers. All drivers are
|
||||
required to implement the sandbox abstraction with business logic to create
|
||||
and manage the isolated environment. For Linux container, an isolated
|
||||
environment can be implemented by using various Linux namespaces
|
||||
(i.e. pid, net, or ipc namespace).
|
||||
2. For docker container, its sandbox could be implemented by using docker
|
||||
container itself. The sandbox container might not do anything real, but
|
||||
has one or multiple Neutron ports (or other resources) attached.
|
||||
The sandbox container is provisioned and managed by Nova (with a
|
||||
Zun-provided docker virt-driver). After the sandbox container is created,
|
||||
the real container can be created with the options
|
||||
``--net=container:<sandbox>``, ``--ipc=container:<sandbox>``,
|
||||
``--pid=container:<sandbox>`` and/or ``--volumes-from=<sandbox>``.
|
||||
This will create a container by joining the namespaces of the sandbox
|
||||
container so that resources in sandbox container can be shared.
|
||||
3. The design should be extensible so that operators can plug-in their
|
||||
own drivers if they are not statisfied by the built-in sandbox
|
||||
implementation(s).
|
||||
|
||||
The diagram below offers an overview of the system architecture. The Zun
|
||||
service may communicate with Nova to create a sandbox that is actually a
|
||||
docker container. Like normal Nova instances, sandboxes are scheduled by Nova
|
||||
scheduler and have Neutron ports attached for providing networking.
|
||||
Containers are created by Zun, and run inside the namespaces of sandboxes.
|
||||
Note that a sandbox can contain one or multiple containers. All containers
|
||||
in a sandbox will be co-located and share the Linux namespaces of the sandbox.
|
||||
|
||||
::
|
||||
|
||||
+---------+
|
||||
| CLI |
|
||||
+----+----+
|
||||
|
|
||||
+----+----+
|
||||
+-------- Nova -------+ +-+ REST +----- Zun -----+
|
||||
| | | +---------+ |
|
||||
| +--+ |
|
||||
| | | |
|
||||
+-----------+---------+ +---------------+-----------+
|
||||
| |
|
||||
+-----------|----+ Compute Host ---------|-----------+
|
||||
| +------+-------+ +-----+-----+ |
|
||||
| +--+ Nova Compute +--+ +---+ Zun Agent +-+ |
|
||||
| | +--------------+ | | +-----------+ | |
|
||||
| | | | | |
|
||||
| | +-----+-------|---+ | |
|
||||
| | | | | | |
|
||||
| +-- Sandbox -+ +-- Sandbox --|-+ +-- Sandbox --|-+ |
|
||||
| | | | | | | | | |
|
||||
| | | | +-----------+ | | +-----------+ | |
|
||||
| | | | | Container | | | | Container | | |
|
||||
| | | | +-----------+ | | +-----------+ | |
|
||||
| | | | | | +-----------+ | |
|
||||
| | | | | | | Container | | |
|
||||
| | | | | | +-----------+ | |
|
||||
| | | | | | | |
|
||||
| +------------+ +---------------+ +---------------+ |
|
||||
| |
|
||||
+----------------------------------------------------+
|
||||
|
||||
|
||||
Design Principles
|
||||
-----------------
|
||||
1. Minimum duplication between Nova and Zun. If possibly, reuse everything that
|
||||
has been implemented in Nova.
|
||||
2. Similar user experience between VMs and containers. In particular, the ways
|
||||
to configure networking of a container should be similar as the VM
|
||||
equivalent.
|
||||
3. Full-featured container APIs.
|
||||
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
1. Directly bind resources (i.e. Neutron ports) to containers. This will have a
|
||||
large amount of duplication between Nova and Zun.
|
||||
2. Use Kuryr. Kuryr is designed for users who use native tool (i.e. docker
|
||||
CLI) to manage containers. In particular, what Kuryr provided to translate
|
||||
API calls to docker to API calls to Neutron. Zun doesn't expose native APIs
|
||||
to end-users so Kuryr cannot address Zun's use cases.
|
||||
|
||||
|
||||
Data model impact
|
||||
-----------------
|
||||
Add a 'sandbox' table. Add a foreign key 'sandbox_id' to the existing table
|
||||
'container'.
|
||||
|
||||
|
||||
REST API impact
|
||||
---------------
|
||||
1. Add an new API endpoint /sandboxes to the REST API interface.
|
||||
2. In the API endpoint of creating a container, add a new option to specify
|
||||
the sandbox where the container will be created from. If the sandbox is not
|
||||
specified, Zun will create a new sandbox for the container.
|
||||
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
None
|
||||
|
||||
|
||||
Notifications impact
|
||||
--------------------
|
||||
None
|
||||
|
||||
|
||||
Other end user impact
|
||||
---------------------
|
||||
None
|
||||
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
Performance penalty is expected since provisioning sandboxes take extra
|
||||
compute resources. In addition, the Nova API will be used to create sandboxes,
|
||||
which might also incur performance penalty.
|
||||
|
||||
|
||||
Other deployer impact
|
||||
---------------------
|
||||
Deployers need to deploy a custom Nova virt-driver for provisioning sandboxes.
|
||||
|
||||
|
||||
Developer impact
|
||||
----------------
|
||||
None
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
Hongbin Lu
|
||||
|
||||
Other contributors:
|
||||
|
||||
|
||||
Work Items
|
||||
----------
|
||||
1. Implement a custom Nova virt-driver to provision sandboxes.
|
||||
2. Implement a new API endpoint for sandboxes.
|
||||
3. Implement unit/integration test.
|
||||
|
||||
|
||||
Dependencies
|
||||
============
|
||||
Add a dependency to Nova
|
||||
|
||||
|
||||
Testing
|
||||
=======
|
||||
Each patch will have unit tests, and Tempest functional tests covered.
|
||||
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
A set of documentation for this new feature will be required.
|
||||
Loading…
Reference in New Issue
Block a user