x/valence
Code Issues Proposed changes

Updating docs

Browse Source 
>> Changed project name from plasma to valence in docs.
>> Fixed docstring indentation.
>> Added features docs.
>> Added driver docs.
>> Hidden licence headers from html files.

Depends-On: Ia02bc00ad168b3c3d01ef6ca9696d43996091070
Change-Id: I1fa382d566165f5e76c84ad864024c0f546ef74c
This commit is contained in:
akhiljain23 2018-04-05 12:38:56 +05:30
parent 47777b216a
commit cf99ff0165
51 changed files with 1214 additions and 157 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

123
README.rst
View File

@ -2,100 +2,79 @@
Openstack Valence Project
=========================
********
Overview
********
Valence is a service for lifecycle management of pooled bare-metal hardware
infrastructure such as Intel(R) Rack Scale architecture which uses Redfish(TM)
as one of the management protocols.
infrastructure. The concept of pooled storage (SSDs or nvmE) disaggregated
from compute nodes and network disaggregated from compute and storage
provides the flexibility to compose and uses as and when the cloud requires
more server resources. Valence provides the capability "compose" hardware nodes
and release resources as needed by the overcloud.
Valence supports Redfish as default management protocol to communicate
to hardware. It supports Rack Scale Design (RSD), Open architecture for management
of disaggregated server hardware resources, which is standardized in Redfish.
Valence also provides capability to manage other vendors disaggregated hardware
using their respective drivers other than Redfish.
:Free software: Apache license
:Wiki: https://wiki.openstack.org/wiki/Valence
:Source: http://git.openstack.org/cgit/openstack/valence
:Bugs: https://bugs.launchpad.net/openstack-valence
.. _valence-features:
=========================
Download and Installation
=========================
******************
Feature Highlights
******************
The following steps capture how to install valence. All installation steps
require super user permissions.
Valence provides the following capabilities for managing disaggregated hardware resources.
**************************
Database etcd installation
**************************
.. _multi-podm-support:
Single node installation reference: https://github.com/coreos/etcd/releases
Multiple Podmanager Support
---------------------------
Distributed installation reference: https://github.com/coreos/etcd/blob/master/Documentation/op-guide/clustering.md
Valence provides the capability to manage multiple podmanagers.
Each podmanager can be configured to use different driver. By default,
``redfishv1`` driver is used.
For development, single node installation is recommended practice.
Currently supported drivers in Valence are:
#. :ref:`redfishv1-driver`
#. :ref:`expether-driver`
********************
Valence installation
********************
Future work include redfish v2 driver support. Other vendors also could implement
their own driver to manage their hardware.
1. Install software dependencies
This provides uniform interface to manage all the disaggregated hardware in datacentre
supporting different protocols.
``$ sudo apt-get install git python-pip python-dev build-essential``
.. _device-orchestration:
2. Clone the Valence code from git repo.
Device Orchestration framework
------------------------------
``$ git clone https://git.openstack.org/openstack/valence``
Valence provides support for the dynamic management of pooled resources like storage,
network and other pci devices which can be connected on demand to a composed node,
giving user the ability to attach or detach the devices dynamically based on workload.
3. Execute the 'install_valence.sh' file present in the Valence root directory.
The install script will automatically install the dependencies listed in the
requirements.txt file.
.. _ironic-provision-driver:
``$ sudo bash install_valence.sh``
Ironic provision driver
-----------------------
4. Check the values in valence.conf located at /etc/valence/valence.conf
Valence supports ``ironic`` provision driver to be able to register the valence composed
node to ironic. Once the node is registered further operations of provisioning/managing
could be directly performed using Ironic_.
``set the ip/credentials of podm for which this Valence will interact``
5. Check the PYTHON_HOME and other variables in /etc/init/valence.conf
6. Initialize etcd database
``$ valence-db-manager init``
Note: The TypeError exception "TypeError: NoneType object is not callable"
is caused by known python-etcd bug, which will not impact this db init
functionality.
https://github.com/jplana/python-etcd/issues/190
7. Start valence service
``$ sudo service valence start``
8. Logs are located at /var/logs/valence/
****************
GUI installation
****************
Please refer to the installation steps in the ui/README file.
For more details on Ironic usage:
`Ironic API guide <https://developer.openstack.org/api-ref/baremetal/>`_.
**********
Components
**********
For more features please refer the Valence blueprints_ for supported and
in-the-pipeline features.
Valence follows the typical OpenStack project setup. The components are listed
below:
valence-api
-----------
A python based daemon based on Flask framework to expose Valence REST APIs.
The api service communicates to the PODM through REST interface using Redfish(TM) specification.
For adding new api please refer https://github.com/openstack/valence/blob/master/doc/source/developer-guide/add_new_api.rst
valence-ui
----------
valence-ui provides a Web-based GUI interface that can be used to explore
Rack Scale Design (RSD) artifacts and compose/disassemble nodes.
valence-ui is implemented using Node.js runtime environment and hosted through apache.
valence-ui makes us of React.js javascript libaray and invoke Valence REST APIs through ajax REST calls.
========
Features
========
Please refer the Valence blueprints for supported and in-the-pipeline features.
``https://blueprints.launchpad.net/openstack-valence``
.. _blueprints: https://blueprints.launchpad.net/openstack-valence
.. _Ironic: https://docs.openstack.org/ironic/latest/

1
api-ref/source/index.rst
View File

@ -15,3 +15,4 @@ This is a reference for the Openstack Valence API
.. include:: valence-api-v1-system.inc
.. include:: valence-api-v1-nodes.inc
.. include:: valence-api-v1-pods.inc
.. include:: valence-api-v1-devices.inc

20
api-ref/source/mockup/devices-get-response.json Normal file
View File

@ -0,0 +1,20 @@
{
"created_at": "2018-04-09 05:57:55 UTC",
"extra": {
"device_name": "PCIe Data Center SSD",
"vendor_name": "Intel Corporation"
},
"node_id": null,
"podm_id": "64420559-0f44-4ca1-82c2-ac9a26f6788b",
"pooled_group_id": "4093",
"properties": {
"device_id": "0x000000000000",
"mac_address": "00:11:22:33:44:55",
"model": "Generation-X"
},
"resource_uri": "devices/0x000000000000",
"state": "free",
"type": "SSD",
"updated_at": "2018-04-09 05:57:55 UTC",
"uuid": "95a6b68f-9c3a-47f8-b93b-c160c940d7b7"
}

20
api-ref/source/mockup/devices-list-response.json Normal file
View File

@ -0,0 +1,20 @@
[
{
"node_id": "0x000000000000",
"podm_id": "11111111-1111-1111-1111-111111111111",
"pooled_group_id": "2223",
"resource_uri": "devices/2x222222222222",
"state": "allocated",
"type": "SSD",
"uuid": "95a6b68f-9c3a-47f8-b93b-c160c940d7b7"
},
{
"node_id": null,
"podm_id": "64420559-0f44-4ca1-82c2-ac9a26f6788b",
"pooled_group_id": "4093",
"resource_uri": "devices/3x333333333333",
"state": "free",
"type": "NIC",
"uuid": "603ba114-574e-48d3-afa2-9fd8d98fe0d2"
}
]

6
api-ref/source/mockup/devices-sync-response.json Normal file
View File

@ -0,0 +1,6 @@
[
{
"podm_id": "64420559-0f44-4ca1-82c2-ac9a26f6788b",
"status": "SUCCESS"
}
]

12
api-ref/source/mockup/pod-manager-create-request.json Normal file
View File

@ -0,0 +1,12 @@
{
"url": "http://0.0.0.0:00000/server",
"name": "podm1",
"driver": "driver_name",
"authentication": [{
"type": "basic",
"auth_items": {
"username": "admin",
"password": "password"
}
}]
}

18
api-ref/source/mockup/pod-manager-create-response.json Normal file
View File

@ -0,0 +1,18 @@
{
"authentication": [
{
"auth_items": {
"password": "***",
"username": "admin"
},
"type": "basic"
}
],
"created_at": "2018-04-16 10:01:27 UTC",
"driver": "driver_name",
"name": "podm1",
"status": "Online",
"updated_at": "2018-04-16 10:01:27 UTC",
"url": "http://0.0.0.0:00000/server",
"uuid": "776a4f7b-195d-49c9-9b52-b25760d85158"
}

119
api-ref/source/valence-api-v1-devices.inc Normal file
View File

@ -0,0 +1,119 @@
.. -*- rst -*-
=======
Devices
=======
List, Searching and Sync of devices are done through the ``/v1/devices``
List Devices
============
.. rest_method:: GET /v1/devices/
Return a list of Devices.
Some filtering is possible by passing in filters with the request.
.. NOTE::
Request format: GET /v1/devices?{filters}
where filters can be combination of keys: node_id, podm_id, state,
type, pooled_group_id, resource_uri.
For example: GET /v1/devices?node_id=0x000000&type=SSD
By default, this query will return all devices with uuid, podm_id,
type, state, node_id, resource_uri and pooled_group_id.
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
Response
--------
.. rest_parameters:: parameters.yaml
- node_id: node_index
- podm_id: podm_uuid
- pooled_group_id: device_group_id
- resource_uri: device_uri
- state: device_state
- type: device_type
- uuid: device_uuid
**Example list of Devices:**
.. literalinclude:: mockup/devices-list-response.json
:language: javascript
Display Device Details
======================
.. rest_method:: GET /v1/devices/{device_id}
Shows details for a Device.
This will return the full representation of the device.
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
.. rest_parameters:: parameters.yaml
- id: device_id
Response
--------
.. rest_parameters:: parameters.yaml
- created_at: created_at
- extra: device_details
- node_id: node_index
- podm_id: podm_uuid
- pooled_group_id: device_group_id,
- properties: device_properties
- resource_uri: device_uri
- state: device_state
- type: device_type
- updated_at: updated_at
- uuid: device_uuid
**Example JSON representation of a Device:**
.. literalinclude:: mockup/devices-get-response.json
:language: javascript
Sync Devices
============
.. rest_method:: POST /v1/devices/sync
Sync all devices managed with podmanager.
Synchronization of devices connected to one podmanager is possible by
passing ``podm_id`` in the body of POST request.
By default it will sync devices connected to all podmanagers one by one.
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
Response
--------
.. rest_parameters:: parameters.yaml
- podm_id: podm_uuid
- status: sync_status
**Example JSON representation of a Sync response:**
.. literalinclude:: mockup/devices-sync-response.json
:language: javascript

53
api-ref/source/valence-api-v1-pods.inc
View File

@ -4,9 +4,59 @@
Pod managers
==============
Listing, searching of Pod Manager resources is done through the ``/v1/pod_managers``
Creating, Listing, searching of Pod Manager resources is done through the
``/v1/pod_managers``
Send feedback to Valence team or [chester.kuo@gmail.com]
Create Pod manager
==================
.. rest_method:: POST /v1/pod_manager
Creates a new Pod Manager with specific URL, name, authentication credentials
and driver(default driver: ``redfishv1``).
Normal response codes: 200
Error response codes: badRequest(400), unauthorized(401), forbidden(403)
Request
-------
.. rest_parameters:: parameters.yaml
- name: podmanager_name
- url: pod_conn_url
- driver: podm_driver
- authentication: auth_details
**Example Pod Manager creation request:**
.. literalinclude:: mockup/pod-manager-create-request.json
:language: javascript
Response
--------
The response will contain the complete pod uuid and name record
The list and example below are representative of the response as of API
.. rest_parameters:: parameters.yaml
- name: podmanager_name
- url: pod_conn_url
- driver: podm_driver
- authentication: auth_details
- created_at: created_at
- updated_at: updated_at
- uuid: pod_uuid
**Example JSON representation of a Pod Manager:**
.. literalinclude:: mockup/pod-manager-create-response.json
:language: javascript
List Pod Manager
=================
@ -97,4 +147,3 @@ Request
.. rest_parameters:: parameters.yaml
- pod_uuid: pod_uuid

4
api-ref/source/valence-api-version.inc
View File

@ -8,7 +8,7 @@ Concepts
========
In order to bring new features to users over time, the Valence API
supports versioning. There are two kinds of versions in Ironic.
supports versioning. There are two kinds of versions in Valence.
- ''major versions'', which have dedicated urls.
- ''microversions'', which can be requested through the use of the
@ -16,7 +16,7 @@ supports versioning. There are two kinds of versions in Ironic.
The Version APIs work differently from other APIs as they do not require authentication.
Alll API requests support the ``OpenStack-API-Version`` header.
All API requests support the ``OpenStack-API-Version`` header.
This header SHALL be supplied with every request.
This help to provide backwards compatibility as we introduced new features in the server.

4
doc/api-mockup/index.json
View File

@ -1,6 +1,6 @@
{
"name" : "OpenStack Plasma API",
"description" : "Plasma is an OpenStack project which aims to provide node composition based on redfish API.",
"name" : "OpenStack Valence API",
"description" : "Valence is an OpenStack project which aims to provide node composition based on redfish API.",
"default_version" : {
"status" : "CURRENT",
"version" : "1.1",

4
doc/api-mockup/v1/index.json
View File

@ -8,7 +8,7 @@
{
"rel" : "describedby",
"type" : "text/html",
"href" : "http://docs.openstack.org/developer/plasma/dev/api-spec-v1.html"
"href" : "http://docs.openstack.org/developer/valence/dev/api-spec-v1.html"
}
],
"nodes" : [
@ -43,7 +43,7 @@
],
"media_types" : [
{
"type" : "application/vnd.openstack.plasma.v1+json",
"type" : "application/vnd.openstack.valence.v1+json",
"base" : "application/json"
}
]

4
doc/source/contributing.rst
View File

@ -1,4 +0,0 @@
============
Contributing
============
.. include:: ../../CONTRIBUTING.rst

22
doc/source/contributors-guide/contributing.rst Normal file
View File

@ -0,0 +1,22 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _valence-contribution:
============
Contributing
============
.. include:: ../../../CONTRIBUTING.rst

6
doc/source/developer-guide/add_new_api.rst
View File

@ -1,4 +1,4 @@
.. _add_new_api:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _add-new-api:
=================================
Add a new API endpoint in Valence
=================================
@ -24,7 +26,7 @@ through the `readme.rst`).
Interacting with Valence
-------------------------
------------------------
Before starting modification/adding a new functionality in Valence, it is suggested
to check that the valence has been setup properly. This could be done by making some

4
doc/source/developer-guide/add_new_developer_guide.rst
View File

@ -1,4 +1,4 @@
.. _valence_add_new_developer_guide:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_add_new_developer_guide:
============================
Add a developer guide doc
============================

9
doc/source/developer-guide/add_new_functional_testing.rst
View File

@ -1,4 +1,4 @@
.. _valence_functional_testcase:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_functional_testcase:
==========================
Add a Functional Test case
==========================
@ -34,10 +36,10 @@ Tests scripts are located in `<valence_root>/valence/tests
Implementing a Test Case
------------------------
Consider implementing an functional testcase for our /example(add_new_api_) API
Consider implementing an functional testcase for our /example(:ref:`add-new-api`) API
The characteristics of /example(add_new_api_) API
The characteristics of /example(:ref:`add-new-api`) API
* REST Method : GET
* Parameters : Nil
@ -81,3 +83,4 @@ running the below command from the valence root directory(where tox.ini is prese
.. code-block:: bash
$ tox -e pep8,py27

8
doc/source/developer-guide/add_new_unit_test.rst
View File

@ -1,4 +1,4 @@
.. _valence_unit_testcase:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
@ -14,9 +14,11 @@
License for the specific language governing permissions and limitations
under the License.
======================
.. _valence_unit_testcase:
====================
Add a Unit Test case
======================
====================
Getting used to writing testing code and running this code in parallel is considered
a good workflow.

37
doc/source/developer-guide/components.rst Normal file
View File

@ -0,0 +1,37 @@
..
Copyright (c) 2017 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _valence-components:
==========
Components
==========
Valence follows the typical OpenStack project setup. The components are listed
below:
valence-api
-----------
A python based daemon based on Flask framework to expose Valence REST APIs.
The api service communicates to the PODM through REST interface using Redfish(TM) specification.
For adding new api please refer https://github.com/openstack/valence/blob/master/doc/source/developer-guide/add_new_api.rst
valence-ui
----------
valence-ui provides a Web-based GUI interface that can be used to explore
Rack Scale Design (RSD) artifacts and compose/disassemble nodes.
valence-ui is implemented using Node.js runtime environment and hosted through apache.
valence-ui makes us of React.js javascript library and invoke Valence REST APIs through ajax REST calls.

4
doc/source/developer-guide/db_development.rst
View File

@ -1,4 +1,4 @@
.. _valence_db_development:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
@ -14,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_db_development:
==============
DB Development
==============

5
doc/source/developer-guide/deploy.rst
View File

@ -1,5 +1,4 @@
.. _valence_deploy:
..
Copyright 2016 Intel Corporation
All Rights Reserved.
@ -15,6 +14,8 @@
License for the specific language governing permissions and limitations
under the License.
.. _valence_deploy:
======================
Deploy Valence Project
======================

304
doc/source/developer-guide/drivers/expether.rst Normal file
View File

@ -0,0 +1,304 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _expether-driver:
===============
ExpEther Driver
===============
Overview
========
The ``ExpEther`` driver enables management of ExpEther Hardware through
Openstack Valence. It relies on EEM REST API to communicate to hardware.
For more details on hardware, please refer ExpEther_.
Prerequisites
=============
ExpEtherManager(EEM) should be configured and reachable from the controller.
It can be installed on node different from Openstack Controller.
``hwdata`` package is used to fetch PCI device details (vendor name,
device name) in human readable format. Retrieval of details is only
supported for 40g devices. If package is not available, the same
are shown as None.
For ubuntu, package can be installed using following command::
$ sudo apt-get install hwdata
Supported Functionalities
=========================
* To be able to use the driver, first the podmanager needs to be created
with connection information of the EEM service.
The details needed for the same are:
- EEM REST URL : http://<ip>:<port>/eem
- Authentication Information (username/password)
- driver : 'expether'
To register the same please refer :ref:`example <create-pod-manager>`
* Once the podmanager is created, the same will be used for subsequent requests
to communicate with hardware.
* Using ``expether`` driver the following functionalities are supported in Valence:
#. Node Management, Server can be composed as per user requested flavor or
the existing node can be made to be managed by Valence.
Currently, for composing node, only supported flavor keys are ``pci_device``.
Please refer :ref:`sample request <compose-node>`
#. All the IO devices would be automatically discovered by the podmanager and
would be managed by Valence. Also supports allocation of PCI devices from the
resource pool to node as per workload demand.
EE node managed by valence can be assigned from the pool of devices as per
user request and the same will be released to pool once the workload decreases.
#. Periodic discovery of all resources from podmanager is supported,
which registers any newly connected resources to valence and keeps the status sync
with podmanager and valence DB.
#. Nodes managed by Valence can be made available to Ironic, for further provisioning using
ironic provision driver supported in valence. For more details
refer :ref:`ironic-provision-driver` and :ref:`sample request <node-register>`
Sample Requests
===============
.. _create-pod-manager:
Create a pod_manager
--------------------
Create a new podmanager connecting to ExpressEther Manager.
Sample request:
.. code-block:: bash
curl -i -X POST "http://localhost:8181/v1/pod_managers" \
-d '{"url": "http://<ip>:<port>/eem", "name": "podm1", \
"driver": "expether", "authentication": [{"type": "basic", \
"auth_items": {"username": "xxx", "password": "xxxx"}}]}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: json
{
"authentication": [
{
"auth_items": {
"password": "xxx",
"username": "xxx"
},
"type": "basic"
}
],
"created_at": "2018-04-20 04:40:01 UTC",
"driver": "expether",
"name": "podm1",
"status": "Online",
"updated_at": "2018-04-20 04:40:01 UTC",
"url": "http://<ip>:<port>/eem",
"uuid": "da5b1fba-e8bb-42be-baff-66ccb74087aa"
}
.. _compose-node:
Compose a node
--------------
#. Using properties:
Only flavor key supported is ``pci_device``.
Example: {"pci_device": {"type": ["NIC"]}}
Sample request:
.. code-block:: bash
curl -i -X POST "http://localhost:8181/v1/nodes" \
-d '{"podm_id": "00000000-0000-0000-0000-000000000000", \
"name": "node1", "properties": {"pci_device": {"type": ["NIC"]}}}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: json
{
"index": "0x000000000000",
"name": "node1",
"resource_uri": "devices/0x000000000000",
"uuid": "bf28249c-a903-4ea9-a440-1ab28b0dab55"
}
#. Using flavor:
Sample request:
.. code-block:: bash
curl -i -X POST "http://localhost:8181/v1/nodes" \
-d '{"podm_id": "00000000-0000-0000-0000-000000000000", \
"name": "node1", "flavor_id": "11111111-1111-1111-1111-111111111111"}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: json
{
"index": "0x000000000000",
"name": "node1",
"resource_uri": "devices/0x000000000000",
"uuid": "1ed6bba0-6354-4f57-aa61-09c15d5955bb"
}
Manage a node
-------------
Register existing node to valence.
Sample request:
.. code-block:: bash
curl -i -g -X POST "http://localhost:8181/v1/nodes/manage" \
-d '{"podm_id": <podm_id>, \
"node_index": <node-index>}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: json
{
"index": "0x8cdf9d535b14",
"name": "0x8cdf9d535b14",
"resource_uri": "devices/0x8cdf9d535b14",
"uuid": "2eebc520-7035-4797-a4ba-3b3dee2ea266"
}
List devices
------------
List all resources.
Sample request:
.. code-block:: bash
curl -i -X GET "http://localhost:8181/v1/devices" \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: json
[
{
"node_id": "0x000000000000",
"podm_id": "da5b1fba-e8bb-42be-baff-66ccb74087aa",
"pooled_group_id": "1234",
"resource_uri": "devices/1x111111111111",
"state": "allocated",
"type": "SSD",
"uuid": "d38b2987-02f1-44c1-bdb6-c5469581d244"
},
{
"node_id": null,
"podm_id": "da5b1fba-e8bb-42be-baff-66ccb74087aa",
"pooled_group_id": "4093",
"resource_uri": "devices/2x222222222222",
"state": "free",
"type": "NIC",
"uuid": "f3f57251-4213-487d-a471-8a2e5b1e18e4"
}
]
Attach/detach a device to node
------------------------------
Attach a resource to node.
Sample request:
.. code-block:: bash
curl -i -X POST "http://localhost:8181/v1/nodes/<node_id>/action" \
-d '{"attach": {"resource_id": "<resource_id>"}}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: console
204 NO CONTENT
Detach a resource to node.
Sample request:
.. code-block:: bash
curl -i -X POST "http://localhost:8181/v1/nodes/<node_id>/action" \
-d '{"detach": {"resource_id": "<resource_id>"}}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: console
204 NO CONTENT
.. _node-register:
Node register
-------------
Register node with ironic.
Sample request:
.. code-block:: bash
curl -i -X POST \
"http://localhost:8181/v1/nodes/bd412ef8-d49e-46f3-a7dd-9879a7435dc9/register" \
-d '{"driver_info": {"username":"admin","password":"password", \
"address":"address"}}' \
-H "Accept:application/json" -H "Content-Type:application/json"
Response:
.. code-block:: json
{
"created_at": "2018-04-20 04:40:01 UTC",
"index": "0x000000000000",
"managed_by": "ironic",
"name": "node1",
"podm_id": "da5b1fba-e8bb-42be-baff-66ccb74087aa",
"resource_uri": "devices/0x000000000000",
"updated_at": "2018-04-20 04:40:01 UTC",
"uuid": "1ed6bba0-6354-4f57-aa61-09c15d5955bb"
}
.. _ExpEther: http://www.expether.org/

28
doc/source/developer-guide/drivers/index.rst Normal file
View File

@ -0,0 +1,28 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _drivers:
Drivers
=======
This section covers information about drivers supported in valence
.. toctree::
:maxdepth: 1
expether
redfishv1

26
doc/source/developer-guide/drivers/redfishv1.rst Normal file
View File

@ -0,0 +1,26 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _redfishv1-driver:
================
RedfishV1 Driver
================
The ``redfish`` driver enables managing pooled bare-metal infrastructure
compliant with the Redfish_ as management protocol.
.. _Redfish: http://redfish.dmtf.org/

32
doc/source/developer-guide/index.rst Normal file
View File

@ -0,0 +1,32 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _developer-guide:
Valence Developers Manual
=========================
.. toctree::
:maxdepth: 2
components.rst
drivers/index
add_new_api.rst
add_new_developer_guide.rst
add_new_functional_testing.rst
add_new_unit_test.rst
db_development.rst
deploy.rst

32
doc/source/end-user-guide/api_ref/index.rst Normal file
View File

@ -0,0 +1,32 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _api-ref:
===========
Valence API
===========
.. toctree::
:maxdepth: 1
valence-api-version.rst
valence-api-v1-pooled.rst
valence-api-v1-flavors.rst
valence-api-v1-system.rst
valence-api-v1-nodes.rst
valence-api-v1-pods.rst
valence-api-v1-devices.rst

1
doc/source/end-user-guide/api_ref/valence-api-v1-devices.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../../../api-ref/source/valence-api-v1-devices.inc

1
doc/source/end-user-guide/api_ref/valence-api-v1-flavors.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../../../api-ref/source/valence-api-v1-flavors.inc

1
doc/source/end-user-guide/api_ref/valence-api-v1-nodes.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../../../api-ref/source/valence-api-v1-nodes.inc

1
doc/source/end-user-guide/api_ref/valence-api-v1-pods.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../../../api-ref/source/valence-api-v1-pods.inc

1
doc/source/end-user-guide/api_ref/valence-api-v1-pooled.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../../../api-ref/source/valence-api-v1-pooled.inc

1
doc/source/end-user-guide/api_ref/valence-api-v1-system.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../../../api-ref/source/valence-api-v1-system.inc

1
doc/source/end-user-guide/api_ref/valence-api-version.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../../../api-ref/source/valence-api-version.inc

26
doc/source/end-user-guide/index.rst Normal file
View File

@ -0,0 +1,26 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _end-user-guide:
Valence End User Manual
=======================
.. toctree::
:maxdepth: 2
api_ref/index
usage.rst

5
doc/source/usage.rst → doc/source/end-user-guide/usage.rst
View File

@ -1,7 +1,8 @@
========
=====
Usage
========
=====
To use valence in a project::
import valence

39
doc/source/index.rst
View File

@ -3,25 +3,44 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to valence's documentation!
========================================================
Contents:
===================================
Welcome to Valence's Documentation!
===================================
Introduction to Valence
=======================
.. toctree::
:maxdepth: 2
readme
installation
usage
contributing
Developers Documentation
========================
Operators Guide
===============
.. toctree::
:maxdepth: 2
operators-guide/index
End User Guide
==============
.. toctree::
:maxdepth: 2
end-user-guide/index
Developers Guide
================
.. toctree::
:maxdepth: 2
developer-guide/index
Contributors Guide
==================
.. toctree::
:glob:
developer-guide/*
contributors-guide/*
Indices and tables
==================

12
doc/source/installation.rst
View File

@ -1,12 +0,0 @@
============
Installation
============
At the command line::
$ pip install valence
Or, if you have virtualenvwrapper installed::
$ mkvirtualenv valence
$ pip install valence

180
doc/source/operators-guide/configuration.rst Normal file
View File

@ -0,0 +1,180 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _valence-conf:
=============================
Valence Configuration Options
=============================
Valence provides configuration file to configure Valence service specific to
your requirements.
This file is typically located at ``/etc/valence/valence.conf``.
The configuration file is organized into the following sections:
* ``[DEFAULT]`` - General configuration
* ``[api]`` - API server configuration
* ``[etcd]`` - etcd configurations
* ``[ironic_client]`` - Options for ironic client
* ``[podm]`` - Configuration options for podm service
You can easily generate and update a sample configuration file
named `valence.conf.sample` by using following commands::
$ git clone https://github.com/openstack/valence.git
$ cd valence/
$ tox -e genconfig
$ vi etc/valence.conf.sample
Some configuration options are mentioned here, it is recommended that you
review all the options so that the valence service is configured for your needs.
#. General configuration of valence service as follow under ``DEFAULT``
section::
[DEFAULT]
# The log file location for valence service
log_file = /var/log/valence/valence.log
# The granularity of log outputs. By default set to info level
# Possible values: info, critical, warning, debug, error, notset
log_level=debug
# The log format
log_format = %(asctime)s %(name)-4s %(levelname)-4s %(message)s
#. The API server configuration values can be set as in following sample
values::
[api]
# The port for the valence API server. (port value)
bind_port=8181
# The listen IP for the valence API server. (IP address value)
bind_host=0.0.0.0
# Enable the integrated stand-alone API to service requests via HTTPS
# instead of HTTP. If there is a front-end service performing HTTPS
# offloading from the service, this option should be False; note, you
# will want to change public API endpoint to represent SSL termination
# URL with 'public_endpoint' option. (boolean value)
enable_ssl_api = false
# Number of workers for valence-api service. The default will be the
# number of CPUs available. (integer value)
workers=4
# The maximum timeout to wait for valence API server to come up.
# (integer value)
timeout=1000
# The maximum number of items returned in a single response from a
# collection resource. (integer value)
max_limit = 1000
# Configuration file for WSGI definition of API. (string value)
api_paste_config = api-paste.ini
# Start API server in debug mode. (boolean value)
debug=true
# The log file location for valence API server (string value)
log_file = /var/log/valence/valence-api.log
# The granularity of API server log outputs. (string value)
# Possible values: info, critical, warning, debug, error
log_level=debug
#. Configure the details of the etcd via ``port`` and ``host`` option.
In the following, replace PORT_NUMBER with the port of etcd service,
and replace HOST_IP with the IP address where the etcd service is running::
[etcd]
# The port for the etcd server. (port value)
port=PORT_NUMBER
# The listen IP for the etcd server. (IP address value)
host=HOST_IP
#. Valence shall communicate with the ironic client in order to create node in
Ironic, which requires the Valence service to be configured with the right
credentials for the ironic client service.
In the configuration section here below:
* replace IDENTITY_IP with the IP of the Identity server
* replace IRONIC_PASSWORD with the password you chose for the ``ironic``
user
* replace PROJECT_NAME with the name of project created for
OpenStack services (e.g. ``service``) ::
[ironic_client]
# The name of user to interact with Ironic API service. (string value)
username = ironic
# Password of the user specified to authorize to communicate with the
# Ironic API service. (string value)
password = IRONIC_PASSWORD
# The project name which the user belongs to. (string value)
project = PROJECT_NAME
# The OpenStack Identity Service endpoint to authorize the user
# against. (string value)
auth_url = http://<IDENTITY_IP>/identity
# ID of a domain the user belongs to. (string value)
user_domain_id = default
# ID of a domain the project belongs to. (string value)
project_domain_id = default
# Version of Ironic API to use in ironicclient. (string value)
api_version = 1
# Optional CA cert file to use in SSL connections. (string value)
os_cacert = None
# Optional PEM-formatted certificate chain file. (string value)
os_cert = None
# Optional PEM-formatted file that contains the private key. (string
# value)
os_key = None
# If set, then the server's certificate will not be verified. (boolean
# value)
insecure = false
#. Options for podmanager services can be set as in following sample::
[podm]
# To enable periodic task to automatically sync resources of podmanager
# with DB. By default it is set to false. (boolean value)
enable_periodic_sync = false
# Time interval(in seconds) after which devices will be synced
# periodically. By default it is set to 30. (integer value)
sync_interval = 30
To enable background synchronization of devices follow simple steps:
* Set 'enable_periodic_sync' in /etc/valence/valence.conf to true
* Set 'sync_interval' to interval value in seconds
* Restart service

29
doc/source/operators-guide/index.rst Normal file
View File

@ -0,0 +1,29 @@
..
Copyright (c) 2018 NEC, Corp.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _operator-guide:
Valence service Operators Guide
===============================
This document outlines various steps and notes for operators to know installation
and configuration options in Valence.
.. toctree::
:maxdepth: 2
installation.rst
configuration.rst

92
doc/source/operators-guide/installation.rst Normal file
View File

@ -0,0 +1,92 @@
..
Copyright 2016 Intel Corporation
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _valence-installation:
=========================
Download and Installation
=========================
The following steps capture how to install valence. All installation steps
require super user permissions.
**************************
Database etcd installation
**************************
Single node installation reference: https://github.com/coreos/etcd/releases
Distributed installation reference: https://github.com/coreos/etcd/blob/master/Documentation/op-guide/clustering.md
For development, single node installation is recommended practice.
********************
Valence Installation
********************
1. Install software dependencies
``$ sudo apt-get install git python-pip python-dev build-essential``
2. Clone the Valence code from git repo.
``$ git clone https://git.openstack.org/openstack/valence``
3. Execute the 'install_valence.sh' file present in the Valence root directory.
The install script will automatically install the dependencies listed in the
requirements.txt file.
``$ sudo bash install_valence.sh``
4. Check and set the values in valence.conf located at /etc/valence/valence.conf
as required.
5. Check the PYTHON_HOME and other variables in /etc/init/valence.conf
6. Initialize etcd database
``$ valence-db-manager init``
Note: The TypeError exception "TypeError: NoneType object is not callable"
is caused by known python-etcd bug, which will not impact this db init
functionality.
https://github.com/jplana/python-etcd/issues/190
7. Start valence service
``$ sudo service valence start``
8. Logs are located at /var/logs/valence/
**********************
Installation using pip
**********************
At the command line::
$ pip install valence
Or, if you have virtualenvwrapper installed::
$ mkvirtualenv valence
$ pip install valence
****************
GUI installation
****************
Please refer to the installation steps in the ui-README_ file.
.. _ui-README: https://github.com/openstack/valence/blob/master/ui/README.md

23
releasenotes/source/conf.py
View File

@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# Plasma Release Notes documentation build configuration file, created by
# Valence Release Notes documentation build configuration file, created by
# sphinx-quickstart on Tue Nov 3 17:40:50 2015.
#
# This file is execfile()d with the current directory set to its
@ -55,8 +55,8 @@ source_suffix = '.rst'
master_doc = 'index'
# General information about the project.
project = u'plasma Release Notes'
copyright = u'2016, Plasma Developers'
project = u'Valence Release Notes'
copyright = u'2016, Valence Developers'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
@ -189,7 +189,7 @@ html_static_path = ['_static']
# html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'PlasmaReleaseNotesdoc'
htmlhelp_basename = 'ValenceReleaseNotesdoc'
# -- Options for LaTeX output ---------------------------------------------
@ -209,8 +209,9 @@ latex_elements = {
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
('index', 'PlasmaReleaseNotes.tex', u'Plasma Release Notes Documentation',
u'Plasma Developers', 'manual'),
('index', 'ValenceReleaseNotes.tex',
u'Valence Release Notes Documentation',
u'Valence Developers', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
@ -239,8 +240,8 @@ latex_documents = [
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'plasmareleasenotes', u'Plasma Release Notes Documentation',
[u'Plasma Developers'], 1)
('index', 'valencereleasenotes', u'Valence Release Notes Documentation',
[u'Valence Developers'], 1)
]
# If true, show URL addresses after external links.
@ -253,9 +254,9 @@ man_pages = [
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'PlasmaReleaseNotes', u'Plasma Release Notes Documentation',
u'Plasma Developers', 'PlasmaReleaseNotes',
'Openstack Plasma Project',
('index', 'ValenceReleaseNotes', u'Valence Release Notes Documentation',
u'Valence Developers', 'ValenceReleaseNotes',
'Openstack Valence Project',
'Miscellaneous'),
]

2
releasenotes/source/index.rst
View File

@ -1,5 +1,5 @@
============================================
plasma Release Notes
Valence Release Notes
============================================
.. toctree::

2
simulator/pod_manager/rsd_v1_2_1/resources/__init__.py
View File

@ -1,5 +1,5 @@
import sys
sys.path.append("..")
sys.path.append("..") # noqa
import uuid

2
valence/api/route.py
View File

@ -22,7 +22,7 @@ from six.moves import http_client
# Note: setup app needs to be called before the valence imports
# for config options initialization to take place.
from valence.api import app as flaskapp
app = flaskapp.get_app()
app = flaskapp.get_app() # noqa
import valence.api.root as api_root
import valence.api.v1.devices as v1_devices

2
valence/cmd/api.py
View File

@ -16,7 +16,7 @@
import logging
import eventlet
eventlet.monkey_patch(os=False)
eventlet.monkey_patch(os=False) # noqa
import gunicorn.app.base
from valence.api.route import app as application

17
valence/common/async.py
View File

@ -35,13 +35,12 @@ def start_periodic_worker(callables):
"""Starts periodic execution of function passed in callables
To enable this:
1. Pass callables in following format
[(func, (arg1, arg2), {}),
(func2, (arg1, arg2), {}),]
2. Decorate func as follow:
@periodics.periodic(spacing=2, enabled=True)
def func():
pass
1. Pass callables in following format:
[(func, (arg1, arg2), {}), (func2, (arg1, arg2), {}),]
2. Decorate func as follow:
@periodics.periodic(spacing=2, enabled=True)
def func(): pass
:param callables: pass functions in this to execute periodically
:returns: Future object
@ -93,10 +92,8 @@ def async(func):
"""Start a job in new background thread.
To start a async job, decorate the function as follows:
Example:
@async.async
def test():
pass
def test(): pass
"""
def wrapper(*args, **kwargs):
LOG.info("starting async thread for function %s", func.__name__)

4
valence/conf/podm.py
View File

@ -44,11 +44,11 @@ podm_opts = [
'Redfish API version that valence will interact with')),
cfg.BoolOpt('enable_periodic_sync',
default=False,
help=_('To enable periodic task to automatically sync'
help=_('To enable periodic task to automatically sync '
'resources of podmanager with DB.')),
cfg.IntOpt('sync_interval',
default=30,
help=_('Time interval(in seconds) after which devices will be'
help=_('Time interval(in seconds) after which devices will be '
'synced periodically.')),
]

6
valence/controller/nodes.py
View File

@ -96,10 +96,8 @@ class Node(object):
Required JSON body:
{
'node_index': <Redfish index of node to manage>
'podm_id': <podmanager id with which node is managed>
}
{'node_index': <Redfish index of node to manage>,
'podm_id': <podmanager id with which node is managed>}
return: Info on managed node.
"""

2
valence/controller/pooled_devices.py
View File

@ -38,7 +38,9 @@ class PooledDevices(object):
"""List all registered devices
:param filters: filter by key, value arguments
Eg: {'podm_id': 'xxxx', 'type': 'SSD'}
:return: List of devices
"""
devices = db_api.Connection.list_devices(filters)

14
valence/podmanagers/expether_manager.py
View File

@ -155,6 +155,7 @@ class ExpEtherManager(object):
(updates group id of device in valence db and EEM to 4093)
:param node_id: index of the node
:raises ExpEtherException if detaching devices fails
"""
try:
@ -171,9 +172,8 @@ class ExpEtherManager(object):
:param request: Contains type_of_action(attach or detach) and
resource_id
Sample request:
{"detach":
{"resource_id": "660a95b3-adaa-42d3-ac3f-dfe7ce6c9986"}
}
{"detach":
{"resource_id": "660a95b3-adaa-42d3-ac3f-dfe7ce6c9986"}}
"""
# NOTE: type_of_action can be attach or detach
action = list(request.keys())[0]
@ -194,6 +194,7 @@ class ExpEtherManager(object):
"""Retrieves list of all connected eesv systems
:return: List of connected eesv's
:raises ExpEtherException if unable to fetch system details
"""
query = "devices/detail?status=eesv"
@ -214,6 +215,7 @@ class ExpEtherManager(object):
:param system_id: User provided system_id
:return: eesv node info
:raises ExpEtherException if unable to fetch details
"""
query = 'devices/' + system_id
@ -392,6 +394,7 @@ class ExpEtherManager(object):
Issues command to check version of EEM.
:return: on or off status of pod_manager
:raises AuthorizationFailure: if wrong credentials are passed
ExpEtherException: if any HTTPError
"""
@ -506,8 +509,9 @@ class ExpEtherManager(object):
{"ipmi_address": "xxx.xxx.xx.xx",
"ipmi_username": "xxxxx",
"ipmi_password": "xxxxx"},
"mac": "11:11:11:11:11:11",
"driver": "agent_ipmitool"}
"mac": "11:11:11:11:11:11",
"driver": "agent_ipmitool"}
:return: node and port arguments
"""
driver = param.pop('driver', 'pxe_ipmitool')

2
valence/tests/unit/api/test_route.py
View File

@ -15,7 +15,7 @@ import unittest
import mock
from valence.common import config
config.parse_args = mock.Mock()
config.parse_args = mock.Mock() # noqa
from valence.api import route