Add train cycle spec

Change-Id: I17d81b3b41ca5b956a35e7d7dc96cb0ecd39f620

doc/source/index.rst

@@ -25,6 +25,7 @@ Here you can find the specs, and spec template, for each release:
    :glob:
    :maxdepth: 1
 
+   specs/train/index
    specs/stein/index
    specs/rocky/index
    specs/queens/index

doc/source/specs/train/approved

@@ -0,0 +1 @@
+../../../../specs/train/approved/

doc/source/specs/train/implemented

@@ -0,0 +1 @@
+../../../../specs/train/implemented/

doc/source/specs/train/index.rst

@@ -0,0 +1,26 @@
+============================
+Watcher Train Specifications
+============================
+
+Template:
+
+.. toctree::
+   :maxdepth: 1
+
+   Specification Template (Train release) <template>
+
+Train implemented specs:
+
+.. toctree::
+   :glob:
+   :maxdepth: 1
+
+   implemented/*
+
+Train approved (but not implemented) specs:
+
+.. toctree::
+   :glob:
+   :maxdepth: 1
+
+   approved/*

doc/source/specs/train/redirects

@@ -0,0 +1 @@
+../../../../specs/train/redirects

doc/source/specs/train/template.rst

@@ -0,0 +1 @@
+../../../../specs/train-template.rst

priorities/train-priorities.rst

@@ -0,0 +1,12 @@
+.. train-priorities:
+
+========================
+Train Project Priorities
+========================
+
+List of priorities the Watcher drivers team is prioritizing in Train.
+
++--------------------------------------+----------------------+
+| Priority                             | Owner                |
++======================================+======================+
++--------------------------------------+----------------------+

specs/stein/approved/show-datamodel-api.rst

@@ -29,10 +29,10 @@ see the information of the instance in the specified scope.
 Proposed change
 ===============
 
-We can refer to the **"strategy list"** command, add the command line interface
+We can refer to the **"strategy list"** command, add the command line
-in the python-watcherclient. After receiving the request, the watcher-api calls
+interface in the python-watcherclient. After receiving the request,
-watcher-decision-engine to retrieve the information about the datamodel,
+the watcher-api calls watcher-decision-engine to retrieve the information
-then parses and returns.
+about the datamodel, then parses and returns.
 
 The command line interface used in watcherclient could be like this:
 
@@ -41,8 +41,8 @@ The command line interface used in watcherclient could be like this:
 In watcherclient, we can add **"datamodel.py,datamodel_shell.py"** to send
 datamodel list request and receive the result.
 
-In watcher-api, we can add **"datamodel.py"** to recieve the python-watcherclient's
+In watcher-api, we can add **"datamodel.py"** to recieve the
-request and call watcher-decision-engine module.
+python-watcherclient's request and call watcher-decision-engine module.
 
 In watcher-decision-engine, we can get datamodel data according to the
 specified scope, then parses the datamodel and return to watcher-api.
@@ -249,7 +249,8 @@ Unit test on the watcher-decision-engine, python-watcherclient, watcher-api.
 Documentation Impact
 ====================
 
-* A documentation explaining how to use **watcher datamodel list [--audit <audit_id>]**
+* A documentation explaining how to use
+  **watcher datamodel list [--audit <audit_id>]**
 
 * Update API Reference
 

specs/train-template.rst

@@ -0,0 +1,379 @@
+..
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
+ License.
+
+ http://creativecommons.org/licenses/by/3.0/legalcode
+
+==========================================
+Example Spec - The title of your blueprint
+==========================================
+
+Include the URL of your launchpad blueprint:
+
+https://blueprints.launchpad.net/watcher/+spec/example
+
+Introduction paragraph -- why are we doing anything? A single paragraph of
+prose that operators can understand. The title and this first paragraph
+should be used as the subject line and body of the commit message
+respectively.
+
+Some notes about the watcher-spec and blueprint process:
+
+* Not all blueprints need a spec. For more information see
+  http://docs.openstack.org/developer/nova/blueprints.html#specs
+
+* The aim of this document is first to define the problem we need to solve,
+  and second agree the overall approach to solve that problem.
+
+* This is not intended to be extensive documentation for a new feature.
+  For example, there is no need to specify the exact configuration changes,
+
+  nor the exact details of any DB model changes. But you should still define
+  that such changes are required, and be clear on how that will affect
+  upgrades.
+
+* You should aim to get your spec approved before writing your code.
+  While you are free to write prototypes and code before getting your spec
+  approved, its possible that the outcome of the spec review process leads
+  you towards a fundamentally different solution than you first envisaged.
+
+* But, API changes are held to a much higher level of scrutiny.
+  As soon as an API change merges, we must assume it could be in production
+  somewhere, and as such, we then need to support that API change forever.
+  To avoid getting that wrong, we do want lots of details about API changes
+  upfront.
+
+Some notes about using this template:
+
+* Your spec should be in ReSTructured text, like this template.
+
+* Please wrap text at 79 columns.
+
+* The filename in the git repository should match the launchpad URL, for
+  example a URL of: https://blueprints.launchpad.net/watcher/+spec/awesome-thing
+  should be named awesome-thing.rst
+
+* Please do not delete any of the sections in this template.  If you have
+  nothing to say for a whole section, just write: None
+
+* For help with syntax, see http://sphinx-doc.org/rest.html
+
+* To test out your formatting, build the docs using tox and see the generated
+  HTML file in doc/build/html/specs/<path_of_your_file>
+
+* If you would like to provide a diagram with your spec, ascii diagrams are
+  required.  http://asciiflow.com/ is a very nice tool to assist with making
+  ascii diagrams.  The reason for this is that the tool used to review specs is
+  based purely on plain text.  Plain text will allow review to proceed without
+  having to look at additional files which can not be viewed in gerrit.  It
+  will also allow inline feedback on the diagram itself.
+
+* If your specification proposes any changes to the Watcher REST API such
+  as changing parameters which can be returned or accepted, or even
+  the semantics of what happens when a client calls into the API, then
+  you should add the APIImpact flag to the commit message. Specifications with
+  the APIImpact flag can be found with the following query:
+
+  https://review.openstack.org/#/q/status:open+project:openstack/watcher-specs+message:apiimpact,n,z
+
+
+Problem description
+===================
+
+A detailed description of the problem. What problem is this blueprint
+addressing?
+
+Use Cases
+----------
+
+What use cases does this address? What impact on actors does this change have?
+Ensure you are clear about the actors in each use case: Developer, End User,
+Deployer etc.
+
+
+Proposed change
+===============
+
+Here is where you cover the change you propose to make in detail. How do you
+propose to solve this problem?
+
+If this is one part of a larger effort make it clear where this piece ends. In
+other words, what's the scope of this effort?
+
+At this point, if you would like to just get feedback on if the problem and
+proposed change fit in Watcher, you can stop here and post this for review to
+get preliminary feedback. If so please say:
+Posting to get preliminary feedback on the scope of this spec.
+
+Alternatives
+------------
+
+What other ways could we do this thing? Why aren't we using those? This doesn't
+have to be a full literature review, but it should demonstrate that thought has
+been put into why the proposed solution is an appropriate one.
+
+Data model impact
+-----------------
+
+Changes which require modifications to the data model often have a wider impact
+on the system.  The community often has strong opinions on how the data model
+should be evolved, from both a functional and performance perspective. It is
+therefore important to capture and gain agreement as early as possible on any
+proposed changes to the data model.
+
+Questions which need to be addressed by this section include:
+
+* What new data objects and/or database schema changes is this going to
+  require?
+
+* What database migrations will accompany this change.
+
+* How will the initial set of new data objects be generated, for example if you
+  need to take into account existing instances, or modify other existing data
+  describe how that will work.
+
+REST API impact
+---------------
+
+Each API method which is either added or changed should have the following
+
+* Specification for the method
+
+  * A description of what the method does suitable for use in
+    user documentation
+
+  * Method type (POST/PUT/GET/DELETE)
+
+  * Normal http response code(s)
+
+  * Expected error http response code(s)
+
+    * A description for each possible error code should be included
+      describing semantic errors which can cause it such as
+      inconsistent parameters supplied to the method, or when an
+      instance is not in an appropriate state for the request to
+      succeed. Errors caused by syntactic problems covered by the JSON
+      schema definition do not need to be included.
+
+  * URL for the resource
+
+  * Parameters which can be passed via the url
+
+  * JSON schema definition for the body data if allowed
+
+  * JSON schema definition for the response data if any
+
+* Example use case including typical API samples for both data supplied
+  by the caller and the response
+
+* Discuss any policy changes, and discuss what things a deployer needs to
+  think about when defining their policy.
+
+Note that the schema should be defined as restrictively as
+possible. Parameters which are required should be marked as such and
+only under exceptional circumstances should additional parameters
+which are not defined in the schema be permitted (e.g.,
+additionalProperties should be False).
+
+Reuse of existing predefined parameter types such as regexps for
+passwords and user defined names is highly encouraged.
+
+Security impact
+---------------
+
+Describe any potential security impact on the system.  Some of the items to
+consider include:
+
+* Does this change touch sensitive data such as tokens, keys, or user data?
+
+* Does this change alter the API in a way that may impact security, such as
+  a new way to access sensitive information or a new way to login?
+
+* Does this change involve cryptography or hashing?
+
+* Does this change require the use of sudo or any elevated privileges?
+
+* Does this change involve using or parsing user-provided data? This could
+  be directly at the API level or indirectly such as changes to a cache layer.
+
+* Can this change enable a resource exhaustion attack, such as allowing a
+  single API interaction to consume significant server resources? Some examples
+  of this include launching subprocesses for each connection, or entity
+  expansion attacks in XML.
+
+For more detailed guidance, please see the OpenStack Security Guidelines as
+a reference (https://wiki.openstack.org/wiki/Security/Guidelines).  These
+guidelines are a work in progress and are designed to help you identify
+security best practices.  For further information, feel free to reach out
+to the OpenStack Security Group at openstack-security@lists.openstack.org.
+
+Notifications impact
+--------------------
+
+Please specify any changes to notifications. Be that an extra notification,
+changes to an existing notification, or removing a notification.
+
+Other end user impact
+---------------------
+
+Aside from the API, are there other ways a user will interact with this
+feature?
+
+* Does this change have an impact on python-watcherclient? What does the user
+  interface there look like?
+
+Performance Impact
+------------------
+
+Describe any potential performance impact on the system, for example
+how often will new code be called, and is there a major change to the calling
+pattern of existing code.
+
+Examples of things to consider here include:
+
+* A periodic task might look like a small addition but if it calls conductor or
+  another service the load is multiplied by the number of nodes in the system.
+
+* Scheduler filters get called once per host for every instance being created,
+  so any latency they introduce is linear with the size of the system.
+
+* A small change in a utility function or a commonly used decorator can have a
+  large impacts on performance.
+
+* Calls which result in a database queries (whether direct or via conductor)
+  can have a profound impact on performance when called in critical sections of
+  the code.
+
+* Will the change include any locking, and if so what considerations are there
+  on holding the lock?
+
+Other deployer impact
+---------------------
+
+Discuss things that will affect how you deploy and configure OpenStack
+that have not already been mentioned, such as:
+
+* What config options are being added? Are the default values ones which will
+  work well in real deployments?
+
+* Is this a change that takes immediate effect after its merged, or is it
+  something that has to be explicitly enabled?
+
+* If this change is a new binary, how would it be deployed?
+
+* Please state anything that those doing continuous deployment, or those
+  upgrading from the previous release, need to be aware of. Also describe
+  any plans to deprecate configuration values or features.  For example, if we
+  change the directory name that instances are stored in, how do we handle
+  instance directories created before the change landed?  Do we move them?  Do
+  we have a special case in the code? Do we assume that the operator will
+  recreate all the instances in their cloud?
+
+Developer impact
+----------------
+
+Discuss things that will affect other developers working on OpenStack.
+
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+
+Who is leading the writing of the code? Or is this a blueprint where you're
+throwing it out there to see who picks it up?
+
+If more than one person is working on the implementation, please designate the
+primary author and contact.
+
+Primary assignee:
+  <launchpad-id or None>
+
+Other contributors:
+  <launchpad-id or None>
+
+Work Items
+----------
+
+Work items or tasks -- break the feature up into the things that need to be
+done to implement it. Those parts might end up being done by different people,
+but we're mostly trying to understand the timeline for implementation.
+
+
+Dependencies
+============
+
+* Include specific references to specs and/or blueprints in Watcher, or in
+  other projects, that this one either depends on or is related to.
+
+* If this requires functionality of another project that is not currently used
+  by Watcher (such as the glance v2 API when we previously only required v1),
+  document that fact.
+
+* Does this feature require any new library dependencies or code otherwise not
+  included in OpenStack? Or does it depend on a specific version of library?
+
+
+Testing
+=======
+
+Please discuss the important scenarios needed to test here, as well as
+specific edge cases we should be ensuring work correctly. For each
+scenario please specify if this requires specialized hardware, a full
+openstack environment, or can be simulated inside the Watcher tree.
+
+Please discuss how the change will be tested. We especially want to know what
+tempest tests will be added. It is assumed that unit test coverage will be
+added so that doesn't need to be mentioned explicitly, but discussion of why
+you think unit tests are sufficient and we don't need to add more tempest
+tests would need to be included.
+
+Is this untestable in gate given current limitations (specific hardware /
+software configurations available)? If so, are there mitigation plans (3rd
+party testing, gate enhancements, etc).
+
+
+Documentation Impact
+====================
+
+What is the impact on the docs team of this change? Some changes might require
+donating resources to the docs team to have the documentation updated. Don't
+repeat details discussed above, but please reference them here.
+
+
+References
+==========
+
+Please add any useful references here. You are not required to have any
+reference. Moreover, this specification should still make sense when your
+references are unavailable. Examples of what you could include are:
+
+* Links to mailing list or IRC discussions
+
+* Links to notes from a summit session
+
+* Links to relevant research, if appropriate
+
+* Related specifications as appropriate (e.g.  if it's an EC2 thing, link the
+  EC2 docs)
+
+* Anything else you feel it is worthwhile to refer to
+
+
+History
+=======
+
+Optional section for liberty intended to be used each time the spec
+is updated to describe new design, API or any database schema
+updated. Useful to let reader understand what's happened along the
+time.
+
+.. list-table:: Revisions
+   :header-rows: 1
+
+   * - Release Name
+     - Description
+   * - Train
+     - Introduced
+

specs/train/approved/show-datamodel-api.rst

@@ -0,0 +1,269 @@
+..
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
+ License.
+
+ http://creativecommons.org/licenses/by/3.0/legalcode
+
+======================
+Add Show Datamodel API
+======================
+
+https://blueprints.launchpad.net/watcher/+spec/show-datamodel-api
+
+
+Problem description
+===================
+
+The datamodel is very important for Watcher to generate resource
+optimization solutions. Currently, it can only be found by looking at
+the log file, which is very inconvenient. Therefore, it is necessary
+to add an api to facilitate the user to quickly view the datamodel.
+
+Use Cases
+----------
+
+As a Watcher user, I want to use Show Datamodel API to
+see the information of the instance in the specified scope.
+
+
+Proposed change
+===============
+
+We can refer to the **"strategy list"** command, add the command line
+interface in the python-watcherclient. After receiving the request,
+the watcher-api calls watcher-decision-engine to retrieve the information
+about the datamodel, then parses and returns.
+
+The command line interface used in watcherclient could be like this:
+
+* watcher datamodel list [--audit <audit_id>]
+
+In watcherclient, we can add **"datamodel.py,datamodel_shell.py"** to send
+datamodel list request and receive the result.
+
+In watcher-api, we can add **"datamodel.py"** to recieve the
+python-watcherclient's request and call watcher-decision-engine module.
+
+In watcher-decision-engine, we can get datamodel data according to the
+specified scope, then parses the datamodel and return to watcher-api.
+
+Alternatives
+------------
+
+None
+
+Data model impact
+-----------------
+
+None
+
+REST API impact
+---------------
+
+Add following **datamodel** REST:
+
+* GET /v1/datamodels
+
+  * Normal http response code(200)
+  * Expected error http response code(400,401)
+
+* Request
+
+  * **audit_uuid (Optional)**: UUID of an audit
+
+* Response
+
+  * **instance_uuid**: UUID of an instance
+  * **node_uuid**: UUID of an compute node
+  * **instance_state**: state of instance
+  * **node_state**: state of compute node
+
+* GET /v1/datamodels/detail
+
+  * Normal http response code(200)
+  * Expected error http response code(400,401)
+
+* Request
+
+  * **audit_uuid (Optional)**: UUID of an audit
+
+* Response
+
+  * **instance_uuid**: UUID of an instance
+  * **instance_state**: state of instance
+  * **instance_name**: name of instance
+  * **instance_vcpus**: number of instance vcpus
+  * **instance_memory**: memory of instance
+  * **instance_disk**: disk of instance
+  * **instance_disk_capacity**: disk capacity of instance
+  * **node_uuid**: UUID of an compute node
+  * **node_state**: state of compute node
+  * **node_name**: name of node
+  * **node_vcpus**: number of compute node vcpus
+  * **node_memory**: memory of node
+  * **node_disk**: disk of node
+  * **node_disk_capacity**: disk capacity of node
+
+
+* Example JSON representation of Datamodel
+
+::
+
+  {
+    "compute": [
+      {
+        "node_uuid": "90d7da5c-d432-4eba-89b4-743c9f1e6cfa",
+        "node_name": "node_1",
+        "node_vcpus": 48,
+        "node_memory": "4096",
+        "node_disk": "40",
+        "node_disk_capacity": "60"
+        "servers": [
+          {
+            "instance_uuid": "9e7cbe91-b391-4394-a42c-68996a4fd555",
+            "instance_state": "active",
+            "instance_name": "vm_4",
+            "instance_vcpus": 16,
+            "instance_memory": "2048",
+            "instance_disk": "10",
+            "instance_disk_capacity": "35",
+          },
+          {
+            "instance_uuid": "8e7cbe91-b391-4394-a42c-68996a4fd555",
+            "instance_state": "active",
+            "instance_name": "vm_5",
+            "instance_vcpus": 16,
+            "instance_memory": "2048",
+            "instance_disk": "10",
+            "instance_disk_capacity": "35",
+          }
+        ]
+      },
+      {
+        "node_uuid": "78d7da5c-d432-4eba-89b4-743c9f1e6cfa",
+        "node_name": "node_2",
+        "node_vcpus": 96,
+        "node_memory": "4096",
+        "node_disk": "60",
+        "node_disk_capacity": "60"
+        "servers": [
+          {
+            "instance_uuid": "6b7cbe91-b391-4394-a42c-68996a4fd55b",
+            "instance_state": "active",
+            "instance_name": "vm_1",
+            "instance_vcpus": 32,
+            "instance_memory": "2048",
+            "instance_disk": "10",
+            "instance_disk_capacity": "35",
+          },
+          {
+            "instance_uuid": "527cbe91-b391-4394-a42c-68996a4fd5e7",
+            "instance_state": "active",
+            "instance_name": "vm_2",
+            "instance_vcpus": 16,
+            "instance_memory": "2048",
+            "instance_disk": "10",
+            "instance_disk_capacity": "35",
+          }
+        ]
+      }
+    ]
+  }
+
+Security impact
+---------------
+
+None
+
+Notifications impact
+--------------------
+
+None
+
+Other end user impact
+---------------------
+
+The user can view the datamodel through the command below
+in python-watcherclient:
+
+* watcher datamodel list
+
+and add the **audit** parameter to filter the datamodel in the
+specified scope:
+
+* watcher datamodel list [--audit <audit_id>]
+
+Performance Impact
+------------------
+
+None
+
+Other deployer impact
+---------------------
+
+None
+
+Developer impact
+----------------
+
+None
+
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+
+Primary assignee:
+  <chenker>
+
+Other contributors:
+  <li-canwei2> , <yumeng-bao>
+
+Work Items
+----------
+
+- Add **watcher datamodel list** command line interface
+  in watcherclient.
+
+- Add verification and processing of request from
+  watcherclient in watcher-api.
+
+- Add parsing, encapsulation, and return of datamodel
+  in watcher-decision-engine.
+
+
+Dependencies
+============
+
+None
+
+
+Testing
+=======
+
+Unit test on the watcher-decision-engine, python-watcherclient, watcher-api.
+
+
+Documentation Impact
+====================
+
+* A documentation explaining how to use
+  **watcher datamodel list [--audit <audit_id>]**
+
+* Update API Reference
+
+* Update REST API Version History
+
+
+References
+==========
+
+None
+
+
+History
+=======
+
+None

specs/train/approved/train-template.rst

@@ -0,0 +1 @@
+../../train-template.rst

specs/train/implemented/train-template.rst

@@ -0,0 +1 @@
+../../train-template.rst