Add unit tests for Trove specs

Add simple unit tests to the Trove specs repo which ensure that
specifications submitted follow the template correctly and address
all the needed sections.

Update tox to run these unit tests automatically.

At the beginning of each release, test_titles.py will need to be updated
and 'current_release' modified to reflect the new release directory.  At
the same time, the new release directory will be created, the index.rst
file in doc/source ammended to include this directory, and any unmerged
specs moved there.  In this way, changes to the template will only
affect specs going forward and avoids having to modify any specs that
have merged in previous releases.

(It will also have the side effect of signalling the 'opening' of the
release to new specs.)

Change-Id: I8f1175bceed63d36d96b6c6c6b4188ef27d3102e

requirements.txt

@@ -1,4 +1,6 @@
 pbr>=0.6,<1.0
 oslosphinx
 sphinx>=1.1.2,<1.2
+testrepository>=0.0.18
+testtools>=0.9.34
 yasfb>=0.5.1

specs/kilo/add-instance-name-to-cli.rst

@@ -7,7 +7,6 @@
  Sections of this template were taken directly from the Nova spec
  template at:
  https://github.com/openstack/nova-specs/blob/master/specs/template.rst
-..
 
 =======================================================
  Add instance name as parameter to various CLI commands
@@ -20,7 +19,7 @@ https://blueprints.launchpad.net/trove/+spec/add-instance-name-to-cli
 The proposal is to allow instance-name to be specified wherever instance-ID
 is currently used throughout the CLI.
 
-Problem description
+Problem Description
 ===================
 
 Currently, only the "trove show" command will take instance-ID or
@@ -30,7 +29,7 @@ the instance-ID be used. It would be helpful to customers to be able
 to use instance-ID or instance-name interchangeably throughtout the CLI.
 
 
-Proposed change
+Proposed Change
 ===============
 
 Allow instance-id or instance-name to be passed in for the following commands:
@@ -111,6 +110,11 @@ Public API
 
 None
 
+Public API Security
+-------------------
+
+None
+
 Internal API
 ------------
 

specs/kilo/associate-flavors-datastores.rst

@@ -14,32 +14,45 @@ Launchpad blueprint:
 
 https://blueprints.launchpad.net/trove/+spec/associate-flavors-datastores
 
-Motivation: Each datastore type has its own hardware requirements (minimum and maximum). Currently, trove lacks the ability to enforce this requirement. This change enables trove with the ability to associate the virtual hardware templates (flavors) with datastore types.
+Motivation:
+Each datastore type has its own hardware requirements (minimum and
+maximum). Currently, trove lacks the ability to enforce this
+requirement. This change enables trove with the ability to associate
+the virtual hardware templates (flavors) with datastore types.
 
-Problem description
+Problem Description
 ===================
 
-Trove supports multiple datastore types. However, each datastore type has its own hardware requirements, which presently, cannot be enforced in trove.
+Trove supports multiple datastore types. However, each datastore type
+has its own hardware requirements, which presently, cannot be enforced
+in trove.
 
 For example:
 MySQL - Minimum System Requirements:
-
 - 2 or more CPU cores
 - 2 or more GB of RAM
 - Disk I/O subsystem applicable for a write-intensive database
 
-Find more information about other datastore specific hardware requirements in the References section.
+Find more information about other datastore specific hardware
+requirements in the References section.
 
-Trove uses virtual hardware templates called 'flavors' in OpenStack. In order for trove to be able to enforce the datastore specific hardware requirements, there needs to be a way in which datastore types can be associated with their minimum flavor (hardware) requirements. This way, the user/administrator can at least be notified while provisioning the datastore with the inappropriate flavor, that does not meet the minimum hardware requirements.
 
-Proposed change
+Trove uses virtual hardware templates called 'flavors' in
+OpenStack. In order for trove to be able to enforce the datastore
+specific hardware requirements, there needs to be a way in which
+datastore types can be associated with their minimum flavor (hardware)
+requirements. This way, the user/administrator can at least be
+notified while provisioning the datastore with the inappropriate
+flavor, that does not meet the minimum hardware requirements.
+
+
+Proposed Change
 ===============
 
-- The trove-manage utility will provide the ability:
+- The trove-manage utility will provide the ability to add and delete
-
+  the datastore version-flavor associations.
- - to add and delete the datastore version-flavor associations.
+- There will be an additional API call in order to get a list of
-
+  flavors for the specified datastore version id.
-- There will be an additional API call in order to get a list of flavors for the specified datastore version id.
 
 
 Configuration
@@ -50,7 +63,12 @@ None
 Database
 --------
 No impact on existing tables.
-One new entity will be created in the trove database: datastore_version_metadata. This will store any additional metadata related to a datastore version including its relation with flavors: key=flavor and the value=flavor_id.
+
+One new entity will be created in the trove database:
+datastore_version_metadata. This will store any additional metadata
+related to a datastore version including its relation with flavors:
+key=flavor and the value=flavor_id.
+
 The blueprint specifies the table attributes in detail.
 
 Public API
@@ -58,6 +76,11 @@ Public API
 
 Does not impact any other API which the user has access to.
 
+Public API Security
+-------------------
+
+None
+
 Internal API
 ------------
 
@@ -94,9 +117,12 @@ Work Items
 ----------
 
 - Add the new database schema for datastore version metadata.
-- trove-manage util extension to associate a flavor list with a datastore version id
+- trove-manage util extension to associate a flavor list with a
-- trove-manage util extension to delete a flavor associated with a datastore version id.
+  datastore version id
-- REST API call to list flavors for a datastore version id (/{tenant_id}/flavors/{datastore_version_id})
+- trove-manage util extension to delete a flavor associated with a
+  datastore version id.
+- REST API call to list flavors for a datastore version id
+  (/{tenant_id}/flavors/{datastore_version_id})
 - Unit and integration tests
 
 
@@ -116,10 +142,12 @@ Documentation Impact
 ====================
 
 - New API call to list flavors given a datastore version id.
-- trove-manage util extensions to add/delete datastore version - flavor associations.
+- trove-manage util extensions to add/delete datastore version -
+  flavor associations.
 
 
 References
 ==========
 
-* Further hardware requirements for different datastore types: https://wiki.openstack.org/wiki/TroveFlavorsPerDatastore
+* Further hardware requirements for different datastore types:
+  https://wiki.openstack.org/wiki/TroveFlavorsPerDatastore

specs/kilo/datastore-visibility.rst

@@ -18,7 +18,7 @@ need for Trove to have a greater control over the customer visibility
 of these datastore types. This change enables Trove to control the
 visibility of the various datastore types.
 
-Problem description
+Problem Description
 ===================
 
 There might be some datastore types, which the deployers require to be
@@ -26,7 +26,7 @@ active but not visible to customers in the production environment.
 Example use case: Say we want to have an active datastore A in production
 and not expose it to customers yet.
 
-Proposed change
+Proposed Change
 ===============
 
 This change suggests adding a visibility attribute to the datastore
@@ -108,11 +108,13 @@ Table datastore_versions::
 Table datastore_version_members::
 
     Field               | Type        | Null | Key         | Default | Extra|
-    -----------------------------------------------------------------------------------------|
+    ------------------------------------------------------------------------|
     id                  | varchar(36) | NO   | PRIMARY     | NULL    |      |
-    datastore_version_id| varchar(36) | YES  | Foreign(datastore_version.id)| NULL    |      |
+    datastore_version_id| varchar(36) | YES  | Foreign [1] | NULL    |      |
     tenant_id           | varchar(36) | NO   |             | NULL    |      |
 
+    [1] datastore_version_id is a Foreign-Key on datastore_version.id
+
 Public API
 ----------
 
@@ -175,12 +177,14 @@ Admin calls related to datastore version members
 
 2. Delete a member:
 
-  DELETE /{tenant_id}/mgmt/datastores/{datastore}/versions/{id}/members/{member_id}
+  DELETE
+  /{tenant_id}/mgmt/datastores/{datastore}/versions/{id}/members/{member_id}
 
 
 3. Get a member:
 
-   GET  /{tenant_id}/mgmt/datastores/{datastore}/versions/{id}/members/{member_id}
+   GET
+   /{tenant_id}/mgmt/datastores/{datastore}/versions/{id}/members/{member_id}
 
    Response::
 
@@ -236,6 +240,10 @@ Admin calls related to datastore version members
        ]
     }
 
+Public API Security
+-------------------
+
+None
 
 Internal API
 ------------

specs/kilo/example-generator.rst

@@ -1,6 +1,8 @@
-This work is licensed under a Creative Commons Attribution 3.0 Unported License.
+..
+  This work is licensed under a Creative Commons Attribution 3.0
+  Unported License.
 
-http://creativecommons.org/licenses/by/3.0/legalcode
+  http://creativecommons.org/licenses/by/3.0/legalcode
 
 =========================
 Example Snippet Generator
@@ -38,6 +40,37 @@ will get run just the same as if the tests had executed against a fully stood
 up Trove environment, with the advantage that certain UUIDs can be altered to
 avoid them changing with every test run.
 
+Configuration
+-------------
+
+None
+
+Database
+--------
+
+None
+
+Public API
+----------
+
+None
+
+Public API Security
+-------------------
+
+None
+
+Internal API
+------------
+
+None
+
+Guest Agent
+-----------
+
+None
+
+
 Data Model Impact
 -----------------
 None.
@@ -97,6 +130,12 @@ Assignee(s)
 Primary assignee:
     Tim Simpson
 
+Milestones
+----------
+
+Target Milestone for completion:
+    Kilo-1
+
 Work Items
 ----------
 * Implement the snippets generator.
@@ -116,7 +155,7 @@ NA
 Functional Tests
 ----------------
 NA
-`
+
 API Tests
 ---------
 NA

specs/kilo/guest-rpc-ping-pong.rst

@@ -34,7 +34,7 @@ Use Cases
 * As a deployer, I want to be able to determine whether RPC connectivity
   between the control-plane and a specific guest is broken.
 
-Proposed change
+Proposed Change
 ===============
 
 **Introduce**::
@@ -82,6 +82,11 @@ Public API
 
 No public API changes.
 
+Public API Security
+-------------------
+
+No public API Security related changes.
+
 
 Management API
 --------------

specs/kilo/mgmt-show-deleted.rst

@@ -35,7 +35,7 @@ Use Cases
 * As a deployer, I want to be able to retrieve information about a deleted
   Trove instance.
 
-Proposed change
+Proposed Change
 ===============
 
 Support the ``?deleted=true/false`` query parameter in
@@ -70,6 +70,10 @@ Public API
 
 No public API changes.
 
+Public API Security
+-------------------
+
+No public API Security related changes.
 
 Management API
 --------------

specs/kilo/native-os-heat-resources.rst

@@ -7,7 +7,7 @@
  Sections of this template were taken directly from the Nova spec
  template at:
  https://github.com/openstack/nova-specs/blob/master/specs/template.rst
-..
+
 
   This template should be in ReSTructured text. The filename in the git
   repository should match the launchpad URL, for example a URL of
@@ -23,7 +23,7 @@ Use native OS::* Heat resources for internal orchestration
 
 https://blueprints.launchpad.net/trove/+spec/native-os-heat-resources
 
-Problem description
+Problem Description
 ===================
 
 A far-fetching goal in Trove is to use OpenStack Orchestration service (Heat)
@@ -38,7 +38,7 @@ and HOT template format to fully leverage available OpenStack functionality,
 as they do not have to keep compatibility with AWS CloudFormation service.
 
 
-Proposed change
+Proposed Change
 ===============
 
 Align default Heat template with latest changes in Heat. That involves:
@@ -62,6 +62,11 @@ Public API
 
 None
 
+Public API Security
+-------------------
+
+No public API Security related changes.
+
 Internal API
 ------------
 

specs/kilo/placeholder.rst

@@ -1,3 +0,0 @@
-======================
-Placeholder Kilo Specs
-======================

specs/kilo/rsync-optional.rst

@@ -33,7 +33,7 @@ Use Cases
 * As a deployer, I want to avoid ssh connectivity between guests and the
   control-plane.
 
-Proposed change
+Proposed Change
 ===============
 
 Add additional elements in trove-integration to stage the guestagent code
@@ -61,6 +61,11 @@ Public API
 
 No public API changes.
 
+Public API Security
+-------------------
+
+No public API Security related changes.
+
 Internal API
 ------------
 

specs/kilo/switch-to-oslo-messaging.rst

@@ -7,20 +7,18 @@
  Sections of this template were taken directly from the Nova spec
  template at:
  https://github.com/openstack/nova-specs/blob/master/specs/template.rst
-..
+
 
 ========================
 Switch to OSLO Messaging
 ========================
 
-Include the URL of your launchpad blueprint:
-
 https://wiki.openstack.org/wiki/Trove-rpc-versioning
 
 This blueprint is to document the project to switch trove to use OSLO
 Messaging.
 
-Problem description
+Problem Description
 ===================
 
 Get Trove on the same page w/ other projects + oslo. This will require
@@ -28,25 +26,32 @@ refactoring the rpc layer
 
 Prevent backward incompatibility between Trove components
 
-* The rpc client can send an older version of a message/call to a newer message handler without issue.
+* The rpc client can send an older version of a message/call to a
-* The rpc client can send a newer version of a message/call to an older message handler without killing it (need to confirm this).
+  newer message handler without issue.
-* The rpc client can check to see if it can send a newer version of a message/call.
+* The rpc client can send a newer version of a message/call to an
+  older message handler without killing it (need to confirm this).
+* The rpc client can check to see if it can send a newer version of a
+  message/call.
 
 * Reduce the need for downtime during deployments.
 
-** Effectively allows for a mix of versions between the control plane and guest agents (in most cases). There may still be updates that will require down time.
+** Effectively allows for a mix of versions between the control plane
+   and guest agents (in most cases). There may still be updates that
+   will require down time.
 
-Proposed change
+Proposed Change
 ===============
 
 * Remove rpc common components from oslo incubator?
 * Use oslo messaging library
 
-** The minimum version will be picked from global-requirements. At the time of this writing, it's >= 1.4.0
+** The minimum version will be picked from global-requirements. At the
+   time of this writing, it's >= 1.4.0
 
 *  Keep track of a "version history" in comments in the code
 
-* Update trove calls to the openstack.common.rpc client to include a version cap param. (This is already supported in the client)
+* Update trove calls to the openstack.common.rpc client to include a
+  version cap param. (This is already supported in the client)
 
 ** Trove API    --> Task Manager
 ** Trove API    --> Guest Agent
@@ -58,7 +63,9 @@ Proposed change
 Configuration
 -------------
 
-oslo.messaging has been designed to be backwards compatible config-wise. All the config compatibilities will be kept until all the projects have been migrated to oslo.messaging.
+oslo.messaging has been designed to be backwards compatible
+config-wise. All the config compatibilities will be kept until all the
+projects have been migrated to oslo.messaging.
 
 Database
 --------
@@ -70,6 +77,11 @@ Public API
 
 NA
 
+Public API Security
+-------------------
+
+NA
+
 Internal API
 ------------
 

specs/liberty/cassandra-backup-restore.rst

@@ -181,7 +181,8 @@ Liberty-1
 Work Items
 ----------
 
-1. Implement functionality needed for resetting cluster name and superuser password.
+1. Implement functionality needed for resetting cluster name and superuser
+   password.
 2. Implement backup/restore API calls.
 
 Upgrade Implications

specs/liberty/cassandra-configuration-groups.rst

@@ -49,6 +49,16 @@ Public API Security
 
 None
 
+Python API
+----------
+
+None (empty section added after merging)
+
+CLI (python-troveclient)
+------------------------
+
+None (empty section added after merging)
+
 Internal API
 ------------
 

specs/liberty/cassandra-database-user-functions.rst

@@ -98,6 +98,16 @@ to a random Trove password which is then stored in a Trove-read-only
 file in '~/.cassandra/cqlshrc' which is also the default location for
 client settings.
 
+Python API
+----------
+
+None (empty section added after merging)
+
+CLI (python-troveclient)
+------------------------
+
+None (empty section added after merging)
+
 Internal API
 ------------
 

specs/liberty/mongodb-backup-restore.rst

@@ -17,9 +17,10 @@ for single instances.
 
 https://blueprints.launchpad.net/trove/+spec/backups-single-instance-mongodb
 
-There is also a previous blueprint that incorporated single-instance backup [7]_.
+There is also a previous blueprint that incorporated single-instance backup
+[7]_.
 
-Problem description
+Problem Description
 ===================
 
 Backup and restore capability is important for eventually moving the MongoDB
@@ -30,7 +31,7 @@ to provide a standard backup and restore strategy for the MongoDB datastore,
 equivalent to the MySQLDump strategy for MySQL-based datastores.
 
 
-Proposed change
+Proposed Change
 ===============
 
 An initial simple backup and restore strategy for MongoDB will be implemented
@@ -63,6 +64,16 @@ Public API Security
 
 None.
 
+Python API
+----------
+
+None (empty section added after merging)
+
+CLI (python-troveclient)
+------------------------
+
+None (empty section added after merging)
+
 Internal API
 ------------
 
@@ -104,8 +115,8 @@ trove/guestagent/strategies/restore/experimental/mongo_impl.py
 Alternatives
 ------------
 
-As of version 3.0, MongoDB supports consistent, point-in-time snapshots with the
+As of version 3.0, MongoDB supports consistent, point-in-time snapshots with
-use of --oplog to the mongodump command for single-server deployments.
+the use of --oplog to the mongodump command for single-server deployments.
 Operations committed after the start of the mongodump command are logged to a
 separate file, and this file can be replayed with the --oplogReplay parameter
 to mongorestore. As previous versions of MongoDB, and sharded installations do

specs/liberty/mongodb-configuration-groups.rst

@@ -15,12 +15,12 @@ MongoDB Configuration Groups
 Launchpad Blueprint:
 https://blueprints.launchpad.net/trove/+spec/mongodb-configuration-groups
 
-Problem description
+Problem Description
 ===================
 
 The MongoDB guestagent currently does not support configuration groups.
 
-Proposed change
+Proposed Change
 ===============
 
 The patch set will implement configuration groups for MongoDB 2.6 and above.
@@ -162,6 +162,16 @@ Public API Security
 
 None
 
+Python API
+----------
+
+None (empty section added after merging)
+
+CLI (python-troveclient)
+------------------------
+
+None (empty section added after merging)
+
 Internal API
 ------------
 

specs/liberty/mongodb-database.rst

@@ -18,14 +18,14 @@ https://blueprints.launchpad.net/trove/+spec/mongodb-database
 Enable MongoDB database management functionality.
 
 
-Problem description
+Problem Description
 ===================
 
 The MongoDB datastore does not support database management features. Allowing
 the user to create, list, and delete databases through the API is essential.
 
 
-Proposed change
+Proposed Change
 ===============
 
 Three standard Trove commands will be enabled for MongoDB:
@@ -68,6 +68,16 @@ Public API Security
 No API Security changes.
 
 
+Python API
+----------
+
+None (empty section added after merging)
+
+CLI (python-troveclient)
+------------------------
+
+None (empty section added after merging)
+
 Internal API
 ------------
 
@@ -93,7 +103,8 @@ The Guest Agent will be changed to support the following manager functions:
    inserted into them. To ensure a database is created, a dummy document will
    be inserted but then deleted.
 
-2. list_databases - Run 'pymongo.MongoClient.database_names()' and return the resulting list.
+2. list_databases - Run 'pymongo.MongoClient.database_names()' and return the
+   resulting list.
 
 3. delete_database - Drop the database with
    'pymongo.MongoClient.drop_database("<dbname>")'. Users associated with the

specs/liberty/mongodb-users.rst

@@ -18,20 +18,20 @@ https://blueprints.launchpad.net/trove/+spec/mongodb-users
 Enable MongoDB users management functionality.
 
 
-Problem description
+Problem Description
 ===================
 
 The MongoDB guest agent does not support users management features. Allowing
 the user to create, list, and delete users, enable/check root access, and
 grant/revoke user access through the API is essential.
 
-MongoDB users are unique to each database. A user is identified by both its name
+MongoDB users are unique to each database. A user is identified by both its
-and database. Therefore the Trove actions requiring user names need
+name and database. Therefore the Trove actions requiring user names need
 the value to be in the following string format: "<database>.<username>".
 MongoDB does not allow the following characters in the database name: /\. "$
 
 
-Proposed change
+Proposed Change
 ===============
 
 Calls to MongoDB will be done in Python via the PyMongo library, which is
@@ -136,6 +136,16 @@ Public API Security
 No API Security changes.
 
 
+Python API
+----------
+
+None (empty section added after merging)
+
+CLI (python-troveclient)
+------------------------
+
+None (empty section added after merging)
+
 Internal API
 ------------
 

specs/liberty/redis-backup-restore.rst

@@ -212,7 +212,7 @@ Milestones
 Target Milestone for completion:
   Liberty-1
 
-Work Item
+Work Items
 ----------
 
 The API calls for backup and restore need to be implemented.

specs/liberty/redis-configuration-groups.rst

@@ -15,12 +15,12 @@ Redis Configuration Groups
 Launchpad Blueprint:
 https://blueprints.launchpad.net/trove/+spec/redis-configuration-groups
 
-Problem description
+Problem Description
 ===================
 
 The Redis guestagent currently does not support configuration groups.
 
-Proposed change
+Proposed Change
 ===============
 
 The patch set will implement configuration groups for Redis 3.0 or above.
@@ -162,6 +162,16 @@ Public API Security
 
 None
 
+Python API
+----------
+
+None (empty section added after merging)
+
+CLI (python-troveclient)
+------------------------
+
+None (empty section added after merging)
+
 Internal API
 ------------
 

specs/template.rst

@@ -21,9 +21,9 @@
     above should remain.
 
 
-=================
+============================
-Title of the Spec
+Template for New Trove Specs
-=================
+============================
 
 .. If section numbers are desired, unindent this
     .. sectnum::
@@ -32,6 +32,7 @@ Title of the Spec
     .. contents::
 
 Introduction paragraph -- what is the motivation for the spec/blueprint?
+(Don't forget to change the title above to something more relevant.)
 
 Launchpad Blueprint:
 https://blueprints.launchpad.net/trove/+spec/name-of-blueprint
@@ -61,6 +62,20 @@ the APIImpact flag can be found with the following query:
 https://review.openstack.org/#/q/status:open+project:openstack/trove-specs+message:apiimpact,n,z
 
 
+Code snippets, etc. should be placed in appropriately marked blocks:
+
+.. code-block:: bash
+
+    # This is a bash command
+    ls -lf
+
+.. code-block:: python
+
+    # sample code
+    for count in range(1, 10):
+        print count
+
+
 Configuration
 -------------
 
@@ -180,6 +195,11 @@ If the change has upgrade implications, also remember to:
 For more information about the DocImpact keyword, refer to
 https://wiki.openstack.org/wiki/Documentation/DocImpact
 
+Note: Documentation for the CLI commands are automatically generated
+from the help strings when a new version of the CLI is released, so
+a DocImpact keyword is not typically required for python-troveclient
+changes.
+
 
 Dependencies
 ============
@@ -214,12 +234,13 @@ References
 
 Please add any useful references here. You are not required to have any
 references. Moreover, this specification should still make sense when your
-references are unavailable. Examples of what you could include are:
+references are unavailable. Examples of what you could include are [1]_,
+[2]_, [3]_ and [4]_.
 
-* Links to mailing list or IRC discussions
+.. [1] Links to mailing list or IRC discussions
 
-* Links to notes from a summit session
+.. [2] Links to notes from a summit session
 
-* Links to relevant research, if appropriate
+.. [3] Links to relevant research, if appropriate
 
-* Anything else you feel it is worthwhile to refer to
+.. [4] Anything else you feel it is worthwhile to refer to

tests/test_titles.py

@@ -0,0 +1,112 @@
+# Copyright [2015] Hewlett-Packard Development Company, L.P.
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import glob
+import re
+
+import docutils.core
+import testtools
+
+
+class TestTitles(testtools.TestCase):
+
+    current_release = 'liberty'
+
+    def _get_title(self, section_tree):
+        section = {
+            'subtitles': [],
+        }
+        for node in section_tree:
+            if node.tagname == 'title':
+                section['name'] = node.rawsource
+            elif node.tagname == 'section':
+                subsection = self._get_title(node)
+                section['subtitles'].append(subsection['name'])
+        return section
+
+    def _get_titles(self, spec):
+        titles = {}
+        for node in spec:
+            if node.tagname == 'section':
+                # Note subsection subtitles are thrown away
+                section = self._get_title(node)
+                titles[section['name']] = section['subtitles']
+        return titles
+
+    def _check_titles(self, filename, expect, actual):
+        missing_sections = [x for x in expect.keys() if x not in actual.keys()]
+        extra_sections = [x for x in actual.keys() if x not in expect.keys()]
+
+        msgs = []
+        if len(missing_sections) > 0:
+            msgs.append("Missing sections: %s" % missing_sections)
+        if len(extra_sections) > 0:
+            msgs.append("Extra sections: %s" % extra_sections)
+
+        for section in expect.keys():
+            if section in actual:
+                missing_subsections = [x for x in expect[section]
+                                       if x not in actual[section]]
+                # extra subsections are allowed
+                if len(missing_subsections) > 0:
+                    msgs.append("Section '%s' is missing subsections: %s"
+                                % (section, missing_subsections))
+
+        if len(msgs) > 0:
+            self.fail("While checking '%s':\n  %s"
+                      % (filename, "\n  ".join(msgs)))
+
+    def _check_lines_wrapping(self, tpl, raw):
+        for i, line in enumerate(raw.split("\n")):
+            # ignore lines that have more than one '/' - this will exclude
+            # web addresses and long file system paths.
+            if line.count('/') > 1:
+                continue
+            self.assertTrue(
+                len(line) < 80,
+                msg="%s:%d: Line limited to a maximum of 79 characters." %
+                (tpl, i+1))
+
+    def _check_no_cr(self, tpl, raw):
+        matches = re.findall('\r', raw)
+        self.assertEqual(
+            0, len(matches),
+            "Found %s literal carriage returns in file %s" %
+            (len(matches), tpl))
+
+    def _check_trailing_spaces(self, tpl, raw):
+        for i, line in enumerate(raw.split("\n")):
+            trailing_spaces = re.findall(" +$", line)
+            self.assertEqual(
+                0, len(trailing_spaces),
+                "Found trailing spaces on line %s of %s" % (i+1, tpl))
+
+    def test_template(self):
+        with open("specs/template.rst") as f:
+            template = f.read()
+        spec = docutils.core.publish_doctree(template)
+        template_titles = self._get_titles(spec)
+
+        files = glob.glob("specs/%s/*" % self.current_release)
+        for filename in files:
+            self.assertTrue(filename.endswith(".rst"),
+                            "spec's file must uses 'rst' extension.")
+            with open(filename) as f:
+                data = f.read()
+
+            spec = docutils.core.publish_doctree(data)
+            titles = self._get_titles(spec)
+            self._check_titles(filename, template_titles, titles)
+            self._check_lines_wrapping(filename, data)
+            self._check_no_cr(filename, data)
+            self._check_trailing_spaces(filename, data)

tox.ini

@@ -1,15 +1,14 @@
 [tox]
 minversion = 1.6
-envlist = docs
+envlist = docs,py27
 skipsdist = True
 
 [testenv]
 usedevelop = True
 install_command = pip install -U {opts} {packages}
-setenv =
+setenv = VIRTUAL_ENV={envdir}
-   VIRTUAL_ENV={envdir}
 deps = -r{toxinidir}/requirements.txt
-       -r{toxinidir}/test-requirements.txt
+commands = python setup.py testr --slowest --testr-args='{posargs}'
 
 [testenv:venv]
 commands = {posargs}