diff --git a/specs/storyboard_integration_tests.rst b/specs/storyboard_integration_tests.rst new file mode 100644 index 0000000..9018200 --- /dev/null +++ b/specs/storyboard_integration_tests.rst @@ -0,0 +1,142 @@ +:: + + This work is licensed under a Creative Commons Attribution 3.0 + Unported License. + http://creativecommons.org/licenses/by/3.0/legalcode + +.. + This template should be in ReSTructured text. Please do not delete + any of the sections in this template. If you have nothing to say + for a whole section, just write: "None". For help with syntax, see + http://sphinx-doc.org/rest.html To test out your formatting, see + http://www.tele3.cz/jbar/rest/rest.html + +============================ +StoryBoard integration tests +============================ + +StoryBoard needs to have integration tests to ensure that all components +are interacting together properly, and to confirm that changes done +in a component are not affecting the normal behaviour of the others. + + +Problem description +=================== + +Rationale +--------- + +StoryBoard is composed of several components that interact together. +At the moment we have storyboard project for the backend, and +storyboard-webclient for the frontend. We also have the python-storyboard +client and in the future more components can be added. + +We have tests to ensure the quality of individual components, but we +don't have anything in place to ensure that changes done in one +project are not affecting the others. There is a subset of tests +created under StoryBoard webclient, but they only run when a change +is done in the frontend, not in the backend. + +There is a strong need to create a set of tests to connect all these +components together. Changes done in StoryBoard backend need to be +tested against webclient and python-storyboard. Changes done either +in the webclient or on python-storyboard need to be tested against +the backend. + + +Proposed change +=============== + +Creation of integration tests for StoryBoard +-------------------------------------------- + +A new macro and related jobs using that need to be created +into the Infrastructure project. These tests need to be +called on check and gate on each affected project, and need to +run a complete suite of tests that ensure that changes done in +one component do not affect the others. + +The related client components (at the moment +storyboard-webclient and python-storyboard client) need to +include integration tests on their repository. These tests will +run against the backend. + +Workflow for an integration test execution +------------------------------------------ + +* A change is created on an StoryBoard component, that triggers + the integration tests: + + - On a change to the backend, all client integration tests + will run. + + - On a change to a client, that client's tests will be run. + +* A setup script that installs the StoryBoard backend is + executed. That script will install StoryBoard API server + along with its dependencies, properly configured to run + the tests. This script will belong to the scope of the + StoryBoard backend project. + +* Integration tests for the StoryBoard webclient and python + client can be run now. The task to set the environment to run + the tests will be in the scope of each client, because they are + using different languages (javascript, python), so the tooling + is totally different. We also ensure that if new projects are + added into the future, can take care of their own tests. + + +Implementation +============== + +Implementation of the integration tests needs to be different +depending on the client, and details are left to the person +implementing that, but following the exposed premises. + +As described, a new builder needs to be created on +project-config, that takes care of the clone and backend +installation. Zuul-cloner will be used to set up the git +repositories for all components at the start of each test job. + +Jobs will be runing in a one-per-one basis, it means: +* storyboard-webclient => storyboard-backend +* python-storyboardclient => storyboard-backend + +A change done in the backend will trigger both jobs, and if +new components are added we need to be sure that all +combinations are covered. + + +Assignee(s) +----------- +Primary Assignee: + TBD + +Work Items +---------- +TBD + +Repositories +------------ +N/A + +Servers +------- +This will involve testing on reusable devstack-trusty nodes. + +DNS Entries +----------- +No new DNS entries. + +Documentation +------------- +TBD + +Security +-------- +TBD + +Testing +------- +Once tests are on place we need to run all possible combinations +to ensure that integration tests are on place and working.