openinfra/refstack
Code Issues Proposed changes

Adding mechanism to build documentation via sphinx

Browse Source 
Change-Id: Iaf1b201ddb6890199dd76ab67022557b7fd0b2fc
This commit is contained in:
Luz Cazares 2016-08-05 17:34:34 +00:00
parent fcac21f1bf
commit 6b5b867bdc
14 changed files with 826 additions and 393 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
README.rst
View File

@ -1,15 +1,17 @@
=========
RefStack RefStack
======== =========
**What is RefStack?** What is RefStack?
#################
- Toolset for testing interoperability between OpenStack clouds. - Toolset for testing interoperability between OpenStack clouds.
- Database backed website supporting collection and publication of - Database backed website supporting collection and publication of
Community Test results for OpenStack. Community Test results for OpenStack.
- User interface to display individual test run results. - User interface to display individual test run results.
**Overview** Overview
##########
RefStack intends on being THE source of tools for interoperability testing RefStack intends on being THE source of tools for interoperability testing
of OpenStack clouds. of OpenStack clouds.
@ -40,7 +42,7 @@ after their software has been incorporated into the distro or cloud.
vendors. It can be used by vendors in house to compare interoperability vendors. It can be used by vendors in house to compare interoperability
data over time. data over time.
* API and UI install docs: http://github.com/openstack/refstack/blob/master/doc/refstack.md * API and UI install docs: http://github.com/openstack/refstack/blob/master/doc/source/refstack.rst
* repository: http://git.openstack.org/cgit/openstack/refstack * repository: http://git.openstack.org/cgit/openstack/refstack
* reviews: https://review.openstack.org/#q,status:open+refstack,n,z * reviews: https://review.openstack.org/#q,status:open+refstack,n,z
@ -50,7 +52,8 @@ after their software has been incorporated into the distro or cloud.
* repository: http://git.openstack.org/cgit/openstack/refstack-client * repository: http://git.openstack.org/cgit/openstack/refstack-client
* reviews: https://review.openstack.org/#q,status:open+refstack-client,n,z * reviews: https://review.openstack.org/#q,status:open+refstack-client,n,z
**Get involved!** Get involved!
###############
* Mailing List: openstack-dev@lists.openstack.org * Mailing List: openstack-dev@lists.openstack.org
* IRC: #refstack on Freenode * IRC: #refstack on Freenode

230
doc/refstack.md
View File

@ -1,230 +0,0 @@
RefStack Quickstart
===================
You can use docker for [one-click setup](run_in_docker.md)
or follow step-by-step instructions below.
####Install API dependencies (on ubuntu 14.x)..
- `sudo apt-get install git python-dev libssl-dev python-setuptools`
- `sudo apt-get install mysql-server python-mysqldb`
- `sudo easy_install -U pip`
- `sudo easy_install -U virtualenv`
####Install RefStack UI dependencies
- `curl -sL https://deb.nodesource.com/setup | sudo bash -`
- `sudo apt-get install nodejs`
####Setup the RefStack database
- Log into MySQL: `mysql -u root -p`
- After authentication, create the database:
`CREATE DATABASE refstack;`
- Create a refstack user:
`CREATE USER 'refstack'@'localhost' IDENTIFIED BY '<your password>';`
or using hash value for your password
`CREATE USER 'refstack'@'localhost'
IDENTIFIED BY PASSWORD '<hash value of your password';`
- Grant privileges:
`GRANT ALL PRIVILEGES ON refstack . * TO 'refstack'@'localhost';`
- Reload privileges:
`FLUSH PRIVILEGES;`
- Exit MySQL: `quit`
####Git you clonin'
- `git clone http://github.com/openstack/refstack`
- `cd refstack`
- Create virtual environment: `virtualenv .venv --system-site-package`
- Source to virtual environment: `source .venv/bin/activate`
####Install RefStack application (on ubuntu 14.x)..
- `pip install .`
####Install needed RefStack UI library dependencies
- `npm install`
####API configuration file preparation
- Make a copy of the sample config file (etc/refstack.conf.sample) and
update it with the correct information of your environment. Examples
of the config parameters with default values are included in the
sample config file.
- You should ensure that the following values in the config file are noted and
properly set:
- `connection` field in the `[database]`section.
For example, if the backend database is MySQL then update:
`#connection = <None>` to
`connection = mysql+pymysql://refstack:<your password>@x.x.x.x/refstack`
- `ui_url` field in the `[DEFAULT]` section.
This should be the URL that the UI can be accessed from. This will likely
be in the form `http://<your server IP>:8000` (8000 being the default port
RefStack is hosted on). For example: `http://192.168.56.101:8000`
- `api_url` field in the `[api]` section.
This should be the URL that the API can be accessed from. This, in most
cases, will be the same as the value for `ui_url` above.
- `app_dev_mode` field in the `[api]` section.
Set this field to true if you aren't creating a production-level RefStack
deployment and are just trying things out or developing. Setting this field
to true will allow you to quickly bring up both the API and UI together,
with the UI files being served by a simple file server that comes with
Pecan.
####Create UI config file
From the RefStack project root directory, create a config.json file
and specify your API endpoint inside this file. This will be something like
{"refstackApiUrl": "http://192.168.56.101:8000/v1"}:
- `cp refstack-ui/app/config.json.sample refstack-ui/app/config.json`
####Openstack OpenID endpoint configuration (optional)
If you are only interested in the uploading and viewing of result sets, then
this section can be ignored. However, in order for user accounts and
authentication to work, you need to make sure you are properly configured
with an OpenStack OpenID endpoint. There are two options:
* Use the official endpoint [openstackid.org](https://openstackid.org)
* Host your own openstackid endpoint
Since openstackid checks for valid top-level domains, in both options you will
likely have to edit the hosts file of the system where your web-browser for
viewing the RefStack site resides. On Linux systems, you would modify
`/etc/hosts`, adding a line like the following:
`<RefStack server IP> <some valid domain name>`
Example:
`192.168.56.101 myrefstack.com`
On Windows, you would do the same in `%SystemRoot%\System32\drivers\etc\hosts`.
Alternatively, you can add a custom DNS record with the domain name mapping
if possible.
Note that doing this requires you to modify the config.json file and the
`api_url` and `ui_url` fields in refstack.conf to use this domain name instead
of the IP.
**Option 1 - Use Official Endpoint**
Using the official site is probably the easiest option as no additional
configuration is needed besides the hosts file modifications as noted above.
RefStack, by default, points to this endpoint.
**Option 2 - Use Local Endpoint**
Instructions for setting this up are outside of the scope of this doc, but you
can get started at [https://github.com/openstack-infra/openstackid]
(https://github.com/openstack-infra/openstackid)
or [https://github.com/fremontlabs/vagrant-openstackid]
(https://github.com/fremontlabs/vagrant-openstackid).
You would then need to modify the `openstack_openid_endpoint` field in the
`[osid]` section in refstack.conf to match the local endpoint.
####Database sync
- Check current revision:
`refstack-manage --config-file /path/to/refstack.conf version`
The response will show the current database revision. If the revision is
`None` (indicating a clear database), the following command should be
performed to upgrade the database to the latest revision:
- Upgrade database to latest revision:
`refstack-manage --config-file /path/to/refstack.conf upgrade --revision head`
- Check current revision:
`refstack-manage --config-file /path/to/refstack.conf version`
Now it should be some revision number other than `None`.
####Start RefStack
A simple way to start refstack is to just kick off gunicorn using the
`refstack-api` executable:
- `refstack-api --env REFSTACK_OSLO_CONFIG=/path/to/refstack.conf`
If `app_dev_mode` is set to true, this will launch both the UI and API.
Now available:
- `http://<your server IP>:8000/v1/results` with response JSON including
records consisting of `<test run id>` and `<upload date>` of the test runs.
The default response is limited to one page of the most recent uploaded test
run records. The number of records per page is configurable via the RefStack
configuration file. Filtering parameters such as page, start_date, and
end_date can also be used to specify the desired records. For example:
GET `http://<your server IP>:8000/v1/results?page=n` will return page *n*
of the data.
- `http://<your server IP>:8000/v1/results/<test run id>` with response JSON
including the detail test results of the specified `<test run id>`
####(Optional) Configure Foundation organization and group
Overall RefStack admin access is given to users belonging to a "Foundation"
organization. To become a Foundation admin, first a "Foundation" organization
must be created. Note that you must have logged into RefStack at least once so
that a user record for your account is created.
- Log into MySQL: `mysql -u root -p`
- Create a group for the "Foundation" organization:
`INSERT INTO refstack.group (id, name, created_at) VALUES (UUID(), 'Foundation Group', NOW());`
- Get the group ID for the group you just created:
`SELECT id from refstack.group WHERE name = 'Foundation Group';`
- Get your OpenID:
`SELECT openid from refstack.user WHERE email = '<your email>';`
- Add your user account to the previously created "Foundation" group. Replace
`<Group ID>` and `<Your OpenID>` with the values retrieved in the two previous steps:
`INSERT INTO refstack.user_to_group (created_by_user, user_openid, group_id, created_at)
VALUES ('<Your OpenID>', '<Your OpenID>', '<Group ID>', NOW());`
- Create the actual "Foundation" organization using this group:
`INSERT INTO refstack.organization (id, type, name, group_id, created_by_user, created_at)
VALUES (UUID(), 0, 'Foundation', '<Group ID>', '<Your OpenID>', NOW());`

56
doc/run_in_docker.md
View File

@ -1,56 +0,0 @@
Run-in-docker manual
===================
The main purpose of the `run-in-docker` script is to provide a convenient way
to create a local setup of RefStack inside a Docker container. It should be
helpful for new developers and also for testing new features.
####Requirements:
- `Docker >= 1.6` (How to update on [Ubuntu](http://www.ubuntuupdates.org/ppa/docker))
####How to use:
Just run the `run-in-docker` script, but is important to first set env[REFSTACK_HOST]
with the public host/IP for your local API. If you want to test RefStack with OpenStackid
you should point a valid local alias here. For example:
`export REFSTACK_HOST=myrefstack.com`
By default 127.0.0.1 is used.
After it completes, check that the site is running on https://127.0.0.1.
The script will build a RefStack docker image with all dependencies, and will run
a container from this image. By default, RefStack will run inside this container.
You also can run `run-in-docker bash` to get access into the container. If you stop
the RefStack server by pressing 'Ctrl-C', the container is kept alive and will be
re-used next time.
You can customize the RefStack API config by editing `docker/templates/refstack.conf.tmpl`.
It is a bash template, so you can use ${SOME\_ENV\_VARIABLE} in it.
This script can make the reviewing process much easier because it creates separate
containers for each review. Containers get names in the form refstack_{REVIEW-TOPIC}.
Database schema changes are automatically handled, too, where the script creates a data
container for each database revision (refstack\_data\_{DATA-BASE-REVISON}) and reuses it
where possible. For example, if a new review uses an existing database revision, that
database container will be used.
####Available script options:
- `-r` Force delete the RefStack container and run it again.
This will update the RefStack config from template noted above.
- `-i` Run a container with isolated MySQL data. By default MySQL data is stored in
a refstack\_data\_{DATA-BASE-REVISON} container. It reuses this container if such
one exists. If you want to drop the DB data, just execute
`sudo docker rm refstack_data_{DATA-BASE-REVISON}`.
- `-b` Force delete RefStack image and build it again. This rebuids the Python and JS
environment for RefStack.
- `-d` Turn on debug information.
- `-h` Print usage message.
####Useful in-container commands/aliases:
- `api-up` - sync project and run the RefStack API
- `api-init-db` - initialize the RefStack database
- `api-db-version` - get current migration version of the RefStack database
- `api-sync` - sync project files in the container with the project files on the host
- `activate` - activate the python virtual env
- `mysql` - open the MySQL console

1
doc/source/README.rst Symbolic link
View File

@ -0,0 +1 @@
../../README.rst

327
doc/source/conf.py Normal file
View File

@ -0,0 +1,327 @@
# -*- coding: utf-8 -*-
#
# Refstack documentation build configuration file, created by
# sphinx-quickstart on Fri Aug 5 01:41:59 2016.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys
import os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = []
# Add any paths that contain templates here, relative to this directory.
#templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Refstack'
copyright = u'2016, OpenStack Foundation'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '1.0'
# The full version, including alpha/beta/rc tags.
release = '1.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['specs/prior/*', 'specs/README.rst', 'specs/template.rst']
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'default'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
#html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'Refstackdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
('index', 'Refstack.tex', u'Refstack Documentation',
u'OpenStack Foundation', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'refstack', u'Refstack Documentation',
[u'OpenStack Foundation'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'Refstack', u'Refstack Documentation',
u'OpenStack Foundation', 'Refstack', 'Toolset for testing interoperability'
' between OpenStack clouds.', 'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
# -- Options for Epub output ----------------------------------------------
# Bibliographic Dublin Core info.
epub_title = u'Refstack'
epub_publisher = u'OpenStack Foundation'
epub_copyright = u'2016, OpenStack Foundation'
# The basename for the epub file. It defaults to the project name.
#epub_basename = u'Refstack'
# The HTML theme for the epub output. Since the default themes are not optimized
# for small screen space, using the same theme for HTML and epub output is
# usually not wise. This defaults to 'epub', a theme designed to save visual
# space.
#epub_theme = 'epub'
# The language of the text. It defaults to the language option
# or en if the language is not set.
#epub_language = ''
# The scheme of the identifier. Typical schemes are ISBN or URL.
#epub_scheme = ''
# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#epub_identifier = ''
# A unique identification for the text.
#epub_uid = ''
# A tuple containing the cover image and cover page html template filenames.
#epub_cover = ()
# A sequence of (type, uri, title) tuples for the guide element of content.opf.
#epub_guide = ()
# HTML files that should be inserted before the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_pre_files = []
# HTML files shat should be inserted after the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_post_files = []
# A list of files that should not be packed into the epub file.
epub_exclude_files = ['search.html']
# The depth of the table of contents in toc.ncx.
#epub_tocdepth = 3
# Allow duplicate toc entries.
#epub_tocdup = True
# Choose between 'default' and 'includehidden'.
#epub_tocscope = 'default'
# Fix unsupported image types using the PIL.
#epub_fix_images = False
# Scale large images.
#epub_max_image_width = 0
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#epub_show_urls = 'inline'
# If false, no index is generated.
#epub_use_index = True

46
doc/source/index.rst Normal file
View File

@ -0,0 +1,46 @@
.. Refstack documentation master file, created by
sphinx-quickstart on Fri Aug 5 01:41:59 2016.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
====================================
Welcome to RefStack's documentation!
====================================
--------
Overview
--------
.. toctree::
:maxdepth: 2
README
refstack
run_in_docker
uploading_private_results
-----
Specs
-----
**Implemented in the Newton Cycle**
.. toctree::
:maxdepth: 1
:glob:
specs/newton/implemented/*
**Implemented in the Mitaka Cycle**
.. toctree::
:maxdepth: 1
:glob:
specs/mitaka/implemented/*
Indices and tables
==================
* :ref:`search`

249
doc/source/refstack.rst Normal file
View File

@ -0,0 +1,249 @@
RefStack Quickstart
===================
You can use docker for `one-click setup <run_in_docker.rst>`__ or follow
step-by-step instructions below.
Install API dependencies (on ubuntu 14.x)..
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``sudo apt-get install git python-dev libssl-dev python-setuptools``
- ``sudo apt-get install mysql-server python-mysqldb``
- ``sudo easy_install -U pip``
- ``sudo easy_install -U virtualenv``
Install RefStack UI dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``curl -sL https://deb.nodesource.com/setup | sudo bash -``
- ``sudo apt-get install nodejs``
Setup the RefStack database
^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Log into MySQL: ``mysql -u root -p``
- After authentication, create the database:
``CREATE DATABASE refstack;``
- Create a refstack user:
``CREATE USER 'refstack'@'localhost' IDENTIFIED BY '<your password>';``
or using hash value for your password
``CREATE USER 'refstack'@'localhost' IDENTIFIED BY PASSWORD '<hash value of your password';``
- Grant privileges:
``GRANT ALL PRIVILEGES ON refstack . * TO 'refstack'@'localhost';``
- Reload privileges:
``FLUSH PRIVILEGES;``
- Exit MySQL: ``quit``
Git you clonin'
^^^^^^^^^^^^^^^
- ``git clone http://github.com/openstack/refstack``
- ``cd refstack``
- Create virtual environment:
``virtualenv .venv --system-site-package``
- Source to virtual environment: ``source .venv/bin/activate``
Install RefStack application (on ubuntu 14.x)..
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``pip install .``
Install needed RefStack UI library dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``npm install``
API configuration file preparation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Make a copy of the sample config file (etc/refstack.conf.sample) and
update it with the correct information of your environment. Examples
of the config parameters with default values are included in the
sample config file.
- You should ensure that the following values in the config file are
noted and properly set:
- ``connection`` field in the ``[database]``\ section.
For example, if the backend database is MySQL then update:
``#connection = <None>`` to
``connection = mysql+pymysql://refstack:<your password>@x.x.x.x/refstack``
- ``ui_url`` field in the ``[DEFAULT]`` section.
This should be the URL that the UI can be accessed from. This will
likely be in the form ``http://<your server IP>:8000`` (8000 being
the default port RefStack is hosted on). For example:
``http://192.168.56.101:8000``
- ``api_url`` field in the ``[api]`` section.
This should be the URL that the API can be accessed from. This, in
most cases, will be the same as the value for ``ui_url`` above.
- ``app_dev_mode`` field in the ``[api]`` section.
Set this field to true if you aren't creating a production-level
RefStack deployment and are just trying things out or developing.
Setting this field to true will allow you to quickly bring up both
the API and UI together, with the UI files being served by a simple
file server that comes with Pecan.
Create UI config file
^^^^^^^^^^^^^^^^^^^^^
From the RefStack project root directory, create a config.json file and
specify your API endpoint inside this file. This will be something like
{"refstackApiUrl": "http://192.168.56.101:8000/v1"}:
- ``cp refstack-ui/app/config.json.sample refstack-ui/app/config.json``
Openstack OpenID endpoint configuration (optional)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you are only interested in the uploading and viewing of result sets,
then this section can be ignored. However, in order for user accounts
and authentication to work, you need to make sure you are properly
configured with an OpenStack OpenID endpoint. There are two options:
- Use the official endpoint
`openstackid.org <https://openstackid.org>`__
- Host your own openstackid endpoint
Since openstackid checks for valid top-level domains, in both options
you will likely have to edit the hosts file of the system where your
web-browser for viewing the RefStack site resides. On Linux systems, you
would modify ``/etc/hosts``, adding a line like the following:
``<RefStack server IP> <some valid domain name>``
Example:
``192.168.56.101 myrefstack.com``
On Windows, you would do the same in
``%SystemRoot%\System32\drivers\etc\hosts``. Alternatively, you can add
a custom DNS record with the domain name mapping if possible.
Note that doing this requires you to modify the config.json file and the
``api_url`` and ``ui_url`` fields in refstack.conf to use this domain
name instead of the IP.
**Option 1 - Use Official Endpoint**
Using the official site is probably the easiest option as no additional
configuration is needed besides the hosts file modifications as noted
above. RefStack, by default, points to this endpoint.
**Option 2 - Use Local Endpoint**
Instructions for setting this up are outside of the scope of this doc,
but you can get started at
[https://github.com/openstack-infra/openstackid]
(https://github.com/openstack-infra/openstackid) or
[https://github.com/fremontlabs/vagrant-openstackid]
(https://github.com/fremontlabs/vagrant-openstackid). You would then
need to modify the ``openstack_openid_endpoint`` field in the ``[osid]``
section in refstack.conf to match the local endpoint.
Database sync
^^^^^^^^^^^^^
- Check current revision:
``refstack-manage --config-file /path/to/refstack.conf version``
The response will show the current database revision. If the revision is
``None`` (indicating a clear database), the following command should be
performed to upgrade the database to the latest revision:
- Upgrade database to latest revision:
``refstack-manage --config-file /path/to/refstack.conf upgrade --revision head``
- Check current revision:
``refstack-manage --config-file /path/to/refstack.conf version``
::
Now it should be some revision number other than `None`.
Start RefStack
^^^^^^^^^^^^^^
A simple way to start refstack is to just kick off gunicorn using the
``refstack-api`` executable:
- ``refstack-api --env REFSTACK_OSLO_CONFIG=/path/to/refstack.conf``
If ``app_dev_mode`` is set to true, this will launch both the UI and
API.
Now available:
- ``http://<your server IP>:8000/v1/results`` with response JSON
including records consisting of ``<test run id>`` and
``<upload date>`` of the test runs. The default response is limited
to one page of the most recent uploaded test run records. The number
of records per page is configurable via the RefStack configuration
file. Filtering parameters such as page, start\_date, and end\_date
can also be used to specify the desired records. For example: GET
``http://<your server IP>:8000/v1/results?page=n`` will return page
*n* of the data.
- ``http://<your server IP>:8000/v1/results/<test run id>`` with
response JSON including the detail test results of the specified
``<test run id>``
(Optional) Configure Foundation organization and group
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Overall RefStack admin access is given to users belonging to a
"Foundation" organization. To become a Foundation admin, first a
"Foundation" organization must be created. Note that you must have
logged into RefStack at least once so that a user record for your
account is created.
- Log into MySQL: ``mysql -u root -p``
- Create a group for the "Foundation" organization:
``INSERT INTO refstack.group (id, name, created_at) VALUES (UUID(), 'Foundation Group', NOW());``
- Get the group ID for the group you just created:
``SELECT id from refstack.group WHERE name = 'Foundation Group';``
- Get your OpenID:
``SELECT openid from refstack.user WHERE email = '<your email>';``
- Add your user account to the previously created "Foundation" group.
Replace ``<Group ID>`` and ``<Your OpenID>`` with the values
retrieved in the two previous steps:
``INSERT INTO refstack.user_to_group (created_by_user, user_openid, group_id, created_at) VALUES ('<Your OpenID>', '<Your OpenID>', '<Group ID>', NOW());``
- Create the actual "Foundation" organization using this group:
``INSERT INTO refstack.organization (id, type, name, group_id, created_by_user, created_at) VALUES (UUID(), 0, 'Foundation', '<Group ID>', '<Your OpenID>', NOW());``

73
doc/source/run_in_docker.rst Normal file
View File

@ -0,0 +1,73 @@
Run-in-docker manual
====================
The main purpose of the ``run-in-docker`` script is to provide a
convenient way to create a local setup of RefStack inside a Docker
container. It should be helpful for new developers and also for testing
new features.
Requirements:
^^^^^^^^^^^^^
- ``Docker >= 1.6`` (How to update on
`Ubuntu <http://www.ubuntuupdates.org/ppa/docker>`__)
How to use:
^^^^^^^^^^^
Just run the ``run-in-docker`` script, but is important to first set
env[REFSTACK\_HOST] with the public host/IP for your local API. If you
want to test RefStack with OpenStackid you should point a valid local
alias here. For example:
``export REFSTACK_HOST=myrefstack.com``
By default 127.0.0.1 is used.
After it completes, check that the site is running on https://127.0.0.1.
The script will build a RefStack docker image with all dependencies, and
will run a container from this image. By default, RefStack will run
inside this container. You also can run ``run-in-docker bash`` to get
access into the container. If you stop the RefStack server by pressing
'Ctrl-C', the container is kept alive and will be re-used next time.
You can customize the RefStack API config by editing
``docker/templates/refstack.conf.tmpl``. It is a bash template, so you
can use ${SOME\_ENV\_VARIABLE} in it.
This script can make the reviewing process much easier because it
creates separate containers for each review. Containers get names in the
form refstack\_{REVIEW-TOPIC}. Database schema changes are automatically
handled, too, where the script creates a data container for each
database revision (refstack\_data\_{DATA-BASE-REVISON}) and reuses it
where possible. For example, if a new review uses an existing database
revision, that database container will be used.
Available script options:
^^^^^^^^^^^^^^^^^^^^^^^^^
- ``-r`` Force delete the RefStack container and run it again. This
will update the RefStack config from template noted above.
- ``-i`` Run a container with isolated MySQL data. By default MySQL
data is stored in a refstack\_data\_{DATA-BASE-REVISON} container. It
reuses this container if such one exists. If you want to drop the DB
data, just execute
``sudo docker rm refstack_data_{DATA-BASE-REVISON}``.
- ``-b`` Force delete RefStack image and build it again. This rebuids
the Python and JS environment for RefStack.
- ``-d`` Turn on debug information.
- ``-h`` Print usage message.
Useful in-container commands/aliases:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``api-up`` - sync project and run the RefStack API
- ``api-init-db`` - initialize the RefStack database
- ``api-db-version`` - get current migration version of the RefStack
database
- ``api-sync`` - sync project files in the container with the project
files on the host
- ``activate`` - activate the python virtual env
- ``mysql`` - open the MySQL console

1
doc/source/specs Symbolic link
View File

@ -0,0 +1 @@
../../specs/

117
doc/source/uploading_private_results.rst Normal file
View File

@ -0,0 +1,117 @@
How to upload test results to RefStack
======================================
RefStack allows test results contributors to submit test results and
have them displayed either anonymously, or identified with a vendor. As
such, test results should be uploaded with validated users. Users will
first log into RefStack with their OpenStack ID to upload their public
keys. RefStack test results can then be uploaded to RefStack using the
corresponding private key. By default, the uploaded data isn't shared,
but authorized users can decide to share the results with the community
anonymously.
The following is a quick guide outlining the steps needed to upload your
first set of test results.
Register an OpenStack ID
^^^^^^^^^^^^^^^^^^^^^^^^
The RefStack server uses OpenStack OpenID for user authentication.
Therefore, the RefStack server requires that anyone who wants to upload
test data to have an OpenStack ID. As you click on the Sign In/Sign Up
link on the RefStack pages, you will be redirected to the official
OpenStack user log in page where you can either log in with your
OpenStack ID or register for one. The registration page can also be
found directly through: https://www.openstack.org/join/register.
Generate ssh keys locally
^^^^^^^^^^^^^^^^^^^^^^^^^
You will need to generate ssh keys locally. If your operating system is
a Linux distro, then you can use the following instructions.
First check for existing keys with command:
- ``ls -al ~/.ssh``
If you see you already have existing public and private keys that you
want to use, you can skip this step; otherwise:
- ``ssh-keygen -t rsa -b 4096 -C "youropenstackid"``
The 'youropenstackid' string is the username you chose when you
registered for your OpenStack ID account. Enter the file name in which
to save the key (/home/you/.ssh/id\_rsa), then press enter. You will be
asked to enter a passphrase. Just press enter again as passphrase
protected keys currently aren't supported. Your ssh keys will then be
generated.
Sign Key with RefStack Client
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\*\* IMPORTANT \*\* You must have the RefStack client on you computer to
complete this step.
Generate a signature for your public key using your private key with
`refstack-client <https://github.com/openstack/refstack-client>`__
- ``./refstack-client sign /path-of-sshkey-folder/key-file-name``
The '/path-of-sshkey-folder' string is the path of the folder where the
generated ssh keys are stored locally. The 'key-file-name' portion
refers to the private key file name. If the command runs correctly,
there will be output like below:
::
Public key:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDSGo2xNDcII1ZaM3H2uKh3iXBzvKIOa5W/5HxKF23yrzwho7nR7td0kgFtZ/4fe0zmkkUuKdUhOACCD3QVyi1N5wIhKAYN1fGt0/305jk7VJ+yYhUPlvo...
Self signature:
19c28fc07e3fbe1085578bd6db2f1f75611dcd2ced068a2195bbca60ae98af7e27faa5b6968c3c5aef58b3fa91bae3df3...
Upload the ssh public key and the signature
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sign into https://refstack.openstack.org with your OpenStack ID. Click
the "Profile" link in the upper right corner. Now click the "Import
public key" button on your profile page. A popup window with two entry
fields will appear. Just copy and paste the key and signature generated
in the previous step into the corresponding textboxes.
Note that the literal strings 'Public key:' and 'Self signature:' from
the refstack-client "sign" command output should not be copied/pasted
into the text boxes. Otherwise you will get an error like:
- ``Bad Request Request doesnt correspond to schema``
Once complete, click the 'Import public key' button.
Upload the test result with refstack-client
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- ``./refstack-client upload /path_to_testresult_json_file --url https://refstack.openstack.org/api -i ~/.ssh/id_rsa``
\*\* NOTE \*\* Users may need to add the '--insecure' optional agrument
to the command string if certificate validation issues occur when
uploading test result. To use with insecure:
- ``./refstack-client upload --insecure /path_to_testresult_json_file --url https://refstack.openstack.org/api -i ~/.ssh/id_rsa``
- The ``path_to_testresult_json_file`` here is the json file of your test result.
- By default, it's in ``.tempest/.testrespository/<test-run-number>.json`` where refstack-client runs from.
- Here '' is a serial number that matches its corresponding subunit file name.
- If the command runs correctly, there will be output like below:
::
Test results will be uploaded to https://refstack.openstack.org/api. Ok? (yes/y): y
Test results uploaded!
URL: https://refstack.openstack.org/#/results/88a1e6f4-707d-4627-b658-b14b7e6ba70d.
You can find your uploaded test results by clicking the 'My Results'
link on the RefStack website.

99
doc/uploading-private-results.md
View File

@ -1,99 +0,0 @@
How to upload test results to RefStack
======================================
RefStack allows test results contributors to submit test results and have them
displayed either anonymously, or identified with a vendor. As such, test
results should be uploaded with validated users. Users will first log into
RefStack with their OpenStack ID to upload their public keys. RefStack test
results can then be uploaded to RefStack using the corresponding private key.
By default, the uploaded data isn't shared, but authorized users can decide to
share the results with the community anonymously.
The following is a quick guide outlining the steps needed to upload your first
set of test results.
####Register an OpenStack ID
The RefStack server uses OpenStack OpenID for user authentication. Therefore,
the RefStack server requires that anyone who wants to upload test data to have
an OpenStack ID. As you click on the Sign In/Sign Up link on the RefStack
pages, you will be redirected to the official OpenStack user log in page where
you can either log in with your OpenStack ID or register for one. The
registration page can also be found directly through:
[https://www.openstack.org/join/register](https://www.openstack.org/join/register).
####Generate ssh keys locally
You will need to generate ssh keys locally. If your operating system is a Linux
distro, then you can use the following instructions.
First check for existing keys with command:
- `ls -al ~/.ssh`
If you see you already have existing public and private keys that you want to
use, you can skip this step; otherwise:
- `ssh-keygen -t rsa -b 4096 -C "youropenstackid"`
The 'youropenstackid' string is the username you chose when you registered for
your OpenStack ID account. Enter the file name in which to save the key
(/home/you/.ssh/id_rsa), then press enter. You will be asked to enter a
passphrase. Just press enter again as passphrase protected keys currently
aren't supported. Your ssh keys will then be generated.
####Sign Key with RefStack Client
** IMPORTANT ** You must have the RefStack client on you computer to complete
this step.
Generate a signature for your public key using your private key with
[refstack-client](https://github.com/openstack/refstack-client)
- `./refstack-client sign /path-of-sshkey-folder/key-file-name`
The '/path-of-sshkey-folder' string is the path of the folder where the
generated ssh keys are stored locally. The 'key-file-name' portion refers to
the private key file name. If the command runs correctly, there will be output
like below:
Public key:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDSGo2xNDcII1ZaM3H2uKh3iXBzvKIOa5W/5HxKF23yrzwho7nR7td0kgFtZ/4fe0zmkkUuKdUhOACCD3QVyi1N5wIhKAYN1fGt0/305jk7VJ+yYhUPlvo...
Self signature:
19c28fc07e3fbe1085578bd6db2f1f75611dcd2ced068a2195bbca60ae98af7e27faa5b6968c3c5aef58b3fa91bae3df3...
####Upload the ssh public key and the signature
Sign into [https://refstack.openstack.org](https://refstack.openstack.org) with
your OpenStack ID. Click the "Profile" link in the upper right corner. Now
click the "Import public key" button on your profile page. A popup window with
two entry fields will appear. Just copy and paste the key and signature
generated in the previous step into the corresponding textboxes.
Note that the literal strings 'Public key:' and 'Self signature:' from the refstack-client "sign" command output should not be copied/pasted into the text boxes. Otherwise you will get an error like:
- `Bad Request Request doesnt correspond to schema`
Once complete, click the 'Import public key' button.
####Upload the test result with refstack-client
- `./refstack-client upload /path_to_testresult_json_file --url https://refstack.openstack.org/api -i ~/.ssh/id_rsa`
** NOTE ** Users may need to add the '--insecure' optional agrument to the command string if certificate validation issues occur when uploading test result. To use with insecure:
- `./refstack-client upload --insecure /path_to_testresult_json_file --url https://refstack.openstack.org/api -i ~/.ssh/id_rsa`
The `path_to_testresult_json_file` here is the json file of your test result.
By default, it's in `.tempest/.testrespository/<test-run-number>.json` where refstack-client runs from.
Here '<test-run-number>' is a serial number that matches its corresponding subunit file name.
If the command runs correctly, there will be output like below:
Test results will be uploaded to https://refstack.openstack.org/api. Ok? (yes/y): y
Test results uploaded!
URL: https://refstack.openstack.org/#/results/88a1e6f4-707d-4627-b658-b14b7e6ba70d.
You can find your uploaded test results by clicking the 'My Results' link on
the RefStack website.

2
refstack-ui/app/components/about/about.html
View File

@ -38,7 +38,7 @@
data over time.</p> data over time.</p>
<ul> <ul>
<li>install docs: <li>install docs:
<a href="https://github.com/openstack/refstack/blob/master/doc/refstack.md">doc/refstack.md</a> <a href="https://github.com/openstack/refstack/blob/master/doc/source/refstack.rst">doc/refstack.rst</a>
</li> </li>
<li>repository: <li>repository:
<a href="http://git.openstack.org/cgit/openstack/refstack">http://git.openstack.org/cgit/openstack/refstack</a> <a href="http://git.openstack.org/cgit/openstack/refstack">http://git.openstack.org/cgit/openstack/refstack</a>

2
refstack-ui/app/components/profile/importPubKeyModal.html
View File

@ -1,7 +1,7 @@
<div class="modal-header"> <div class="modal-header">
<h4>Import Public Key</h4> <h4>Import Public Key</h4>
<p>Instructions for adding a public key and signature can be found <p>Instructions for adding a public key and signature can be found
<a href="https://github.com/openstack/refstack/blob/master/doc/uploading-private-results.md#generate-ssh-keys-locally" <a href="https://github.com/openstack/refstack/blob/master/doc/source/uploading_private_results.rst#generate-ssh-keys-locally"
target="_blank" target="_blank"
title="How to generate and upload SSH key and signature with refstack-client">here. title="How to generate and upload SSH key and signature with refstack-client">here.
</a> </a>

1
test-requirements.txt
View File

@ -2,6 +2,7 @@ coverage>=3.6
pep8==1.5.7 pep8==1.5.7
pyflakes==0.8.1 pyflakes==0.8.1
flake8==2.2.4 flake8==2.2.4
sphinx>=1.1.2,!=1.2.0,<1.3
httmock>=1.2.4 httmock>=1.2.4
mock mock
oslotest>=1.2.0 # Apache-2.0 oslotest>=1.2.0 # Apache-2.0