Update documentation, add playbook for developers

The patch updates the user-faced documentation and adds a playbook
together with a doc meant mainly for RefStack developers which
helps with running the server instance locally.

Change-Id: I268fdf717c8f0e2cd0c130c560306bf54e48f348

doc/source/index.rst

@@ -42,4 +42,3 @@ Indices and tables
 ------------------
 
 * :ref:`search`
-

doc/source/run_in_docker.rst

@@ -1,6 +1,111 @@
 Run-in-docker manual
 ====================
 
+Option 1 - Using an ansible playbook
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+These steps are meant for RefStack developers to help them with setting up
+a local refstack instance.
+
+In production RefStack server is managed by a set of playbooks and ansible roles
+defined in `system-config <https://opendev.org/opendev/system-config.git>`__
+repository. This option takes advantage of those.
+
+The RefStack server is running on Ubuntu 20.04 LTS in the production.
+
+You can find an ansible playbook in ``playbooks`` directory which semi-automates
+the process of running refstack server in a container.
+
+Execute the playbook by::
+
+    $ ansible-playbook playbooks/run-refstack-in-container.yaml
+
+In order to avoid setting certificates and https protocol (it's simpler and more
+than enough for a testing instance), edit
+``/etc/apache2/sites-enabled/000-default.conf`` like following:
+
+* remove VirtualHost section for the port 80 and change the port of VirtualHost from 443 to 80
+* Turn off the SSLEngine (`SSLEngine on -> SSLEngine off`)
+* Remove SSLCertificate lines
+
+and then restart the apache service so that it loads the new configuration::
+
+    $ systemctl restart apache2
+
+
+How to edit refstack files within the container
+-----------------------------------------------
+
+List the running container by::
+
+    $ docker container list
+
+You can enter the container by::
+
+    $ sudo docker exec -it <container name> /bin/bash
+
+If you wanna install new packages like f.e. vim, do the following::
+    $ apt update
+    $ apt install vim
+
+Edit what's needed, backend is installed under
+``/usr/local/lib/python3.7/site-packages/refstack/`` and frontend source files
+can be found at ``/refstack-ui``
+
+After you made the changes, make pecan to reload the files served::
+
+    $ apt install procps  # to install pkill command
+    $ pkill pecan
+
+Killing pecan will kick you out of the container, however, pecan serves the
+edited files now and you may re-enter the container.
+
+Installing refstack with changes put for a review
+-------------------------------------------------
+
+In order to do this, you will need to rebuild the refstack image built by the
+playbook.
+
+Go to the location where the playbook downloaded system-config, default in
+``/tmp/refstack-docker`` and edit the refstack's Dockerfile::
+
+    $ cd /tmp/refstack-docker
+    $ vim ./refstack-docker-files/Dockerfile
+
+Replace::
+
+    $ RUN git clone https://opendev.org/osf/refstack /tmp/src
+
+by::
+
+    $ RUN git clone https://opendev.org/osf/refstack.git /tmp/src \
+      && cd /tmp/src && git fetch "https://review.opendev.org/osf/refstack" \
+      refs/changes/37/<change id/<patchset number> && git checkout -b \
+      change-<change id>-<patchset number> FETCH_HEAD
+
+Then rebuild the image::
+
+    $ docker image build -f Dockerfile -t <name:tag> .
+
+Edit the ``docker-compose.yaml`` stored (by default) in
+``/etc/refstack-docker/docker-compose.yaml`` and change the the image
+(under `refstack-api`) to your image name and tag you set in the previous step.
+
+After then spin a new container using the new image::
+
+    $ cd /etc/refstack-docker
+    $ docker-compose down  # if refstack container is already running
+    $ docker-compose up -d
+
+To see the server's logs use the following command::
+
+    $ docker container logs -f <container name>
+
+
+Option 2 - Using a script
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+NOTE: This is currently outdated, follow the Option 1 for now.
+
 The main purpose of the ``run-in-docker`` script is to provide a
 convenient way to create a local setup of RefStack inside a Docker
 container. It should be helpful for new developers and also for testing
@@ -70,4 +175,3 @@ Useful in-container commands/aliases:
    files on the host
 -  ``activate`` - activate the python virtual env
 -  ``mysql`` - open the MySQL console
-

doc/source/uploading_private_results.rst

@@ -5,7 +5,7 @@ RefStack allows test results contributors to submit test results and
 have them displayed either anonymously, or identified with a vendor. As
 such, test results should be uploaded with validated users. Users will
 first log into RefStack with their OpenStack ID to upload their public
-keys. RefStack test results can then be uploaded to RefStack using the
+keys. RefStack test results need to be uploaded to RefStack using the
 corresponding private key. By default, the uploaded data isn't shared,
 but authorized users can decide to share the results with the community
 anonymously.
@@ -19,10 +19,11 @@ Register an OpenStack ID
 The RefStack server uses OpenStack OpenID for user authentication.
 Therefore, the RefStack server requires that anyone who wants to upload
 test data to have an OpenStack ID. As you click on the Sign In/Sign Up
-link on the RefStack pages, you will be redirected to the official
+link on the `RefStack pages <https://refstack.openstack.org/#/>`__, you
-OpenStack user log in page where you can either log in with your
+will be redirected to the official OpenStack user log in page where you
-OpenStack ID or register for one. The registration page can also be
+can either log in with your OpenStack ID or register for one.
-found directly through: https://www.openstack.org/join/register.
+The registration page can also be found directly through:
+https://www.openstack.org/join/register.
 
 Generate ssh keys locally
 ^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -30,18 +31,18 @@ Generate ssh keys locally
 You will need to generate ssh keys locally. If your operating system is
 a Linux distro, then you can use the following instructions.
 
-First check for existing keys with command:
+First check for existing keys with command::
 
--  ``ls -al ~/.ssh``
+    $ ls -al ~/.ssh
 
 If you see you already have existing public and private keys that you
-want to use, you can skip this step; otherwise:
+want to use, you can skip this step; otherwise::
 
--  ``ssh-keygen -m PEM -t rsa -b 4096 -C "youropenstackid"``
+    $ ssh-keygen -m PEM -t rsa -b 4096 -C "youropenstackid"
 
-The 'youropenstackid' string is the username you chose when you
+The `youropenstackid` string is the username you chose when you
 registered for your OpenStack ID account. Enter the file name in which
-to save the key (/home/you/.ssh/id\_rsa), then press enter. You will be
+to save the key (``/home/you/.ssh/id\_rsa``), then press enter. You will be
 asked to enter a passphrase. Just press enter again as passphrase
 protected keys currently aren't supported. Your ssh keys will then be
 generated.
@@ -49,15 +50,17 @@ generated.
 Sign Key with RefStack Client
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-\*\* IMPORTANT \*\* You must have the RefStack client on you computer to
+**IMPORTANT** You must have the RefStack client on you computer to
 complete this step.
+Follow `its README <https://opendev.org/osf/refstack-client>`__ on how to
+install it.
 
 Generate a signature for your public key using your private key with
-`refstack-client <https://opendev.org/osf/refstack-client>`__
+`refstack-client <https://opendev.org/osf/refstack-client>`__::
 
--  ``./refstack-client sign /path-of-sshkey-folder/key-file-name``
+    $ ./refstack-client sign /path-of-sshkey-folder/key-file-name
 
-The '/path-of-sshkey-folder' string is the path of the folder where the
+The ``/path-of-sshkey-folder`` string is the path of the folder where the
 generated ssh keys are stored locally. The 'key-file-name' portion
 refers to the private key file name. If the command runs correctly,
 there will be output like below:
@@ -74,37 +77,37 @@ Upload the ssh public key and the signature
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Sign into https://refstack.openstack.org with your OpenStack ID. Click
-the "Profile" link in the upper right corner. Now click the "Import
+the `Profile` link in the upper right corner. Now click the `Import
-public key" button on your profile page. A popup window with two entry
+public key` button on your profile page. A popup window with two entry
 fields will appear. Just copy and paste the key and signature generated
 in the previous step into the corresponding textboxes.
 
-Note that the literal strings 'Public key:' and 'Self signature:' from
+Note that the literal strings `Public key:` and `Self signature:` from
-the refstack-client "sign" command output should not be copied/pasted
+the ``refstack-client sign`` command output **should not** be copied/pasted
-into the text boxes. Otherwise you will get an error like:
+into the text boxes. Otherwise you will get an error like::
 
--  ``Bad Request Request doesnt correspond to schema``
+    Bad Request Request doesn't correspond to schema
 
-Once complete, click the 'Import public key' button.
+Once complete, click the `Import public key` button.
 
 Upload the test result with refstack-client
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
--  ``./refstack-client upload /path_to_testresult_json_file  --url https://refstack.openstack.org/api -i  ~/.ssh/id_rsa``
+The results can be uploaded using the ``refstack-client`` by::
 
-\*\* NOTE \*\* Users may need to add the '--insecure' optional argument
+    $ ./refstack-client upload /path_to_testresult_json_file \
+      --url https://refstack.openstack.org/api -i  ~/.ssh/id_rsa
+
+**NOTE** Users may need to add the `--insecure` optional argument
 to the command string if certificate validation issues occur when
-uploading test result. To use with insecure:
+uploading test result. Usage with the insecure argument::
 
--  ``./refstack-client upload --insecure /path_to_testresult_json_file  --url https://refstack.openstack.org/api -i  ~/.ssh/id_rsa``
+    $ ./refstack-client upload --insecure /path_to_testresult_json_file \
+      --url https://refstack.openstack.org/api -i  ~/.ssh/id_rsa``
 
-- The ``path_to_testresult_json_file`` here is the json file of your test result.
+The ``path_to_testresult_json_file`` there is the json file of the test result.
-
+By default, it's in ``.tempest/.stestr/<test-run-number>.json``. If the command
-- By default, it's in ``.tempest/.stestr/<test-run-number>.json`` where refstack-client runs from.
+runs correctly, there will be output like below:
-
-- Here '' is a serial number that matches its corresponding subunit file name.
-
-- If the command runs correctly, there will be output like below:
 
 ::
 
@@ -112,6 +115,5 @@ uploading test result. To use with insecure:
     Test results uploaded!
     URL: https://refstack.openstack.org/#/results/88a1e6f4-707d-4627-b658-b14b7e6ba70d.
 
-You can find your uploaded test results by clicking the 'My Results'
+You can find your uploaded test results by clicking the `My Results`
 link on the RefStack website.
-

doc/source/vendor_product.rst

@@ -88,4 +88,4 @@ Product version
 
 A default version is created whenever a product is created. The name of the
 default version is blank. The default version is used for products that have
-no version.  Users can add new product versions to the product as needed.
+no version. Users can add new product versions to the product as needed.

playbooks/README.rst

@@ -0,0 +1,12 @@
+Playbook for running refstack locally
+######################################
+
+The playbook is meant for developers to help them with debugging and
+reviewing new changes in the refstack project.
+
+The playbook semi-automates running the refstack server on the localhost.
+It downloads refstack role and templates from
+`system-config <https://opendev.org/opendev/system-config.git>`__ repository
+which is used for deploying and maintaining upstream servers, one of which is
+refstack. Then it builds the refstack image and spins a container using the
+refstack role.

playbooks/run-refstack-in-container.yml

@@ -0,0 +1,52 @@
+---
+- hosts: localhost
+  become: true
+  gather_facts: true
+  vars:
+    # dir where refstack files for buidling an image, running a container
+    # will be stored
+    refstack_dir: /tmp/refstack-docker
+    refstack_openid_endpoint: ''
+    # ip address of the machine you're running this on
+    refstack_url: # 'http://<IP address of your server>'
+    # the default credentials for the refstack database
+    refstack_db_username: refstack
+    refstack_db_password: Jz4ooq9TL7nc3hX3
+    refstack_root_db_password: KbgY3r9HYnEYpgRP
+  tasks:
+    - name: Clone system-config repository
+      git:
+        repo: https://opendev.org/opendev/system-config.git
+        dest: "{{ refstack_dir }}/system-config"
+
+    - name: Extract docker files
+      copy:
+        src: "{{ refstack_dir }}/system-config/docker/refstack/"
+        dest: "{{ refstack_dir }}/refstack-docker-files"
+        remote_src: yes
+
+    - name: Extract refstack role
+      copy:
+        src: "{{ refstack_dir }}/system-config/playbooks/roles/refstack/"
+        dest: "{{ refstack_dir }}/refstack-role"
+        remote_src: yes
+
+    - name: Delete the rest of system-config content
+      file:
+        state: absent
+        path: "{{ refstack_dir }}/system-config"
+
+    - name: Install Docker
+      apt:
+        name:
+          - docker
+          - docker-compose
+        state: present
+
+    - name: Build refstack image
+      command: docker image build -f Dockerfile -t refstack:1 .
+      args:
+        chdir: "{{ refstack_dir }}/refstack-docker-files"
+
+    - include_role:
+        name: "{{ refstack_dir }}/refstack-role"