Rename certification/ to tasks/

Every day Rally step by step closer to the point where Rally is going to be
generic framework and it would be possible to split Rally & Rally OpenStack Plugins

One of the blockers for doing this is certification & samples directories and
their contents. Current apporach creates a lot of confusion:

- Rally Users are running samples against production
- Rally samples are actually not that usefull, because creating from them
  tasks takes a lot of time
- Rally Users don't know what certification task is and how to use it
- Samples are hard to ship/find in packaged way (when rally is installed)
- Samples are going to be impossible to ship after the split
- We have to keep hunderds of yaml/json files in actual state
  in our repo (extra work for developers)

Proposed approach is to:

- Create mechansim CLI command that generates samples using plugin names
- Rename certification -> tasks
- Put in tasks directory pre created and tested tasks for different use cases:
  key performance metrics, functional testing, load testing of key functionality,
  ha Testing
- Remove samples as they are not needed anymore

Change-Id: I4c45224c3af637d17dab0edea038c69a2bae8b1a

Dockerfile

@@ -14,7 +14,7 @@ RUN echo '[ ! -z "$TERM" -a -r /etc/motd ] && cat /etc/motd' \
             >> /etc/bash.bashrc; echo '\
 ╔═════════════════════════════════════════════════════════════════════════════╗\n\
 ║ Welcome to Rally Docker container!                                          ║\n\
-║  Rally certification tasks, samples and docs are located at ~/source/       ║\n\
+║  Rally pre created tasks, samples and docs are located at ~/source/         ║\n\
 ║  Rally configuration and DB are in ~/.rally/                                ║\n\
 ║  Rally at readthedocs - http://rally.readthedocs.org                        ║\n\
 ║  How to contribute - http://rally.readthedocs.org/en/latest/contribute.html ║\n\

doc/source/install_and_upgrade/install.rst

@@ -135,7 +135,7 @@ be done via the following steps:
    likely to work if your home directory has excessively open
    permissions (e.g., ``0755``), which is not recommended.
 
-You can find all task samples, docs and certification tasks at /opt/rally/.
+You can find all task samples, docs and pre created tasks at /opt/rally/.
 Also you may want to save the last command as an alias:
 
 .. code-block:: bash

tasks/README.rst

@@ -0,0 +1,6 @@
+================================
+Rally Tasks For Production Usage
+================================
+
+
+Detailed Instruction TBD

tasks/openstack/README.rst

@@ -0,0 +1,50 @@
+============================
+OpenStack Certification Task
+============================
+
+How To Validate & Run Task
+--------------------------
+
+To validate task with your own parameters run:
+
+.. code-block:: console
+
+  $ rally task validate task.yaml --task-args-file task_arguments.yaml
+
+
+To start task with your own parameters run:
+
+.. code-block:: console
+
+  $ rally task start task.yaml --task-args-file task_arguments.yaml
+
+
+Task Arguments
+--------------
+
+File task_arguments.yaml contains all task options:
+
++------------------------+----------------------------------------------------+
+| Name                   | Description                                        |
++========================+====================================================+
+| service_list           | List of services which should be tested            |
++------------------------+----------------------------------------------------+
+| smoke                  | Dry run without load from 1 user                   |
++------------------------+----------------------------------------------------+
+| use_existing_users     | In case of testing cloud with r/o Keystone e.g. AD |
++------------------------+----------------------------------------------------+
+| image_name             | Images name that exist in cloud                    |
++------------------------+----------------------------------------------------+
+| flavor_name            | Flavor name that exist in cloud                    |
++------------------------+----------------------------------------------------+
+| glance_image_location  | URL of image that is used to test Glance upload    |
++------------------------+----------------------------------------------------+
+| users_amount           | Expected amount of users                           |
++------------------------+----------------------------------------------------+
+| tenants_amount         | Expected amount of tenants                         |
++------------------------+----------------------------------------------------+
+| controllers_amount     | Amount of OpenStack API nodes (controllers)        |
++------------------------+----------------------------------------------------+
+
+All options have default values, hoverer user should change them to reflect
+configuration and size of tested OpenStack cloud.

tasks/openstack/macro/macro.yaml

@@ -0,0 +1,95 @@
+{%- macro user_context(tenants,users_per_tenant, use_existing_users) -%}
+{%- if use_existing_users and caller is not defined -%} {}
+{%- else %}
+  {%- if not use_existing_users %}
+        users:
+          tenants: {{ tenants }}
+          users_per_tenant: {{ users_per_tenant }}
+  {%- endif %}
+  {%- if caller is defined %}
+    {{ caller() }}
+  {%- endif %}
+{%- endif %}
+{%- endmacro %}
+
+{%- macro vm_params(image=none, flavor=none, size=none) %}
+{%- if flavor is not none %}
+        flavor:
+          name: {{ flavor }}
+{%- endif %}
+{%- if image is not none %}
+        image:
+          name: {{ image }}
+{%- endif %}
+{%- if size is not none %}
+        size: {{ size }}
+{%- endif %}
+{%- endmacro %}
+
+{%- macro unlimited_volumes() %}
+          cinder:
+            gigabytes: -1
+            snapshots: -1
+            volumes: -1
+{%- endmacro %}
+
+{%- macro constant_runner(concurrency=1, times=1, is_smoke=True) %}
+        type: "constant"
+  {%- if is_smoke %}
+        concurrency: 1
+        times: 1
+  {%- else %}
+        concurrency: {{ concurrency }}
+        times: {{ times }}
+  {%- endif %}
+{%- endmacro %}
+
+{%- macro rps_runner(rps=1, times=1, is_smoke=True) %}
+        type: rps
+  {%- if is_smoke %}
+        rps: 1
+        times: 1
+  {%- else %}
+        rps: {{ rps }}
+        times: {{ times }}
+  {%- endif %}
+{%- endmacro %}
+
+{%- macro no_failures_sla() %}
+        failure_rate:
+          max: 0
+{%- endmacro %}
+
+{%- macro volumes(size=1, volumes_per_tenant=1) %}
+        volumes:
+          size: {{ size }}
+          volumes_per_tenant: {{ volumes_per_tenant }}
+{%- endmacro %}
+
+{%- macro unlimited_nova(keypairs=false) %}
+          nova:
+            cores: -1
+            floating_ips: -1
+            instances: -1
+            {%- if keypairs %}
+            key_pairs: -1
+            {%- endif %}
+            ram: -1
+            security_group_rules: -1
+            security_groups: -1
+{%- endmacro %}
+
+{%- macro unlimited_neutron() %}
+{% if "neutron" in service_list %}
+          neutron:
+            network: -1
+            port: -1
+            subnet: -1
+{% endif %}
+{%- endmacro %}
+
+{%- macro glance_args(location, container="bare", type="qcow2") %}
+        container_format: {{ container }}
+        disk_format: {{ type }}
+        image_location: {{ location }}
+{%- endmacro %}

tasks/openstack/scenario/authentication.yaml

@@ -0,0 +1,8 @@
+  Authenticate.keystone:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ rps_runner(rps=15*controllers_amount, times=20000*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}

tasks/openstack/scenario/cinder.yaml

@@ -0,0 +1,191 @@
+  CinderVolumes.create_and_attach_volume:
+    -
+      args:
+        {{ vm_params(image_name,flavor_name,1) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(30, 10*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_and_delete_snapshot:
+    -
+      args:
+        force: false
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {{ volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_and_delete_volume:
+    -
+      args:
+        size:
+          max: 1
+          min: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(100, 33*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+    -
+      args:
+        {{ vm_params(image_name,none,1) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(100, 33*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+    -
+      args:
+        size: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(100, 33*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_and_extend_volume:
+    -
+      args:
+        new_size: 2
+        size: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(100, 33*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_and_list_snapshots:
+    -
+      args:
+        detailed: true
+        force: false
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {{ volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(100, 33*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_and_list_volume:
+    -
+      args:
+        detailed: true
+        {{ vm_params(image_name,none,1) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(100, 33*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+    -
+      args:
+        detailed: true
+        size: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(100, 33*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_and_upload_volume_to_image:
+    -
+      args:
+        container_format: "bare"
+        disk_format: "raw"
+        do_delete: true
+        force: false
+        size: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(40, 13*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_from_volume_and_delete_volume:
+    -
+      args:
+        size: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        {{ volumes() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  CinderVolumes.create_nested_snapshots_and_attach_volume:
+    -
+      args:
+        nested_level: 1
+        size:
+          max: 1
+          min: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_volumes() }}
+        servers:
+          {{ vm_params(image_name,flavor_name,none)|indent(2,true) }}
+          servers_per_tenant: 1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(10, 3*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  Quotas.cinder_update_and_delete:
+    -
+      args:
+        max_quota: 1024
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}

tasks/openstack/scenario/glance.yaml

@@ -0,0 +1,30 @@
+  GlanceImages.create_and_delete_image:
+    -
+      args:
+        {{ glance_args(location=glance_image_location) }}
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  GlanceImages.create_and_list_image:
+    -
+      args:
+        {{ glance_args(location=glance_image_location) }}
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  GlanceImages.list_images:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}

tasks/openstack/scenario/keystone.yaml

@@ -0,0 +1,62 @@
+  KeystoneBasic.add_and_remove_user_role:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 7*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  KeystoneBasic.create_add_and_list_user_roles:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 7*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  KeystoneBasic.create_and_list_tenants:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount),times=min(200, 10*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  KeystoneBasic.create_and_delete_role:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 7*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  KeystoneBasic.create_and_delete_service:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 7*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  KeystoneBasic.get_entities:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 3*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  KeystoneBasic.create_update_and_delete_tenant:
+    -
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 7*controllers_amount),times=min(200, 67*controllers_amount), is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}

tasks/openstack/scenario/neutron.yaml

@@ -0,0 +1,245 @@
+  NeutronNetworks.create_and_delete_networks:
+    -
+      args:
+        network_create_args: {}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_delete_ports:
+    -
+      args:
+        network_create_args: {}
+        port_create_args: {}
+        ports_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            port: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_delete_routers:
+    -
+      args:
+        network_create_args: {}
+        router_create_args: {}
+        subnet_cidr_start: "1.1.0.0/30"
+        subnet_create_args: {}
+        subnets_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            subnet: -1
+            port: -1
+            router: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_delete_subnets:
+    -
+      args:
+        network_create_args: {}
+        subnet_cidr_start: "1.1.0.0/30"
+        subnet_create_args: {}
+        subnets_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            subnet: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_list_networks:
+    -
+      args:
+        network_create_args: {}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_list_ports:
+    -
+      args:
+        network_create_args: {}
+        port_create_args: {}
+        ports_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            port: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_list_routers:
+    -
+      args:
+        network_create_args: {}
+        router_create_args: {}
+        subnet_cidr_start: "1.1.0.0/30"
+        subnet_create_args: {}
+        subnets_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            subnet: -1
+            router: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_list_subnets:
+    -
+      args:
+        network_create_args: {}
+        subnet_cidr_start: "1.1.0.0/30"
+        subnet_create_args: {}
+        subnets_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            subnet: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_update_networks:
+    -
+      args:
+        network_create_args: {}
+        network_update_args:
+          admin_state_up: false
+          name: "_updated"
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_update_ports:
+    -
+      args:
+        network_create_args: {}
+        port_create_args: {}
+        port_update_args:
+          admin_state_up: false
+          device_id: "dummy_id"
+          device_owner: "dummy_owner"
+          name: "_port_updated"
+        ports_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            port: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_update_routers:
+    -
+      args:
+        network_create_args: {}
+        router_create_args: {}
+        router_update_args:
+          admin_state_up: false
+          name: "_router_updated"
+        subnet_cidr_start: "1.1.0.0/30"
+        subnet_create_args: {}
+        subnets_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            subnet: -1
+            port: -1
+            router: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NeutronNetworks.create_and_update_subnets:
+    -
+      args:
+        network_create_args: {}
+        subnet_cidr_start: "1.4.0.0/16"
+        subnet_create_args: {}
+        subnet_update_args:
+          enable_dhcp: false
+          name: "_subnet_updated"
+        subnets_per_network: 1
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          neutron:
+            network: -1
+            subnet: -1
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  Quotas.neutron_update:
+    -
+      args:
+        max_quota: 1024
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=2*controllers_amount, times=8*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}

tasks/openstack/scenario/nova.yaml

@@ -0,0 +1,195 @@
+  NovaKeypair.boot_and_delete_server_with_keypair:
+    -
+      args:
+        {{ vm_params(image_name, flavor_name) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova(keypairs=true) }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaKeypair.create_and_delete_keypair:
+    -
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_nova(keypairs=true) }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=67*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaKeypair.create_and_list_keypairs:
+    -
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        quotas:
+          {{ unlimited_nova(keypairs=true) }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=67*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaServers.boot_and_bounce_server:
+    -
+      args:
+        actions:
+          -
+            hard_reboot: 1
+          -
+            soft_reboot: 1
+          -
+            stop_start: 1
+          -
+            rescue_unrescue: 1
+        {{ vm_params(image_name, flavor_name) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaServers.boot_and_delete_server:
+    -
+      args:
+        {{ vm_params(image_name, flavor_name) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaServers.boot_and_list_server:
+    -
+      args:
+        detailed: true
+        {{ vm_params(image_name, flavor_name) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 10*controllers_amount), times=333*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaServers.boot_and_rebuild_server:
+    -
+      args:
+        {{ vm_params(flavor=flavor_name) }}
+        from_image:
+          name: {{ image_name }}
+        to_image:
+          name: {{ image_name }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaServers.boot_server_from_volume_and_delete:
+    -
+      args:
+        {{ vm_params(image_name, flavor_name) }}
+        volume_size: 5
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_volumes() }}
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 3*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaServers.pause_and_unpause_server:
+    -
+      args:
+        {{ vm_params(image_name, flavor_name) }}
+        force_delete: false
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  NovaServers.snapshot_server:
+    -
+      args:
+        {{ vm_params(image_name, flavor_name) }}
+      context:
+        {% call user_context(tenants_amount, users_amount, use_existing_users) %}
+        network:
+          networks_per_tenant: 1
+          start_cidr: "100.1.0.0/25"
+        quotas:
+          {{ unlimited_neutron() }}
+          {{ unlimited_nova() }}
+        {% endcall %}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}
+
+  Quotas.nova_update_and_delete:
+    -
+      args:
+        max_quota: 1024
+      context:
+        {{ user_context(tenants_amount, users_amount, use_existing_users) }}
+      runner:
+        {{ constant_runner(concurrency=min(50, 2*controllers_amount), times=17*controllers_amount, is_smoke=smoke) }}
+      sla:
+        {{ no_failures_sla() }}

tasks/openstack/task.yaml

@@ -0,0 +1,42 @@
+{%- set glance_image_location = glance_image_location|default("https://download.cirros-cloud.net/0.3.5/cirros-0.3.5-i386-disk.img") %}
+{%- set image_name = image_name|default("^(cirros.*-disk|TestVM)$") %}
+{%- set flavor_name = flavor_name|default("m1.tiny") %}
+{%- set use_existing_users = use_existing_users|default(false) %}
+{%- set service_list = service_list|default(["authentication", "cinder", "keystone", "nova", "glance", "neutron"]) %}
+{%- set smoke = smoke|default(true) %}
+{%- set controllers_amount = controllers_amount|default(1) %}
+{%- if smoke %}
+{%- set users_amount = 1 %}
+{%- set tenants_amount = 1 %}
+{%- else %}
+{%- set users_amount = users_amount|default(1) %}
+{%- set tenants_amount = tenants_amount|default(1) %}
+{%- endif %}
+
+{%- from "macro/macro.yaml" import user_context, vm_params, unlimited_volumes, constant_runner, rps_runner, no_failures_sla -%}
+{%- from "macro/macro.yaml" import volumes, unlimited_nova, unlimited_neutron, glance_args -%}
+
+---
+{% if "authentication" in service_list %}
+{%- include "scenario/authentication.yaml"-%}
+{% endif %}
+
+{% if "cinder" in service_list %}
+{%- include "scenario/cinder.yaml"-%}
+{% endif %}
+
+{% if "keystone" in service_list %}
+{%- include "scenario/keystone.yaml"-%}
+{% endif %}
+
+{% if "nova" in service_list %}
+{%- include "scenario/nova.yaml"-%}
+{% endif %}
+
+{% if "glance" in service_list %}
+{%- include "scenario/glance.yaml"-%}
+{% endif %}
+
+{% if "neutron" in service_list %}
+{%- include "scenario/neutron.yaml"-%}
+{% endif %}

tasks/openstack/task_arguments.yaml

@@ -0,0 +1,20 @@
+---
+  service_list:
+    - authentication
+    - nova
+    - neutron
+    - keystone
+    - cinder
+    - glance
+  use_existing_users: false
+  image_name: "^(cirros.*-disk|TestVM)$"
+  flavor_name: "m1.tiny"
+  glance_image_location: ""
+  smoke: true
+  users_amount: 1
+  tenants_amount: 1
+  controllers_amount: 3
+  compute_amount: 77
+  storage_amount: 20
+  network_amount: 1
+

tests/ci/hooks/certification_post_test_hook.sh

@@ -5,7 +5,7 @@ source $SCRIPT_DIR/../rally_gate_functions.sh
 
 setUp
 
-TASK=$RALLY_DIR/certification/openstack/task.yaml
+TASK=$RALLY_DIR/tasks/openstack/task.yaml
 TASK_ARGS=$RALLY_DIR/rally-jobs/certifcation_task_args.yaml
 
 TASK_ARGS="--task-args-file $TASK_ARGS"