From 3e99e21f079dd0563db198d71a34023b6dac8245 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C5=81ukasz=20Ole=C5=9B?= Date: Tue, 19 Jan 2016 23:39:21 +0100 Subject: [PATCH] Move all text from README to documentation Change-Id: I038dd56fa82e3915305fca79218eaaf1b2303056 --- README.md | 304 ------------------------------------ README.rst | 6 + doc/source/index.rst | 4 +- doc/source/installation.rst | 79 ++++++++++ doc/source/usage.rst | 188 ++++++++++++++++++++++ setup.cfg | 2 +- 6 files changed, 277 insertions(+), 306 deletions(-) delete mode 100644 README.md create mode 100644 README.rst create mode 100644 doc/source/installation.rst create mode 100644 doc/source/usage.rst diff --git a/README.md b/README.md deleted file mode 100644 index ba96f79f..00000000 --- a/README.md +++ /dev/null @@ -1,304 +0,0 @@ -[![Build Status](https://travis-ci.org/Mirantis/solar.svg?branch=master)](https://travis-ci.org/Mirantis/solar) [![Coverage Status](https://coveralls.io/repos/Mirantis/solar/badge.svg?branch=master&service=github)](https://coveralls.io/github/Mirantis/solar?branch=master) - -# Requirements - -## Supported development platforms - -Linux or MacOS - -## Additional software - -[VirtualBox](https://www.virtualbox.org/wiki/Downloads): 5.x - -[Vagrant](http://www.vagrantup.com/downloads.html): 1.7.x - -Note: Make sure that [Vagrant VirtualBox Guest plugin](https://github.com/dotless-de/vagrant-vbguest) is installed `vagrant plugin install vagrant-vbguest` - -Note: If you are using VirtualBox 5.0 it's worth uncommenting paravirtprovider setting in `vagrant-settings.yaml` for speed improvements: -```bash -paravirtprovider: kvm -``` - -For details see Customizing `vagrant-settings.yaml` section. - -# Setup development env - -Setup environment: -```bash -cd solar -vagrant up -``` - -Login into vm, the code is available in /vagrant directory -```bash -vagrant ssh -solar --help -``` - - -Get ssh details for running slave nodes (vagrant/vagrant): -```bash -vagrant ssh-config -``` - -You can make/restore snapshots of boxes (this is way faster than reprovisioning them) -with the `snapshotter.py` script: - -```bash -./snapshotter.py take -n my-snapshot -./snapshotter.py show -./snapshotter.py restore -n my-snapshot -``` -`snapshoter.py` to run requires python module `click`. - -1. On debian based systems you can install it via `sudo aptitude install python-click-cli`, -2. On fedora 22 you can install it via `sudo dnf install python-click`, -3. If you use virtualenv or similar tool then you can install it just with `pip install click`, -4. If you don't have virtualenv and your operating system does not provide package for it then `sudo pip install click`. -5. If you don't have `pip` then [install it](https://pip.pypa.io/en/stable/installing/) and then execute command step 4. - -# Solar usage - -For now all commands should be executed from `solar-dev` machine from `/vagrant` directory. - -Basic flow is: - -1. Create some resources (look at `solar-resources/examples/openstack/openstack.py`) and connect - them between each other, and place them on nodes. -1. Run `solar changes stage` (this stages the changes) -1. Run `solar changes process` (this prepares orchestrator graph, returning - change UUID) -1. Run `solar orch run-once ` (or `solar orch run-once last` - to run the lastly created graph) -1. Observe progress of orch with `watch 'solar orch report '` - (or `watch 'solar orch report last'`). - -Some very simple cluster setup: -```bash -cd /vagrant - -solar resource create nodes templates/nodes '{"count": 2}' -solar resource create mariadb_service resources/mariadb_service '{"image": "mariadb", "root_password": "mariadb", "port": 3306}' -solar resource create keystone_db resources/mariadb_db/ '{"db_name": "keystone_db", "login_user": "root"}' -solar resource create keystone_db_user resources/mariadb_user/ user_name=keystone user_password=keystone # another valid format - -solar connect node1 mariadb_service -solar connect node1 keystone_db -solar connect mariadb_service keystone_db '{"root_password": "login_password", "port": "login_port", "ip": "db_host"}' -# solar connect mariadb_service keystone_db_user 'root_password->login_password port->login_port' # another valid format -solar connect keystone_db keystone_db_user - -solar changes stage -solar changes process -# -solar orch run-once # or solar orch run-once last -watch 'solar orch report ' # or solar orch report last -``` - -You can fiddle with the above configuration like this: -```bash -solar resource update keystone_db_user '{"user_password": "new_keystone_password"}' -solar resource update keystone_db_user user_password=new_keystone_password # another valid format - -solar changes stage -solar changes process - -solar orch run-once -``` - -To get data for the resource `bar` (raw and pretty-JSON): -```bash -solar resource show --tag 'resources/bar' -solar resource show --json --tag 'resources/bar' | jq . -solar resource show --name 'resource_name' -solar resource show --name 'resource_name' --json | jq . -``` - -To clear all resources/connections: -```bash -solar resource clear_all -``` - -Show the connections/graph: -```bash -solar connections show -solar connections graph -``` - -You can also limit graph to show only specific resources: - -```bash -solar connections graph --start-with mariadb_service --end-with keystone_db -``` - -You can make sure that all input values are correct and mapped without duplicating your values with this command: -```bash -solar resource validate -``` - -Disconnect -```bash -solar disconnect mariadb_service node1 -``` - -Tag a resource: -```bash -solar resource tag node1 test-tags # Remove tags -solar resource tag node1 test-tag --delete -``` - -# Low level API - -## Usage: - -Creating resources: - -```python -from solar.core.resource import composer as cr -node1 = cr.create('node1', 'resources/ro_node/', 'rs/', {'ip':'10.0.0.3', 'ssh_key' : '/vagrant/tmp/keys/ssh_private', 'ssh_user':'vagrant'})[0] - -node2 = cr.create('node2', 'resources/ro_node/', 'rs/', {'ip':'10.0.0.4', 'ssh_key' : '/vagrant/tmp/keys/ssh_private', 'ssh_user':'vagrant'})[0] - -keystone_db_data = cr.create('mariadb_keystone_data', 'resources/data_container/', 'rs/', {'image' : 'mariadb', 'export_volumes' : ['/var/lib/mysql'], 'ip': '', 'ssh_user': '', 'ssh_key': ''}, connections={'ip' : 'node2.ip', 'ssh_key':'node2.ssh_key', 'ssh_user':'node2.ssh_user'})[0] - -nova_db_data = cr.create('mariadb_nova_data', 'resources/data_container/', 'rs/', {'image' : 'mariadb', 'export_volumes' : ['/var/lib/mysql'], 'ip': '', 'ssh_user': '', 'ssh_key': ''}, connections={'ip' : 'node1.ip', 'ssh_key':'node1.ssh_key', 'ssh_user':'node1.ssh_user'})[0] -``` - -To make connection after resource is created use `signal.connect`. - -To test notifications: - -```python -keystone_db_data.args # displays node2 IP - -node2.update({'ip': '10.0.0.5'}) - -keystone_db_data.args # updated IP -``` - -If you close the Python shell you can load the resources like this: - -```python -from solar.core import resource -node1 = resource.load('rs/node1') - -node2 = resource.load('rs/node2') - -keystone_db_data = resource.load('rs/mariadb_keystone_data') - -nova_db_data = resource.load('rs/mariadb_nova_data') -``` -Connections are loaded automatically. - -You can also load all resources at once: - -```python -from solar.core import resource -all_resources = resource.load_all('rs') -``` - -## Dry run - -Solar CLI has possibility to show dry run of actions to be performed. -To see what will happen when you run Puppet action, for example, try this: - -```bash -solar resource action keystone_puppet run -d -``` - -This should print out something like this: - -```bash -EXECUTED: -73c6cb1cf7f6cdd38d04dd2d0a0729f8: (0, 'SSH RUN', ('sudo cat /tmp/puppet-modules/Puppetfile',), {}) -3dd4d7773ce74187d5108ace0717ef29: (1, 'SSH SUDO', ('mv "1038cb062449340bdc4832138dca18cba75caaf8" "/tmp/puppet-modules/Puppetfile"',), {}) -ae5ad2455fe2b02ba46b4b7727eff01a: (2, 'SSH RUN', ('sudo librarian-puppet install',), {}) -208764fa257ed3159d1788f73c755f44: (3, 'SSH SUDO', ('puppet apply -vd /tmp/action.pp',), {}) -``` - -By default every mocked command returns an empty string. If you want it to return -something else (to check how would dry run behave in different situation) you provide -a mapping (in JSON format), something along the lines of: - -```bash -solar resource action keystone_puppet run -d -m "{\"73c\": \"mod 'openstack-keystone'\n\"}" -``` - -The above means the return string of first command (with hash `73c6c...`) will be -as specified in the mapping. Notice that in mapping you don't have to specify the -whole hash, just it's unique beginning. Also, you don't have to specify the whole -return string in mapping. Dry run executor can read file and return it's contents -instead, just use the `>` operator when specifying hash: - -```bash -solar resource action keystone_puppet run -d -m "{\"73c>\": \"./Puppetlabs-file\"}" -``` - -# Resource compiling - -You can compile all `meta.yaml` definitions into Python code with classes that -derive from `Resource`. To do this run - -```bash -solar resource compile_all -``` - -This generates file `resources_compiled.py` in the main directory (do not commit -this file into the repo). Then you can import classes from that file, create -their instances and assign values just like these were normal properties. -If your editor supports Python static checking, you will have autocompletion -there too. An example on how to create a node with this: - -```python -import resources_compiled - -node1 = resources_compiled.RoNodeResource('node1', None, {}) -node1.ip = '10.0.0.3' -node1.ssh_key = '/vagrant/.vagrant/machines/solar-dev1/virtualbox/private_key' -node1.ssh_user = 'vagrant' -``` - -# Higher-level API - -There's also a higher-level API that allows to write resource instances in more -functional way, and in particular avoid `for` loops. Here's an example: - -```python -from solar import template - -nodes = template.nodes_from('templates/riak_nodes') - -riak_services = nodes.on_each( - 'resources/riak_node', - { - 'riak_self_name': 'riak{num}', - 'riak_hostname': 'riak_server{num}.solar', - 'riak_name': 'riak{num}@riak_server{num}.solar', - } -) - -riak_master_service = riak_services.take(0) -riak_slave_services = riak_services.tail() - -riak_master_service.connect_list( - riak_slave_services, - { - 'riak_name': 'join_to', - } -) -``` - -For full Riak example, please look at `solar-resources/examples/riak/riaks-template.py`. - -Full documentation of individual functions is found in the `solar/template.py` file. - - -# Customizing vagrant-settings.yaml - -Solar is shipped with sane defaults in `vagrant-setting.yaml_defaults`. If you need to adjust them for your needs, e.g. changing resource allocation for VirtualBox machines, you should just compy the file to `vagrant-setting.yaml` and make your modifications. - -# Image based provisioning with Solar - -* In `vagrant-setting.yaml_defaults` or `vagrant-settings.yaml` file uncomment `preprovisioned: false` line. -* Run `vagrant up`, it will take some time because it builds image for bootstrap and IBP images. -* Now you can run provisioning `/vagrant/solar-resources/examples/provisioning/provision.sh` diff --git a/README.rst b/README.rst new file mode 100644 index 00000000..2784bb2f --- /dev/null +++ b/README.rst @@ -0,0 +1,6 @@ +Solar README +===================== + +Solar is a resource manager and orchestration engine for distributed systems. + +Please read the `Documentation `_ to see how to install and use Solar. For start checkout our `Wordpress tutorial `_ diff --git a/doc/source/index.rst b/doc/source/index.rst index 06974403..47b60906 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -11,6 +11,9 @@ Contents: .. toctree:: :maxdepth: 2 + installation + usage + tutorials/index glossary resource resource_repository @@ -19,7 +22,6 @@ Contents: handler_ansible examples deployment_plan - tutorials/index Indices and tables diff --git a/doc/source/installation.rst b/doc/source/installation.rst new file mode 100644 index 00000000..4d9df630 --- /dev/null +++ b/doc/source/installation.rst @@ -0,0 +1,79 @@ +Installation +============ + +Supported development platforms +------------------------------- + +Linux or MacOS + +Additional software +------------------- + +`VirtualBox `_ 5.x, +`Vagrant `_ 1.7.x + +Note: Make sure that `Vagrant VirtualBox Guest plugin `_ is installed + +.. code-block:: bash + + vagrant plugin install vagrant-vbguest + +Note: If you are using VirtualBox 5.0 it's worth uncommenting paravirtprovider setting in `vagrant-settings.yaml` for speed improvements: + +.. code-block:: bash + + paravirtprovider: kvm + +For details see Customizing `vagrant-settings.yaml` section. + +Setup development env +--------------------- + +Setup environment: + +.. code-block:: bash + + cd solar + vagrant up + +Login into vm, the code is available in /vagrant directory + +.. code-block:: bash + + vagrant ssh + solar --help + +Get ssh details for running slave nodes (vagrant/vagrant): + +.. code-block:: bash + + vagrant ssh-config + +You can make/restore snapshots of boxes (this is way faster than reprovisioning them) +with the `snapshotter.py` script: + +.. code-block:: bash + + ./snapshotter.py take -n my-snapshot + ./snapshotter.py show + ./snapshotter.py restore -n my-snapshot + +`snapshoter.py` to run requires python module `click`. + +1. On debian based systems you can install it via `sudo aptitude install python-click-cli`, +2. On fedora 22 you can install it via `sudo dnf install python-click`, +3. If you use virtualenv or similar tool then you can install it just with `pip install click`, +4. If you don't have virtualenv and your operating system does not provide package for it then `sudo pip install click`. +5. If you don't have `pip` then [install it](https://pip.pypa.io/en/stable/installing/) and then execute command step 4. + +Customizing vagrant-settings.yaml +--------------------------------- + +Solar is shipped with sane defaults in `vagrant-setting.yaml_defaults`. If you need to adjust them for your needs, e.g. changing resource allocation for VirtualBox machines, you should just compy the file to `vagrant-setting.yaml` and make your modifications. + +Image based provisioning with Solar +------------------------------------- + +* In `vagrant-setting.yaml_defaults` or `vagrant-settings.yaml` file uncomment `preprovisioned: false` line. +* Run `vagrant up`, it will take some time because it builds image for bootstrap and IBP images. +* Now you can run provisioning `/vagrant/solar-resources/examples/provisioning/provision.sh` diff --git a/doc/source/usage.rst b/doc/source/usage.rst new file mode 100644 index 00000000..32dec7ba --- /dev/null +++ b/doc/source/usage.rst @@ -0,0 +1,188 @@ +Usage +===== + +CLI API +---------- + +For now all commands should be executed from `solar-dev` machine from `/vagrant` directory. + +1. Create some resources (look at `solar-resources/examples/openstack/openstack.py`) and connect + them between each other, and place them on nodes. +2. Run `solar changes stage` (this stages the changes) +3. Run `solar changes process` (this prepares orchestrator graph, returning + change UUID) +4. Run `solar orch run-once ` (or `solar orch run-once last` + to run the lastly created graph) +5. Observe progress of orch with `watch 'solar orch report '` + (or `watch 'solar orch report last'`). + +Some very simple cluster setup: + +.. code-block:: bash + + cd /vagrant + + solar resource create nodes templates/nodes '{"count": 2}' + solar resource create mariadb_service resources/mariadb_service '{"image": "mariadb", "root_password": "mariadb", "port": 3306}' + solar resource create keystone_db resources/mariadb_db/ '{"db_name": "keystone_db", "login_user": "root"}' + solar resource create keystone_db_user resources/mariadb_user/ user_name=keystone user_password=keystone # another valid format + + solar connect node1 mariadb_service + solar connect node1 keystone_db + solar connect mariadb_service keystone_db '{"root_password": "login_password", "port": "login_port", "ip": "db_host"}' + # solar connect mariadb_service keystone_db_user 'root_password->login_password port->login_port' # another valid format + solar connect keystone_db keystone_db_user + + solar changes stage + solar changes process + # + solar orch run-once # or solar orch run-once last + watch 'solar orch report ' # or solar orch report last + +You can fiddle with the above configuration like this: + +.. code-block:: bash + + solar resource update keystone_db_user '{"user_password": "new_keystone_password"}' + solar resource update keystone_db_user user_password=new_keystone_password # another valid format + + solar changes stage + solar changes process + + solar orch run-once + +To get data for the resource `bar` (raw and pretty-JSON): + +.. code-block:: bash + + solar resource show --tag 'resources/bar' + solar resource show --json --tag 'resources/bar' | jq . + solar resource show --name 'resource_name' + solar resource show --name 'resource_name' --json | jq . + +To clear all resources/connections: + +.. code-block:: bash + + solar resource clear_all + +Show the connections/graph: + +.. code-block:: bash + + solar connections show + solar connections graph + +You can also limit graph to show only specific resources: + +.. code-block:: bash + + solar connections graph --start-with mariadb_service --end-with keystone_db + +You can make sure that all input values are correct and mapped without duplicating your values with this command: + +.. code-block:: bash + + solar resource validate + +Disconnect + +.. code-block:: bash + + solar disconnect mariadb_service node1 + +Tag a resource: + +.. code-block:: bash + + solar resource tag node1 test-tags # Remove tags + solar resource tag node1 test-tag --delete + +Python API +---------- + +Usage +~~~~~ + +Creating resources: + +.. code-block:: python + + from solar.core.resource import composer as cr + node1 = cr.create('node1', 'resources/ro_node/', 'rs/', {'ip':'10.0.0.3', 'ssh_key' : '/vagrant/tmp/keys/ssh_private', 'ssh_user':'vagrant'})[0] + + node2 = cr.create('node2', 'resources/ro_node/', 'rs/', {'ip':'10.0.0.4', 'ssh_key' : '/vagrant/tmp/keys/ssh_private', 'ssh_user':'vagrant'})[0] + + keystone_db_data = cr.create('mariadb_keystone_data', 'resources/data_container/', 'rs/', {'image' : 'mariadb', 'export_volumes' : ['/var/lib/mysql'], 'ip': '', 'ssh_user': '', 'ssh_key': ''}, connections={'ip' : 'node2.ip', 'ssh_key':'node2.ssh_key', 'ssh_user':'node2.ssh_user'})[0] + + nova_db_data = cr.create('mariadb_nova_data', 'resources/data_container/', 'rs/', {'image' : 'mariadb', 'export_volumes' : ['/var/lib/mysql'], 'ip': '', 'ssh_user': '', 'ssh_key': ''}, connections={'ip' : 'node1.ip', 'ssh_key':'node1.ssh_key', 'ssh_user':'node1.ssh_user'})[0] + +To make connection after resource is created use `signal.connect`. + +To test notifications: + +.. code-block:: python + + keystone_db_data.args # displays node2 IP + node2.update({'ip': '10.0.0.5'}) + keystone_db_data.args # updated IP + +If you close the Python shell you can load the resources like this: + +.. code-block:: python + + from solar.core import resource + node1 = resource.load('rs/node1') + + node2 = resource.load('rs/node2') + + keystone_db_data = resource.load('rs/mariadb_keystone_data') + + nova_db_data = resource.load('rs/mariadb_nova_data') + +Connections are loaded automatically. + +You can also load all resources at once: + +.. code-block:: python + + from solar.core import resource + all_resources = resource.load_all('rs') + +Dry run +------- + +Solar CLI has possibility to show dry run of actions to be performed. +To see what will happen when you run Puppet action, for example, try this: + +.. code-block:: python + + solar resource action keystone_puppet run -d + +This should print out something like this: + +.. code-block:: python + + EXECUTED: + 73c6cb1cf7f6cdd38d04dd2d0a0729f8: (0, 'SSH RUN', ('sudo cat /tmp/puppet-modules/Puppetfile',), {}) + 3dd4d7773ce74187d5108ace0717ef29: (1, 'SSH SUDO', ('mv "1038cb062449340bdc4832138dca18cba75caaf8" "/tmp/puppet-modules/Puppetfile"',), {}) + ae5ad2455fe2b02ba46b4b7727eff01a: (2, 'SSH RUN', ('sudo librarian-puppet install',), {}) + 208764fa257ed3159d1788f73c755f44: (3, 'SSH SUDO', ('puppet apply -vd /tmp/action.pp',), {}) + +By default every mocked command returns an empty string. If you want it to return +something else (to check how would dry run behave in different situation) you provide +a mapping (in JSON format), something along the lines of: + +.. code-block:: python + + solar resource action keystone_puppet run -d -m "{\"73c\": \"mod 'openstack-keystone'\n\"}" + +The above means the return string of first command (with hash `73c6c...`) will be +as specified in the mapping. Notice that in mapping you don't have to specify the +whole hash, just it's unique beginning. Also, you don't have to specify the whole +return string in mapping. Dry run executor can read file and return it's contents +instead, just use the `>` operator when specifying hash: + +.. code-block:: python + + solar resource action keystone_puppet run -d -m "{\"73c>\": \"./Puppetlabs-file\"}" diff --git a/setup.cfg b/setup.cfg index 54b81bb6..d775a699 100644 --- a/setup.cfg +++ b/setup.cfg @@ -4,7 +4,7 @@ version = 0.0.1 author = Mirantis Inc. author-email = product@mirantis.com summary = Deployment tool -description-file = README.md +description-file = README.rst license = Apache-2 home-page = http://mirantis.com classifier =