openstack/api-site
Code Issues Proposed changes

Merge "Text updates to firstapp tutorial"

Browse Source
This commit is contained in:
Jenkins 2015-05-01 01:28:01 +00:00 committed by Gerrit Code Review
commit e7040ded4f
4 changed files with 88 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

38
openstack-firstapp/doc/source/section1.rst
View File

@ -20,26 +20,14 @@ What you will learn
Deploying applications in a cloud environment can be very different from Deploying applications in a cloud environment can be very different from
deploying them in a traditional IT environment. This guide teaches you how to deploying them in a traditional IT environment. This guide teaches you how to
deploy applications on OpenStack and some best practices for cloud application deploy applications on OpenStack and some best practices for cloud application
development. Overall, this guide describes: development.
* :doc:`/section1`: How to manage resources to develop a basic cloud application
* :doc:`/section2`: The architecture of a sample cloud-based application
* :doc:`/section3`: The importance of message queues
* :doc:`/section4`: How to scale up and down in response to changes in
application load
* :doc:`/section5`: How to use object or block storage to create persistence
* :doc:`/section6`: How to orchestrate your cloud for better control of the
environment
* :doc:`/section7`: Networking choices and actions to help relieve
potential congestion
* :doc:`/section8`: Advice for developers who are new to operations
* :doc:`/section9`: How to distribute your application across multiple regions or clouds
A general overview A general overview
~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~
This tutorial shows two applications. The first application is a simple This tutorial shows two applications. The first application is a simple
fractal generator that uses mathematical equations to generate images. We show fractal generator that uses mathematical equations to generate beautiful
`fractal images <http://en.wikipedia.org/wiki/Fractal>`_ . We show
you this application in its entirety so that you can compare it to the second, you this application in its entirety so that you can compare it to the second,
more robust, application. more robust, application.
@ -50,10 +38,10 @@ The second application is an OpenStack application that enables you to:
* Make cloud-related architecture decisions such as turning * Make cloud-related architecture decisions such as turning
functions into micro-services and modularizing them. functions into micro-services and modularizing them.
* Scale available resources up and down. * Scale available resources up and down.
* Use object and block storage for file and database persistence. * Use Object and Block storage for file and database persistence.
* Use Orchestration services to automatically adjust to the environment. * Use Orchestration services to automatically adjust to the environment.
* Customize networking for better performance and segregation. * Customize networking for better performance and segregation.
* Learn some crazy things that you might not think to do ;) * Explore and apply advanced OpenStack cloud features.
Choose your OpenStack SDK Choose your OpenStack SDK
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
@ -185,7 +173,7 @@ How you'll interact with OpenStack
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In this tutorial, you interact with your OpenStack cloud through one of the In this tutorial, you interact with your OpenStack cloud through one of the
SDKs listed in "Choosing your OpenStack SDK." The code snippets in this SDKs listed in "Choose your OpenStack SDK." The code snippets in this
initial version of the guide assume that you're using Libcloud. initial version of the guide assume that you're using Libcloud.
.. only:: fog .. only:: fog
@ -619,14 +607,16 @@ Don't worry if these concepts are not yet completely clear. In
* :doc:`/section8`: Get advice about operations * :doc:`/section8`: Get advice about operations
* :doc:`/section9`: Learn some crazy things that you might not think to do ;) * :doc:`/section9`: Learn some crazy things that you might not think to do ;)
Full example code .. todo:: List the next sections here or simply reference section2.
~~~~~~~~~~~~~~~~~
The following file contains all the code snippets that this tutorial Complete code sample
describes. This comprehensive code sample lets you see and run the code ~~~~~~~~~~~~~~~~~~~~
snippets in context.
Before you run this program, confirm that you have set your authentication The following file contains all of the code from this section of the
tutorial. This comprehensive code sample lets you view and run the code
as a single script.
Before you run this script, confirm that you have set your authentication
information, the flavor ID, and image ID. information, the flavor ID, and image ID.
.. only:: libcloud .. only:: libcloud

59
openstack-firstapp/doc/source/section2.rst
View File

@ -2,14 +2,6 @@
Introduction to the fractals application architecture Introduction to the fractals application architecture
===================================================== =====================================================
This tutorial works with a scalable cloud application that generates
`fractals <http://en.wikipedia.org/wiki/Fractal>`_ - beautiful images made
using only mathematics, like the following image.
.. figure:: images/fractal-example.png
:scale: 50%
:align: left
This section introduces the application architecture and explains how it was This section introduces the application architecture and explains how it was
designed to take advantage of cloud features in general and OpenStack in designed to take advantage of cloud features in general and OpenStack in
particular. It also describes some commands in the previous section. particular. It also describes some commands in the previous section.
@ -45,8 +37,12 @@ particular. It also describes some commands in the previous section.
Cloud application architecture principles Cloud application architecture principles
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Cloud applications typically share several design principles. These principles Cloud applications typically share several design principles.
influenced many Fractals application design decisions. These principles influenced the design of the Fractals application.
.. todo:: Do you want to state the core design priniciples or assume
the reader can follow below.
Modularity and micro-services Modularity and micro-services
----------------------------- -----------------------------
@ -142,7 +138,7 @@ logical application functions, we can change this easily.
.. graphviz:: images/architecture.dot .. graphviz:: images/architecture.dot
Message queues are used to facilitate communication between the Message queues are used to facilitate communication between the
Fractal application services. The Fractal application uses a so-called `work queue Fractal application services. The Fractal application uses a `work queue
<https://www.rabbitmq.com/tutorials/tutorial-two-python.html>`_ (or <https://www.rabbitmq.com/tutorials/tutorial-two-python.html>`_ (or
task queue) to distribute tasks to the worker services. task queue) to distribute tasks to the worker services.
@ -152,8 +148,8 @@ multiple clerks. The message queue in our application provides a feed
of work requests that can be taken one-at-a-time by worker services, of work requests that can be taken one-at-a-time by worker services,
whether there is a single worker service or hundreds of them. whether there is a single worker service or hundreds of them.
This is a `useful pattern <https://msdn.microsoft.com/en-us/library/dn568101.aspx>`_ for many This is a `useful pattern <https://msdn.microsoft.com/en-us/library/dn568101.aspx>`_
cloud applications that have long lists of requests coming in and a for many cloud applications that have long lists of requests coming in and a
pool of resources from which to service them. This also means that a pool of resources from which to service them. This also means that a
worker may crash and the tasks will be processed by other workers. worker may crash and the tasks will be processed by other workers.
@ -554,7 +550,7 @@ with :code:`faafo get --help`, :code:`faafo list --help`, and
.. note:: The application stores the generated fractal images directly .. note:: The application stores the generated fractal images directly
in the database used by the API service instance. Storing in the database used by the API service instance. Storing
image files in database is not good practice. We're doing it image files in a database is not good practice. We're doing it
here as an example only as an easy way to allow multiple here as an example only as an easy way to allow multiple
instances to have access to the data. For best practice, we instances to have access to the data. For best practice, we
recommend storing objects in Object Storage, which is recommend storing objects in Object Storage, which is
@ -564,37 +560,36 @@ Next steps
~~~~~~~~~~ ~~~~~~~~~~
You should now have a basic understanding of the architecture of You should now have a basic understanding of the architecture of
cloud-based applications. In addition, you now have had practice cloud-based applications. In addition, you have had practice
starting new instances, automatically configuring them at boot, and starting new instances, automatically configuring them at boot, and
even modularizing an application so that you may use multiple even modularizing an application so that you may use multiple
instances to run it. These are the basic steps for requesting and instances to run it. These are the basic steps for requesting and
using compute resources in order to run your application on an using compute resources in order to run your application on an
OpenStack cloud. OpenStack cloud.
From here, you should go to :doc:`/section3` to learn how to scale the From here, you should go to :doc:`/section3` to learn how to scale your
application further. Alternately, you may jump to any of these application further. Alternatively, you may jump to any of these
sections: sections:
* :doc:`/section4`: to learn how to make your application more durable * :doc:`/section4`: Learn how to use Object Storage to make your application more durable
using Object Storage * :doc:`/section5`: Migrate the database to block storage, or use
* :doc:`/section5`: to migrate the database to block storage, or use the database-as-a-service component
the database-as-as-service component * :doc:`/section6`: Automatically orchestrate the application
* :doc:`/section6`: to automatically orchestrate the application * :doc:`/section7`: Learn about more complex networking
* :doc:`/section7`: to learn about more complex networking * :doc:`/section8`: Get advice about operations
* :doc:`/section8`: for advice for developers new to operations * :doc:`/section9`: Learn some crazy things that you might not think to do ;)
Full example code Complete code sample
~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
Here's every code snippet into a single file, in case you want to run The following file contains all of the code from this section of the tutorial.
it all in one, or you are so experienced you don't need instruction ;) This comprehensive code sample lets you view and run the code as a single script.
If you are going to use this, don't forget to set your authentication
information and the flavor and image ID. Before you run this script, confirm that you have set your authentication
information, the flavor ID, and image ID.
.. only:: libcloud .. only:: libcloud
.. literalinclude:: ../../samples/libcloud/section2.py .. literalinclude:: ../../samples/libcloud/section2.py
:language: python :language: python

85
openstack-firstapp/doc/source/section3.rst
View File

@ -11,8 +11,8 @@ Scaling out
One of the most-often cited reasons for designing applications using One of the most-often cited reasons for designing applications using
cloud patterns is the ability to **scale out**. That is: to add cloud patterns is the ability to **scale out**. That is: to add
additional resources as required. This is in contrast to the previous additional resources as required. This is in contrast to the previous
mentality of increasing capacity by scaling the size of existing strategy of increasing capacity by scaling up the size of existing
resources up. In order for scale out to be feasible, you'll need to resources. In order for scale out to be feasible, you'll need to
do two things: do two things:
* Architect your application to make use of additional resources. * Architect your application to make use of additional resources.
@ -60,14 +60,15 @@ will be clear that we need more resources.
Generate load Generate load
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
You can test for yourself what happens when the Fractals app is under You can test for yourself what happens when the Fractals application is under
loaded by: load by:
* maxing out the CPU of the existing worker instances (loading the worker) * maxing out the CPU of the existing worker instances (loading the worker)
* generating a lot of API requests (load up the API) * generating a lot of API requests (load up the API)
Generate a lot of worker load
----------------------------- Create a greater number of tasks
--------------------------------
Use SSH to login to the controller instance, :code:`app-controller`, Use SSH to login to the controller instance, :code:`app-controller`,
using the previous added SSH keypair. using the previous added SSH keypair.
@ -80,7 +81,7 @@ using the previous added SSH keypair.
controller instance and USERNAME to the appropriate controller instance and USERNAME to the appropriate
username. username.
Call the Fractal app's command line interface (:code:`faafo`) to Call the Fractal application's command line interface (:code:`faafo`) to
request the generation of 5 large fractals. request the generation of 5 large fractals.
:: ::
@ -99,8 +100,9 @@ of more than 1 means we are at capacity.
.. note:: Replace :code:`IP_WORKER` with the IP address of the worker .. note:: Replace :code:`IP_WORKER` with the IP address of the worker
instance and USERNAME to the appropriate username. instance and USERNAME to the appropriate username.
Generate a lot of API load
-------------------------- Create a greater number of API service requests
-----------------------------------------------
API load is a slightly different problem to the previous one regarding API load is a slightly different problem to the previous one regarding
capacity to work. We can simulate many requests to the API as follows: capacity to work. We can simulate many requests to the API as follows:
@ -116,7 +118,7 @@ using the previous added SSH keypair.
controller instance and USERNAME to the appropriate controller instance and USERNAME to the appropriate
username. username.
Call the Fractal app's command line interface (:code:`faafo`) in a for Call the Fractal application's command line interface (:code:`faafo`) in a for
loop to send many requests to the API. The following command will loop to send many requests to the API. The following command will
request a random set of fractals, 500 times: request a random set of fractals, 500 times:
@ -150,9 +152,8 @@ Remove the old app
------------------ ------------------
Go ahead and delete the existing instances and security groups you Go ahead and delete the existing instances and security groups you
created in previous sections. Remember; when components in the cloud created in previous sections. Remember, when instances in the cloud
aren't doing what you want them to do, just remove them and re-create are no longer working, remove them and re-create something new.
something new.
.. only:: libcloud .. only:: libcloud
@ -177,8 +178,8 @@ required security groups.
A Floating IP helper function A Floating IP helper function
----------------------------- -----------------------------
Define a short function to locate unused or allocate a new floating Define a short function to locate unused IPs or allocate a new floating
IP. This saves a few lines of boring code and prevents you from IP. This saves a few lines of code and prevents you from
reaching your Floating IP quota too quickly. reaching your Floating IP quota too quickly.
.. only:: libcloud .. only:: libcloud
@ -190,10 +191,11 @@ reaching your Floating IP quota too quickly.
Splitting off the database and message queue Splitting off the database and message queue
-------------------------------------------- --------------------------------------------
Prior to scaling out our application services like the API service or Prior to scaling out our application services, like the API service or
the workers we have to add a central database and messaging instance, the workers, we have to add a central database and messaging instance,
called :code:`app-services`, that will be used to track the state of called :code:`app-services`. The database and messaging queue will be used
the fractals and to coordinate the communication between the services. to track the state of the fractals and to coordinate the communication
between the services.
.. only:: libcloud .. only:: libcloud
@ -235,7 +237,6 @@ explain in :doc:`/section7`.
load simulation. load simulation.
Scaling the workers Scaling the workers
------------------- -------------------
@ -277,7 +278,7 @@ You will see that the Fractals app has a few new features.
.. note:: Replace :code:`IP_API_1` with the IP address of the first .. note:: Replace :code:`IP_API_1` with the IP address of the first
API instance and USERNAME to the appropriate username. API instance and USERNAME to the appropriate username.
Use the Fractal app's command line interface to generate fractals Use the Fractal application's command line interface to generate fractals
:code:`faafo create`. Watch the progress of fractal generation with :code:`faafo create`. Watch the progress of fractal generation with
the :code:`faafo list`. Use :code:`faafo UUID` to examine some of the the :code:`faafo list`. Use :code:`faafo UUID` to examine some of the
fractals. The generated_by field will show which worker created the fractals. The generated_by field will show which worker created the
@ -339,7 +340,7 @@ redundant web services. If one dies, the others can be used.
corresponding Floating IPs. Replace FRACTAL_UUID the UUID corresponding Floating IPs. Replace FRACTAL_UUID the UUID
of an existing fractal. of an existing fractal.
Go ahead and test the fault tolerance. Start killing workers and API Go ahead and test the fault tolerance. Start destroying workers and API
instances. As long as you have one of each, your application should instances. As long as you have one of each, your application should
be fine. There is one weak point though. The database contains the be fine. There is one weak point though. The database contains the
fractals and fractal metadata. If you lose that instance, the fractals and fractal metadata. If you lose that instance, the
@ -364,36 +365,38 @@ and autoscaling capabilities to do steps like this automatically.
Next steps Next steps
~~~~~~~~~~ ~~~~~~~~~~
You should now be fairly confident about starting new instance, and You should be fairly confident now about starting new instances, and
about segregating services of an application between them. distributing services from an application amongst the instances.
As mentioned in :doc:`/section2` the generated fractals images will be As mentioned in :doc:`/section2` the generated fractal images will be
saved on the local filesystem of the API service instances. Because we saved on the local filesystem of the API service instances. Because we
now have multiple API instances up and running the generated fractal now have multiple API instances up and running, the fractal
images will be spreaded accross multiple API services, stored on local images will be spread across multiple API services. This results in a number of
instance filesystems. This ends in a lot of :code:`IOError: [Errno 2] :code:`IOError: [Errno 2] No such file or directory` exceptions when trying to download a
No such file or directory` exceptions when trying to download a
fractal image from an API service instance not holding the fractal fractal image from an API service instance not holding the fractal
image on its local filesystem. image on its local filesystem.
From here, you should go to :doc:`/section4` to learn how to use From here, you should go to :doc:`/section4` to learn how to use
Object Storage to solve this problem in a elegant way. Alternately, Object Storage to solve this problem in a elegant way. Alternatively,
you may jump to any of these sections: you may jump to any of these sections:
* :doc:`/section5`: to migrate the database to block storage, or use * :doc:`/section5`: Migrate the database to block storage, or use
the database-as-as-service component the database-as-a-service component
* :doc:`/section6`: to automatically orchestrate the application * :doc:`/section6`: Automatically orchestrate your application
* :doc:`/section7`: to learn about more complex networking * :doc:`/section7`: Learn about complex networking
* :doc:`/section8`: for advice for developers new to operations * :doc:`/section8`: Get advice about operations
* :doc:`/section9`: Learn some crazy things that you might not think to do ;)
Full example code Complete code sample
~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
Here's every code snippet into a single file, in case you want to run The following file contains all of the code from this
it all in one, or you are so experienced you don't need instruction ;) section of the tutorial. This comprehensive code sample lets you view
If you are going to use this, don't forget to set your authentication and run the code as a single script.
information and the flavor and image ID.
Before you run this script, confirm that you have set your authentication
information, the flavor ID, and image ID.
.. only:: libcloud .. only:: libcloud

6
openstack-firstapp/doc/source/section6.rst
View File

@ -1,6 +1,6 @@
========================== =============
Section Six: Orchestration Orchestration
========================== =============
.. todo:: Needs to be restructured so that the fractals app is used as the example for the explanatory material. .. todo:: Needs to be restructured so that the fractals app is used as the example for the explanatory material.