openstack/swift
Code Issues Proposed changes

merged with trunk

Browse Source
This commit is contained in:
John Dickinson 2010-08-22 13:22:27 -05:00
commit 078ced3ce6
5 changed files with 245 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
bin/swift-init
View File

@ -180,7 +180,7 @@ if command == 'restart':
for server in servers:
do_start(server)
if command == 'reload':
if command == 'reload' or command == 'force-reload':
for server in servers:
do_stop(server, graceful=True)
do_start(server)

96
doc/source/admin_guide.rst
View File

@ -108,8 +108,100 @@ different distro or OS, some care should be taken before using in production.
Cluster Health
--------------
TODO: Greg, add docs here about how to use swift-stats-populate, and
swift-stats-report
There is a swift-stats-report tool for measuring overall cluster health. This
is accomplished by checking if a set of deliberately distributed containers and
objects are currently in their proper places within the cluster.
For instance, a common deployment has three replicas of each object. The health
of that object can be measured by checking if each replica is in its proper
place. If only 2 of the 3 is in place the object's heath can be said to be at
66.66%, where 100% would be perfect.
A single object's health, especially an older object, usually reflects the
health of that entire partition the object is in. If we make enough objects on
a distinct percentage of the partitions in the cluster, we can get a pretty
valid estimate of the overall cluster health. In practice, about 1% partition
coverage seems to balance well between accuracy and the amount of time it takes
to gather results.
The first thing that needs to be done to provide this health value is create a
new account solely for this usage. Next, we need to place the containers and
objects throughout the system so that they are on distinct partitions. The
swift-stats-populate tool does this by making up random container and object
names until they fall on distinct partitions. Last, and repeatedly for the life
of the cluster, we need to run the swift-stats-report tool to check the health
of each of these containers and objects.
These tools need direct access to the entire cluster and to the ring files
(installing them on an auth server or a proxy server will probably do). Both
swift-stats-populate and swift-stats-report use the same configuration file,
/etc/swift/stats.conf. Example conf file::
[stats]
auth_url = http://saio:11000/v1.0
auth_user = test:tester
auth_key = testing
There are also options for the conf file for specifying the dispersion coverage
(defaults to 1%), retries, concurrency, CSV output file, etc. though usually
the defaults are fine.
Once the configuration is in place, run `swift-stats-populate -d` to populate
the containers and objects throughout the cluster.
Now that those containers and objects are in place, you can run
`swift-stats-report -d` to get a dispersion report, or the overall health of
the cluster. Here is an example of a cluster in perfect health::
$ swift-stats-report -d
Queried 2621 containers for dispersion reporting, 19s, 0 retries
100.00% of container copies found (7863 of 7863)
Sample represents 1.00% of the container partition space
Queried 2619 objects for dispersion reporting, 7s, 0 retries
100.00% of object copies found (7857 of 7857)
Sample represents 1.00% of the object partition space
Now I'll deliberately double the weight of a device in the object ring (with
replication turned off) and rerun the dispersion report to show what impact
that has::
$ swift-ring-builder object.builder set_weight d0 200
$ swift-ring-builder object.builder rebalance
...
$ swift-stats-report -d
Queried 2621 containers for dispersion reporting, 8s, 0 retries
100.00% of container copies found (7863 of 7863)
Sample represents 1.00% of the container partition space
Queried 2619 objects for dispersion reporting, 7s, 0 retries
There were 1763 partitions missing one copy.
77.56% of object copies found (6094 of 7857)
Sample represents 1.00% of the object partition space
You can see the health of the objects in the cluster has gone down
significantly. Of course, I only have four devices in this test environment, in
a production environment with many many devices the impact of one device change
is much less. Next, I'll run the replicators to get everything put back into
place and then rerun the dispersion report::
... start object replicators and monitor logs until they're caught up ...
$ swift-stats-report -d
Queried 2621 containers for dispersion reporting, 17s, 0 retries
100.00% of container copies found (7863 of 7863)
Sample represents 1.00% of the container partition space
Queried 2619 objects for dispersion reporting, 7s, 0 retries
100.00% of object copies found (7857 of 7857)
Sample represents 1.00% of the object partition space
So that's a summation of how to use swift-stats-report to monitor the health of
a cluster. There are a few other things it can do, such as performance
monitoring, but those are currently in their infancy and little used. For
instance, you can run `swift-stats-populate -p` and `swift-stats-report -p` to
get performance timings (warning: the initial populate takes a while). These
timings are dumped into a CSV file (/etc/swift/stats.csv by default) and can
then be graphed to see how cluster performance is trending.
------------------------
Debugging Tips and Tools

143
doc/source/debian_package_guide.rst Normal file
View File

@ -0,0 +1,143 @@
=======================================================
Building and Deploying Custom Debian packages for Swift
=======================================================
This documents setting up the prerequisites, downloading the code and building
Debian packages necessary to deploy the various components of the swift
project code. These steps were done on a server running
*Ubuntu 10.04 LTS (Lucid Lynx)*, but should give a good idea what to do on
other distros.
------------------------------------------
Instructions for Deploying Swift-Core PPAs
------------------------------------------
These packages are built from the current development branch "trunk"
* Add swift-core/ppa repository. As root:::
apt-get install python-software-properties
add-apt-repository ppa:swift-core/trunk
apt-get update
* Install the swift base packages::
apt-get install python-swift
apt-get install swift
* Install the swift packages depending on your implementations::
apt-get install swift-auth
apt-get install swift-proxy
apt-get install swift-account
apt-get install swift-container
apt-get install swift-object
* Copy sample configuration files to `/etc/swift` directory
and rename them to `*.conf files`::
cp -a /usr/share/doc/swift/*.conf-sample /etc/swift/
cd /etc/swift ; rename 's/\-sample$//' *.conf-sample
* For servers running the swift-account, swift-container or
swift-object the rsync.conf file should be moved to
the `/etc` directory::
cd /etc/swift
mv rsyncd.conf /etc
* Modify configuration files to meet your implementation requirements
the defaults have been not been geared to a multi-server implementation.
---------------------------------------------------
Instructions for Building Debian Packages for Swift
---------------------------------------------------
* Add swift-core/ppa repository and install prerequisites. As root::
apt-get install python-software-properties
add-apt-repository ppa:swift-core/ppa
apt-get update
apt-get install curl gcc bzr python-configobj python-coverage python-dev
python-nose python-setuptools python-simplejson python-xattr
python-webob python-eventlet python-greenlet debhelper python-sphinx
python-all python-openssl
* As you
#. Tell bzr who you are::
bzr whoami '<Your Name> <youremail@.example.com>'
bzr lp-login <your launchpad id>
#. Create a local bazaar repository for dev/testing::
bzr init-repo swift
#. Pull down the swift/debian files::
cd swift
bzr branch lp:~swift-core/swift/debian
#. If you want to merge in a branch::
cd debian
bzr merge lp:<path-to-branch>
#. Create the debian packages::
cd debian
bzr bd --builder='debuild -uc -us'
#. Upload packages to your target servers::
cd ..
scp *.deb root@<swift-target-server>:~/.
----------------------------------------------------
Instructions for Deploying Debian Packages for Swift
----------------------------------------------------
* On a Target Server, As root:
#. Setup the swift ppa::
add-apt-repository ppa:swift-core/ppa
apt-get update
#. Install dependencies::
apt-get install rsync python-openssl python-setuptools python-webob
python-simplejson python-xattr python-greenlet python-eventlet
#. Install base packages::
dpkg -i python-swift_<version>_all.deb
dpkg -i swift_<version>_all.deb
#. Install packages depending on your implementation::
dpkg -i swift-auth_<version>_all.deb
dpkg -i swift-proxy_<version>_all.deb
dpkg -i swift-account_<version>_all.deb
dpkg -i swift-container_<version>_all.deb
dpkg -i swift-object_<version>_all.deb
dpkg -i swift-doc_<version>_all.deb
#. Copy sample configuration files to `/etc/swift` directory
and rename them to `*.conf files`::
cp -a /usr/share/doc/swift/*.conf-sample /etc/swift/
cd /etc/swift
rename 's/\-sample$//' *.conf-sample
#. For servers running the swift-account, swift-container or
swift-object the rsync.conf file should be moved to
the `/etc` directory::
cd /etc/swift/
mv rsyncd.conf /etc
#. Modify configuration files to meet your implementation requirements
the defaults have been not been geared to a multi-server implementation.

8
doc/source/development_saio.rst
View File

@ -11,8 +11,12 @@ virtual machine will emulate running a four node Swift cluster. It assumes
you're using *VMware Fusion 3* on *Mac OS X Snow Leopard*, but should give a
good idea what to do on other environments.
* Get the *Ubuntu 10.04 LTS (Lucid Lynx)* server image from:
http://cdimage.ubuntu.com/releases/10.04/release/ubuntu-10.04-dvd-amd64.iso
* Get the *Ubuntu 10.04 LTS (Lucid Lynx)* server image:
- Ubuntu Server ISO: http://releases.ubuntu.com/10.04/ubuntu-10.04.1-server-amd64.iso (682 MB)
- Ubuntu Live/Install: http://cdimage.ubuntu.com/releases/10.04/release/ubuntu-10.04-dvd-amd64.iso (4.1 GB)
- Ubuntu Mirrors: https://launchpad.net/ubuntu/+cdmirrors
* Create guest virtual machine:
#. `Continue without disc`

1
doc/source/index.rst
View File

@ -40,6 +40,7 @@ Deployment:
deployment_guide
admin_guide
debian_package_guide
End User Guides: