1880351f1a
Imagine a 3-zone ring, and consider a partition in that ring with replicas placed as follows: * replica 0 is on device A (zone 2) * replica 1 is on device B (zone 1) * replica 2 is on device C (zone 2) Further, imagine that there are zero parts_wanted in all of zone 3; that is, zone 3 is completely full. However, zones 1 and 2 each have at least one parts_wanted on at least one device. When the ring builder goes to gather replicas to move, it gathers replica 0 because there are three zones available, but the replicas are only in two of them. Then, it places replica 0 in zone 1 or 2 somewhere because those are the only zones with parts_wanted. Notice that this does *not* do anything to spread the partition out better. Then, on the next rebalance, replica 0 gets picked up and moved (again) but doesn't improve its placement (again). If your builder has min_part_hours > 0 (and it should), then replicas 1 and 2 cannot move at all. A coworker observed the bug because a customer had such a partition, and its replica 2 was on a zero-weight device. He thought it odd that a zero-weight device should still have one partition on it despite the ring having been rebalanced dozens of times. Even if you don't have zero-weight devices, having a bunch of partitions trade places on each rebalance isn't particularly good. Note that this only happens with an unbalanceable ring; if the ring *can* balance, the gathered partitions will swap places, but they will get spread across more zones, so they won't get gathered up again on the next rebalance. Change-Id: I8f44f032caac25c44778a497dedf23f5cb61b6bb Closes-Bug: 1400083 |
||
---|---|---|
bin | ||
doc | ||
etc | ||
examples | ||
swift | ||
test | ||
.coveragerc | ||
.functests | ||
.gitignore | ||
.gitreview | ||
.mailmap | ||
.probetests | ||
.unittests | ||
AUTHORS | ||
babel.cfg | ||
CHANGELOG | ||
CONTRIBUTING.md | ||
LICENSE | ||
MANIFEST.in | ||
README.md | ||
requirements.txt | ||
setup.cfg | ||
setup.py | ||
test-requirements.txt | ||
tox.ini |
Swift
A distributed object storage system designed to scale from a single machine to thousands of servers. Swift is optimized for multi-tenancy and high concurrency. Swift is ideal for backups, web and mobile content, and any other unstructured data that can grow without bound.
Swift provides a simple, REST-based API fully documented at http://docs.openstack.org/.
Swift was originally developed as the basis for Rackspace's Cloud Files and was open-sourced in 2010 as part of the OpenStack project. It has since grown to include contributions from many companies and has spawned a thriving ecosystem of 3rd party tools. Swift's contributors are listed in the AUTHORS file.
Docs
To build documentation install sphinx (pip install sphinx
), run
python setup.py build_sphinx
, and then browse to /doc/build/html/index.html.
These docs are auto-generated after every commit and available online at
http://docs.openstack.org/developer/swift/.
For Developers
The best place to get started is the "SAIO - Swift All In One". This document will walk you through setting up a development cluster of Swift in a VM. The SAIO environment is ideal for running small-scale tests against swift and trying out new features and bug fixes.
You can run unit tests with .unittests
and functional tests with
.functests
.
If you would like to start contributing, check out these notes to help you get started.
Code Organization
- bin/: Executable scripts that are the processes run by the deployer
- doc/: Documentation
- etc/: Sample config files
- swift/: Core code
- account/: account server
- common/: code shared by different modules
- middleware/: "standard", officially-supported middleware
- ring/: code implementing Swift's ring
- container/: container server
- obj/: object server
- proxy/: proxy server
- test/: Unit and functional tests
Data Flow
Swift is a WSGI application and uses eventlet's WSGI server. After the
processes are running, the entry point for new requests is the Application
class in swift/proxy/server.py
. From there, a controller is chosen, and the
request is processed. The proxy may choose to forward the request to a back-
end server. For example, the entry point for requests to the object server is
the ObjectController
class in swift/obj/server.py
.
For Deployers
Deployer docs are also available at http://docs.openstack.org/developer/swift/. A good starting point is at http://docs.openstack.org/developer/swift/deployment_guide.html
You can run functional tests against a swift cluster with .functests
. These
functional tests require /etc/swift/test.conf
to run. A sample config file
can be found in this source tree in test/sample.conf
.
For Client Apps
For client applications, official Python language bindings are provided at http://github.com/openstack/python-swiftclient.
Complete API documentation at http://docs.openstack.org/api/openstack-object-storage/1.0/content/
For more information come hang out in #openstack-swift on freenode.
Thanks,
The Swift Development Team