openstack/swift
Code Issues Proposed changes

Remove Lucid/ppa instructions from SAIO guide

Browse Source 
Lucid won't EOL until May of 2014; but I stopped trusting that ppa a long time
ago. Besides with the requires for dnspython and mock where they're at you
almost can't install swift from source on any stock distro and expect tests to
pass with system packages - so we're looking at pypi for depends regardless.

While I'm in there:

* more explanation of <your-user-name> and a helpful find/sed for configs
* group the "setup ~/.bashrc" stuff with the "setup ~/bin" stuff
* some updates/fixes from my experience installing on CentOS
* remove region warnings from remakerings

Change-Id: Ie2e6b06959ab699d853e07e5b7e8cda7036a44fe
This commit is contained in:
Pete Zaitcev 2013-06-13 18:28:00 -06:00
parent 7757144da5
commit 11aaaf1f3f
1 changed files with 92 additions and 68 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

160
doc/source/development_saio.rst
View File

@ -9,44 +9,48 @@ Instructions for setting up a development VM
This section documents setting up a virtual machine for doing Swift development. This section documents setting up a virtual machine for doing Swift development.
The virtual machine will emulate running a four node Swift cluster. The virtual machine will emulate running a four node Swift cluster.
* Get either Ubuntu 12.04 LTS (Precise Pangolin) or Ubuntu 10.04 LTS (Lucid Lynx) server image. * Get an Ubuntu 12.04 LTS (Precise Pangolin) server image or try something Fedora/CentOS.
* Create guest virtual machine from the Ubuntu image. * Create guest virtual machine from the image.
Additional information about setting up a Swift development snapshot on other distributions is Additional information about setting up a Swift development snapshot on other distributions is
available on the wiki at http://wiki.openstack.org/SAIOInstructions. available on the wiki at http://wiki.openstack.org/SAIOInstructions.
----------------------------------------- ----------------------------
Installing dependencies and the core code What's in a <your-user-name>
----------------------------------------- ----------------------------
* On Ubuntu,
Much of the configuration described in this guide requires escalated root
privileges; however, we assume that administrator logs in an unprivileged
user. Swift processes also run under a separate user and group, set by
configuration option, and refered as <your-user-name>:<your-group-name>.
The default user is `swift`, which may not exist on your system.
Perform the following commands as root.
-----------------------
Installing dependencies
-----------------------
* On apt based systems,
#. `apt-get install python-software-properties`
#. `add-apt-repository ppa:swift-core/release`
#. `apt-get update` #. `apt-get update`
#. `apt-get install curl gcc git-core memcached python-coverage python-dev #. `apt-get install curl gcc memcached rsync sqlite3 xfsprogs git-core python-setuptools`
python-nose python-setuptools python-simplejson python-xattr sqlite3 #. `apt-get install python-coverage python-dev python-nose python-simplejson
xfsprogs python-eventlet python-greenlet python-pastedeploy python-xattr python-eventlet python-greenlet python-pastedeploy
python-netifaces python-pip` python-netifaces python-pip python-dnspython python-mock`
#. `pip install mock`
#. `pip install dnspython`
#. Install anything else you want, like screen, ssh, vim, etc.
* On Fedora, * On yum based systems,
#. `yum install xinetd rsync` #. `yum install curl gcc memcached rsync sqlite xfsprogs git-core xinetd python-setuptools`
#. `yum install memcached` #. `yum install python-coverage python-devel python-nose python-simplejson
#. `yum install python-netifaces python-nose python-mock python-dns` python-xattr python-eventlet python-greenlet python-pastedeploy
#. `yum install git gcc` python-netifaces python-pip python-dnspython python-mock`
This installs all necessary dependencies, and you could use your guest user This installs necessary system dependencies; and *most* of the python
or root for starting process. But you must first config it correctly dependencies. Later in the process setuptools/distribute or pip will
(`user = <your-user-name>`) in object-server.conf or proxy.conf, and then use install and/or upgrade some other stuff - it's getting harder to avoid.
it to be used in every place where this manual calls for You can also install anything else you want, like screen, ssh, vim, etc.
`<your-user-name>:<your-group-name>`.
Ensure that you are installing the version of Swift that corresponds to
this document. If not, enable the correct update repositories.
Next, choose either :ref:`partition-section` or :ref:`loopback-section`. Next, choose either :ref:`partition-section` or :ref:`loopback-section`.
@ -114,8 +118,8 @@ Setting up rsync
#. Create /etc/rsyncd.conf:: #. Create /etc/rsyncd.conf::
uid = <Your user name> uid = <your-user-name>
gid = <Your group name> gid = <your-group-name>
log file = /var/log/rsyncd.log log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid pid file = /var/run/rsyncd.pid
address = 127.0.0.1 address = 127.0.0.1
@ -144,7 +148,6 @@ Setting up rsync
read only = false read only = false
lock file = /var/lock/account6042.lock lock file = /var/lock/account6042.lock
[container6011] [container6011]
max connections = 25 max connections = 25
path = /srv/1/node/ path = /srv/1/node/
@ -201,19 +204,29 @@ Setting up rsync
disable = no disable = no
#. On Ubuntu `service rsync restart` #. On Ubuntu, run `service rsync restart`, on xinetd based systems
run `service xinetd restart`.
#. Verify rsync is accepting connections for all servers::
rsync rsync://pub@localhost/
------------------ ------------------
Starting memcached Starting memcached
------------------ ------------------
On Fedora, make sure that memcached runs, running this if necessary: On non-Ubuntu distros you need to ensure memcached is running:
* `service memcached start`
* `chkconfig memcached on`
or:
* `systemctl enable memcached.service` * `systemctl enable memcached.service`
* `systemctl start memcached.service` * `systemctl start memcached.service`
If this is not done, tokens of tempauth expire immediately and accessing The tempauth middleware stores tokens in memcached. If memcached is not
Swift with curl becomes impossible. running, tokens cannot be validated, and accessing Swift becomes impossible.
--------------------------------------------------- ---------------------------------------------------
Optional: Setting up rsyslog for individual logging Optional: Setting up rsyslog for individual logging
@ -257,27 +270,25 @@ Optional: Setting up rsyslog for individual logging
#. `chmod -R g+w /var/log/swift` #. `chmod -R g+w /var/log/swift`
#. `service rsyslog restart` #. `service rsyslog restart`
------------------------------------------------ ----------------
Getting the code and setting up test environment Getting the code
------------------------------------------------ ----------------
Sample configuration files are provided with all defaults in line-by-line comments. You can do the following commands as administrator user.
Do these commands as you on guest. #. Check out the python-swiftclient repo
`git clone https://github.com/openstack/python-swiftclient.git`
#. Build a development installation of python-swiftclient
`cd ~/python-swiftclient; sudo python setup.py develop; cd -`
#. Check out the swift repo
`git clone https://github.com/openstack/swift.git`
#. Build a development installation of swift
`cd ~/swift; sudo python setup.py develop; cd -`
#. Install swift's test dependencies
`sudo pip install -r swift/test-requirements.txt`
#. `mkdir ~/bin` Do the following commands as root, but verify that Swift has access
#. Check out the swift repo with `git clone https://github.com/openstack/swift.git` to resulting configuration files.
#. Build a development installation of swift, for example:
`cd ~/swift; sudo python setup.py develop`
#. Check out the python-swiftclient repo with `git clone https://github.com/openstack/python-swiftclient.git`
#. Build a development installation of python-swiftclient, for example:
`cd ~/python-swiftclient; sudo python setup.py develop`
#. Edit `~/.bashrc` and add to the end::
export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf
export PATH=${PATH}:~/bin
#. `. ~/.bashrc`
--------------------- ---------------------
Configuring each node Configuring each node
@ -669,10 +680,16 @@ Sample configuration files are provided with all defaults in line-by-line commen
[object-auditor] [object-auditor]
#. Update <your-user-name>::
find /etc/swift/ -name \*.conf | xargs sed -i "s/<your-user-name>/${USER}/"
------------------------------------ ------------------------------------
Setting up scripts for running Swift Setting up scripts for running Swift
------------------------------------ ------------------------------------
#. `mkdir ~/bin`
#. Create `~/bin/resetswift.` #. Create `~/bin/resetswift.`
If you are using a loopback device substitute `/dev/sdb1` with `/srv/swift-disk`. If you are using a loopback device substitute `/dev/sdb1` with `/srv/swift-disk`.
@ -703,22 +720,22 @@ Setting up scripts for running Swift
rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz
swift-ring-builder object.builder create 18 3 1 swift-ring-builder object.builder create 18 3 1
swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1 swift-ring-builder object.builder add r1z1-127.0.0.1:6010/sdb1 1
swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1 swift-ring-builder object.builder add r1z2-127.0.0.1:6020/sdb2 1
swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1 swift-ring-builder object.builder add r1z3-127.0.0.1:6030/sdb3 1
swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1 swift-ring-builder object.builder add r1z4-127.0.0.1:6040/sdb4 1
swift-ring-builder object.builder rebalance swift-ring-builder object.builder rebalance
swift-ring-builder container.builder create 18 3 1 swift-ring-builder container.builder create 18 3 1
swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1 swift-ring-builder container.builder add r1z1-127.0.0.1:6011/sdb1 1
swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1 swift-ring-builder container.builder add r1z2-127.0.0.1:6021/sdb2 1
swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1 swift-ring-builder container.builder add r1z3-127.0.0.1:6031/sdb3 1
swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1 swift-ring-builder container.builder add r1z4-127.0.0.1:6041/sdb4 1
swift-ring-builder container.builder rebalance swift-ring-builder container.builder rebalance
swift-ring-builder account.builder create 18 3 1 swift-ring-builder account.builder create 18 3 1
swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1 swift-ring-builder account.builder add r1z1-127.0.0.1:6012/sdb1 1
swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1 swift-ring-builder account.builder add r1z2-127.0.0.1:6022/sdb2 1
swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1 swift-ring-builder account.builder add r1z3-127.0.0.1:6032/sdb3 1
swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1 swift-ring-builder account.builder add r1z4-127.0.0.1:6042/sdb4 1
swift-ring-builder account.builder rebalance swift-ring-builder account.builder rebalance
#. Create `~/bin/startmain`:: #. Create `~/bin/startmain`::
@ -734,16 +751,23 @@ Setting up scripts for running Swift
swift-init rest start swift-init rest start
#. `chmod +x ~/bin/*` #. `chmod +x ~/bin/*`
#. Edit `~/.bashrc` and add to the end::
export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf
export PATH=${PATH}:~/bin
#. `. ~/.bashrc`
#. `remakerings` #. `remakerings`
#. `cp ~/swift/test/sample.conf /etc/swift/test.conf` #. `cp ~/swift/test/sample.conf /etc/swift/test.conf`
#. `cd ~/swift; ./.unittests` #. `~/swift/.unittests`
#. `startmain` (The ``Unable to increase file descriptor limit. Running as non-root?`` warnings are expected and ok.) #. `startmain` (The ``Unable to increase file descriptor limit. Running as non-root?`` warnings are expected and ok.)
#. Get an `X-Storage-Url` and `X-Auth-Token`: ``curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0`` #. Get an `X-Storage-Url` and `X-Auth-Token`: ``curl -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0``
#. Check that you can GET account: ``curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>`` #. Check that you can GET account: ``curl -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>``
#. Check that `swift` works: `swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat` #. Check that `swift` works: `swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat`
#. `cd ~/swift; ./.functests` (Note: functional tests will first delete #. `~/swift/.functests` (Note: functional tests will first delete
everything in the configured accounts.) everything in the configured accounts.)
#. `cd ~/swift; ./.probetests` (Note: probe tests will reset your #. `~/swift/.probetests` (Note: probe tests will reset your
environment as they call `resetswift` for each test.) environment as they call `resetswift` for each test.)
---------------- ----------------