diff --git a/doc/source/howto_installmultinode.rst b/doc/source/howto_installmultinode.rst index bec4f9510b..a314474993 100644 --- a/doc/source/howto_installmultinode.rst +++ b/doc/source/howto_installmultinode.rst @@ -56,6 +56,9 @@ Network Setup Notes This document refers to two networks. An external network for connecting to the Proxy server, and a storage network that is not accessibile from outside the cluster, to which all of the nodes are connected. All of the Swift services, as well as the rsync daemon on the Storage nodes are configured to listen on their STORAGE_LOCAL_NET IP addresses. +.. note:: + Run all commands as the root user + General OS configuration and partitioning for each node ------------------------------------------------------- @@ -73,16 +76,27 @@ General OS configuration and partitioning for each node mkdir -p /etc/swift chown -R swift:swift /etc/swift/ -#. Create /etc/swift/swift.conf:: +#. On the first node only, create /etc/swift/swift.conf:: + cat >/etc/swift/swift.conf < +#. Modify memcached to listen on the default interfaces. Preferably this should be on a local, non-public network. Edit the IP address in /etc/memcached.conf, for example:: + + perl -pi -e "s/-l 127.0.0.1/-l $PROXY_LOCAL_NET_IP/" /etc/memcached.conf #. Restart the memcached server:: @@ -113,12 +129,16 @@ Configure the Proxy node #. Create /etc/swift/proxy-server.conf:: + cat >/etc/swift/proxy-server.conf <:8080/v1 + default_swift_cluster = local#https://$PROXY_LOCAL_NET_IP:8080/v1 # Highly recommended to change this key to something else! super_admin_key = swauthkey @@ -148,6 +168,7 @@ Configure the Proxy node [filter:cache] use = egg:swift#memcache memcache_servers = :11211 + EOF .. note:: @@ -166,11 +187,15 @@ Configure the Proxy node For more information on building rings, see :doc:`overview_ring`. -#. For every storage device on each node add entries to each ring:: +#. For every storage device in /srv/node on each node add entries to each ring:: - swift-ring-builder account.builder add z-:6002/ 100 - swift-ring-builder container.builder add z-:6001/ 100 - swift-ring-builder object.builder add z-:6000/ 100 + export ZONE= # set the zone number for that storage device + export STORAGE_LOCAL_NET_IP= # and the IP address + export WEIGHT=100 # relative weight (higher for bigger/faster disks) + export DEVICE=sdb1 + swift-ring-builder account.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6002/$DEVICE $WEIGHT + swift-ring-builder container.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6001/$DEVICE $WEIGHT + swift-ring-builder object.builder add z$ZONE-$STORAGE_LOCAL_NET_IP:6000/$DEVICE $WEIGHT .. note:: Assuming there are 5 zones with 1 node per zone, ZONE should start at @@ -217,6 +242,7 @@ Configure the Auth node #. Create /etc/swift/auth-server.conf:: + cat >/etc/swift/auth-server.conf <:8080/v1 # Highly recommended to change this key to something else! super_admin_key = devauth - + EOF + #. Start Auth services:: swift-init auth start @@ -242,10 +269,11 @@ Configure the Storage nodes .. note:: Swift *should* work on any modern filesystem that supports - Extended Attributes (XATTRS). We currently recommend XFS as it + Extended Attributes (XATTRS). We currently recommend XFS as it demonstrated the best overall performance for the swift use case after - considerable testing and benchmarking at Rackspace. It is also the - only filesystem that has been thoroughly tested. + considerable testing and benchmarking at Rackspace. It is also the + only filesystem that has been thoroughly tested. These instructions + assume that you are going to devote /dev/sdb1 to an XFS filesystem. #. Install Storage node packages:: @@ -263,11 +291,12 @@ Configure the Storage nodes #. Create /etc/rsyncd.conf:: + cat >/etc/rsyncd.conf < + address = $STORAGE_LOCAL_NET_IP [account] max connections = 2 @@ -286,10 +315,11 @@ Configure the Storage nodes path = /srv/node/ read only = false lock file = /var/lock/object.lock + EOF -#. Edit the following line in /etc/default/rsync:: +#. Edit the RSYNC_ENABLE= line in /etc/default/rsync:: - RSYNC_ENABLE=true + perl -pi -e 's/RSYNC_ENABLE=false/RSYNC_ENABLE=true/' /etc/default/rsync #. Start rsync daemon:: @@ -301,8 +331,9 @@ Configure the Storage nodes #. Create /etc/swift/account-server.conf:: + cat >/etc/swift/account-server.conf < + bind_ip = $STORAGE_LOCAL_NET_IP workers = 2 [pipeline:main] @@ -316,9 +347,11 @@ Configure the Storage nodes [account-auditor] [account-reaper] + EOF #. Create /etc/swift/container-server.conf:: + cat >/etc/swift/container-server.conf < workers = 2 @@ -334,9 +367,11 @@ Configure the Storage nodes [container-updater] [container-auditor] + EOF #. Create /etc/swift/object-server.conf:: + cat >/etc/swift/object-server.conf < workers = 2 @@ -352,20 +387,33 @@ Configure the Storage nodes [object-updater] [object-auditor] + EOF -#. Start the storage services:: +#. Start the storage services. If you use this command, it will try to start every +service for which a configuration file exists, and throw a warning for any +configuration files which don't exist:: - swift-init object-server start - swift-init object-replicator start - swift-init object-updater start - swift-init object-auditor start - swift-init container-server start - swift-init container-replicator start - swift-init container-updater start - swift-init container-auditor start - swift-init account-server start - swift-init account-replicator start - swift-init account-auditor start + swift-init all start + +Or, if you want to start them one at a time, run them as below. Note that if the +server program in question generates any output on its stdout or stderr, swift-init +has already redirected the command's output to /dev/null. If you encounter any +difficulty, stop the server and run it by hand from the command line. Any server +may be started using "swift-$SERVER-$SERVICE /etc/swift/$SERVER-config", where +$SERVER might be object, continer, or account, and $SERVICE might be server, +replicator, updater, or auditor.:: + + swift-init object-server start + swift-init object-replicator start + swift-init object-updater start + swift-init object-auditor start + swift-init container-server start + swift-init container-replicator start + swift-init container-updater start + swift-init container-auditor start + swift-init account-server start + swift-init account-replicator start + swift-init account-auditor start Create Swift admin account and test ----------------------------------- @@ -395,7 +443,7 @@ You run these commands from the Auth node. curl -k -v -H 'X-Auth-Token: ' -#. Check that ``st`` works:: +#. Check that ``st`` works (at this point, expect zero containers, zero objects, and zero bytes):: st -A https://:11000/v1.0 -U system:root -K testpass stat @@ -408,6 +456,22 @@ You run these commands from the Auth node. st -A https://:11000/v1.0 -U system:root -K testpass download myfiles +#. Use ``st`` to save a backup of your builder files to a container named 'builders'. Very important not to lose your builders!:: + + st -A https://:11000/v1.0 -U system:root -K testpass upload builders /etc/swift/*.builder + +#. Use ``st`` to list your containers:: + + st -A https://:11000/v1.0 -U system:root -K testpass list + +#. Use ``st`` to list the contents of your 'builders' container:: + + st -A https://:11000/v1.0 -U system:root -K testpass list builders + +#. Use ``st`` to download all files from the 'builders' container:: + + st -A https://:11000/v1.0 -U system:root -K testpass download builders + .. _add-proxy-server: Adding a Proxy Server