openstack/devstack
Code Issues Proposed changes

Change the restart_rpc_backend loop to accomodate async rabbitmq

Browse Source 
Some distros have converted to systemd for starting RabbitMQ.  This has
resulted in:
---
[Call Trace]
./stack.sh:904:restart_rpc_backend
/home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201:die
[ERROR] /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201 Failed to set rabbitmq password
Error on exit
World dumping... see /opt/stack/logs/worlddump-2015-05-29-031618.txt for details
---

Because 'restart_service rabbitmq-server' returns before the server is ready to
accept connections.

Alter the retry loop to only restart the rabbitmq-server every second time
through the loop.  Allowing time for the slow rabbit to start.

Closes-Bug: 1449056
Change-Id: Ibb291c1ecfd109f9ed10b5f194933364985cc1ce
This commit is contained in:
Tony Breeds 2015-05-15 12:51:43 +10:00
parent 493d3b3aff
commit 6bc905c348
1 changed files with 12 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
lib/rpc_backend
View File

@ -194,13 +194,22 @@ function restart_rpc_backend {
# NOTE(bnemec): Retry initial rabbitmq configuration to deal with # NOTE(bnemec): Retry initial rabbitmq configuration to deal with
# the fact that sometimes it fails to start properly. # the fact that sometimes it fails to start properly.
# Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144100 # Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1144100
# NOTE(tonyb): Extend the orginal retry logic to only restart rabbitmq
# every second time around the loop.
# See: https://bugs.launchpad.net/devstack/+bug/1449056 for details on
# why this is needed. This can bee seen on vivid and Debian unstable
# (May 2015)
# TODO(tonyb): Remove this when Debian and Ubuntu have a fixed systemd
# service file.
local i local i
for i in `seq 10`; do for i in `seq 20`; do
local rc=0 local rc=0
[[ $i -eq "10" ]] && die $LINENO "Failed to set rabbitmq password" [[ $i -eq "20" ]] && die $LINENO "Failed to set rabbitmq password"
if [[ $(( i % 2 )) == "0" ]] ; then
restart_service rabbitmq-server restart_service rabbitmq-server
fi
rabbit_setuser "$RABBIT_USERID" "$RABBIT_PASSWORD" || rc=$? rabbit_setuser "$RABBIT_USERID" "$RABBIT_PASSWORD" || rc=$?
if [ $rc -ne 0 ]; then if [ $rc -ne 0 ]; then