openstack/devstack
Code Issues Proposed changes

Fix async race updating nova configs

Browse Source 
The configure_neutron_nova function updates nova configs. While that is
still running we separately update nova configs in stack.sh. This can
result in unexpected configs (that don't work). Fix this by waiting for
configure_neutron_nova to complete its work before we do nova config
updates directly in stack.sh.

For specifics we say that:

  [neutron]
  project_domain_name = Default

was missing from both nova.conf and nova-cpu.conf and instances could
not be created because keystone complained about not finding domain in
project. The strong suspicion here is that on some systems
configure_neutron_nova would write out project_domain_name while the
stack.sh inisets were running resulting in stack.sh overwriting the
project_domain_name content.

One theory is that disabling swift makes this problem more likely as
there is swift work in the middle of the async period. This is supported
by the fact that our job that hits this problem does indeed disable
swift.

Change-Id: I0961d882d555a21233c6b4fbfc077cfe33b88499
This commit is contained in:
Clark Boylan 2021-04-29 11:46:35 -07:00
parent bab78dede2
commit 06b7352478
1 changed files with 9 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
stack.sh
View File

@ -1238,17 +1238,21 @@ fi
# deployments. This ensures the keys match across nova and cinder across all
# hosts.
FIXED_KEY=${FIXED_KEY:-bae3516cc1c0eb18b05440eba8012a4a880a2ee04d584a9c1579445e675b12defdc716ec}
if is_service_enabled nova; then
iniset $NOVA_CONF key_manager fixed_key "$FIXED_KEY"
iniset $NOVA_CPU_CONF key_manager fixed_key "$FIXED_KEY"
fi
if is_service_enabled cinder; then
iniset $CINDER_CONF key_manager fixed_key "$FIXED_KEY"
fi
async_wait configure_neutron_nova
# NOTE(clarkb): This must come after async_wait configure_neutron_nova because
# configure_neutron_nova modifies $NOVA_CONF and $NOVA_CPU_CONF as well. If
# we don't wait then these two ini updates race either other and can result
# in unexpected configs.
if is_service_enabled nova; then
iniset $NOVA_CONF key_manager fixed_key "$FIXED_KEY"
iniset $NOVA_CPU_CONF key_manager fixed_key "$FIXED_KEY"
fi
# Launch the nova-api and wait for it to answer before continuing
if is_service_enabled n-api; then
echo_summary "Starting Nova API"