Work around OVN DB leader election race condition

After OVN DB leader restarts there is a period before a new leader has
been elected where the old leader is returned in the cluster status.
This can result in a failure to apply the connection settings if a
different leader is elected.

Wait for a few seconds for the leader election to complete.

Change-Id: I20f08c986fa6b4b3ec668dad649e69f23119796b
Closes-Bug: #2059124

ansible/roles/ovn-db/defaults/main.yml

@@ -94,3 +94,5 @@ ovn_sb_command: >-
   --db-sb-pidfile=/run/ovn/ovnsb_db.pid
   --db-sb-file=/var/lib/openvswitch/ovn-sb/ovnsb.db
   --ovn-sb-logfile=/var/log/kolla/openvswitch/ovn-sb-db.log
+# Workaround: pause after restarting containers to allow for leader election.
+ovn_leader_election_pause: 5

ansible/roles/ovn-db/tasks/bootstrap-db.yml

@@ -1,4 +1,14 @@
 ---
+# NOTE(mgoddard): After OVN DB leader restarts there is a period before a new
+# leader has been elected where the old leader is returned in the cluster
+# status. This can result in a failure to apply the connection settings if a
+# different leader is elected.  Wait for a few seconds for the leader election
+# to complete.
+- name: Wait for leader election
+  pause:
+    seconds: "{{ ovn_leader_election_pause }}"
+  when: ovn_nb_db_cluster_exists | default(false) or ovn_sb_db_cluster_exists | default(false)
+
 - name: Get OVN_Northbound cluster leader
   become: true
   command: "{{ kolla_container_engine }} exec ovn_nb_db ovs-appctl -t /var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound"

releasenotes/notes/workaround-ovn-leader-election-f7f75e93e9300d96.yaml

@@ -0,0 +1,6 @@
+---
+fixes:
+  - |
+    Fixes an issue where OVN northbound or southbound database deployment could
+    fail when a new leader is elected. `LP#2059124
+    <https://bugs.launchpad.net/kolla-ansible/+bug/2059124>`__