Fix mariadb_recovery when mariadb container is missing

Mariadb recovery fails if a cluster has previously been deployed, but any of
the mariadb containers do not exist.

Steps to reproduce
==================

* Deploy a mariadb galera cluster
* Remove the mariadb container from at least one host (docker rm -f mariadb)
* Run kolla-ansible mariadb_recovery

Expected results
================

The cluster is recovered, and a new container deployed where necessary.

Actual results
==============

The task 'Stop MariaDB containers' fails on any host where the container does
not exist.

Solution
========

This change fixes the issue by using the 'ignore_missing' flag for kolla_docker
with the stop_container action. This means the task does not fail when the
container does not exist. It is also necessary to swap some 'docker cp'
commands for 'cp' on the host, using the path to the volume.

Closes-Bug: #1907658

Change-Id: Ibd4a6adeb8443e12c45cbab65f501392ffb16fc7

ansible/roles/mariadb/tasks/recover_cluster.yml

@@ -26,6 +26,7 @@
       kolla_docker:
         name: "{{ mariadb_service.container_name }}"
         action: "stop_container"
+        ignore_missing: true
 
     # Run wsrep recovery with detach=false to block until completion. Use a
     # different container name to avoid the mariadb container being removed.
@@ -47,7 +48,7 @@
 
     - name: Copying MariaDB log file to /tmp
       become: true
-      command: "docker cp {{ mariadb_service.container_name }}:/var/log/kolla/mariadb/mariadb.log /tmp/mariadb_tmp.log"
+      command: "cp {{ docker_runtime_directory or '/var/lib/docker' }}/volumes/kolla_logs/_data/mariadb/mariadb.log /tmp/mariadb_tmp.log"
 
     # Look for sequence number in logs. Format is:
     # WSREP: Recovered position: <UUID>:<seqno>.
@@ -100,18 +101,10 @@
   set_fact:
     bootstrap_host: "{{ mariadb_recover_inventory_name }}"
 
-- name: Copying grastate.dat file from MariaDB container in bootstrap host
-  become: true
-  command: "docker cp {{ mariadb_service.container_name }}:/var/lib/mysql/grastate.dat /tmp/kolla_mariadb_grastate.dat"
-  changed_when: false
-  when:
-    - bootstrap_host is defined
-    - bootstrap_host == inventory_hostname
-
 - name: Set grastate.dat file from MariaDB container in bootstrap host
   become: true
   lineinfile:
-    dest: /tmp/kolla_mariadb_grastate.dat
+    dest: "{{ docker_runtime_directory or '/var/lib/docker' }}/volumes/mariadb/_data/grastate.dat"
     regexp: 'safe_to_bootstrap:(.*)$'
     line: 'safe_to_bootstrap: 1'
     state: present
@@ -119,14 +112,6 @@
     - bootstrap_host is defined
     - bootstrap_host == inventory_hostname
 
-- name: Copying grastate.dat file to mariadb container
-  become: true
-  command: docker cp /tmp/kolla_mariadb_grastate.dat {{ mariadb_service.container_name }}:/var/lib/mysql/grastate.dat
-  changed_when: false
-  when:
-    - bootstrap_host is defined
-    - bootstrap_host == inventory_hostname
-
 - name: Starting first MariaDB container
   become: true
   kolla_docker:

releasenotes/notes/bug-1907658-a24ddc45f63893b5.yaml

@@ -0,0 +1,6 @@
+---
+fixes:
+  - |
+    Fixes an issue with ``kolla-ansible mariadb_recovery`` when the ``mariadb``
+    container does not exist on one or more hosts. `LP#1907658
+    <https://bugs.launchpad.net/kolla-ansible/+bug/1907658>`__