The main goal of this PS is to make sure the migrator can complete the migrations even if mariadb galera cluster dropped the migrator connection leaving the database in inconsistent state. It may happen that migration_log has a record of a successfully performed migration while the database scheme misses an entity so any further attempts to re-run the migrator fail because of missed entity the migrator expects to be present.
Also the migrator is running mariadb image as a main one and grafana binaries are mounted as /usr/share/grafana. Migrator job container is running under nobody user uid.
This PS runs migrator in a safe way:
- prepares database backup
- runs a single instance of grafana as migrator with log file as a background process in a loop
- constantly checks the log file in the main process
- in case of the migrations completed it stops grafana-server process and completed the job
- in case of a migration error it restores the previously prepared backup so the grafana-server that is running in a background loop can re-try the migration
- the database operations are prefixed with code that makes sure the database is reachable.
Change-Id: I4e1542b62777f25c08ddd2cb74f0a0e7bfea5145
00846e2e02