Add DB2 full online backups for DB2
Currently Trove only supports full offline backups for DB2. This spec details how to configure DB2 for full online backups & restore. Implements blueprint: db2-online-backup Change-Id: I260133431d12ea1f4aea63e49bfd408cf2454431
This commit is contained in:
Mariam John
mariam john
parent
3ad634780d
commit
7d1f3617d5
267
specs/newton/db2-online-backup.rst
Normal file
267
specs/newton/db2-online-backup.rst
Normal file
@ -0,0 +1,267 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
Sections of this template were taken directly from the Nova spec
|
||||
template at:
|
||||
https://github.com/openstack/nova-specs/blob/master/specs/juno-template.rst
|
||||
|
||||
..
|
||||
This template should be in ReSTructured text. The filename in the git
|
||||
repository should match the launchpad URL, for example a URL of
|
||||
https://blueprints.launchpad.net/trove/+spec/awesome-thing should be named
|
||||
awesome-thing.rst.
|
||||
|
||||
Please do not delete any of the sections in this template. If you
|
||||
have nothing to say for a whole section, just write: None
|
||||
|
||||
Note: This comment may be removed if desired, however the license notice
|
||||
above should remain.
|
||||
|
||||
|
||||
====================================
|
||||
DB2 Full Online Backup and Restore
|
||||
====================================
|
||||
|
||||
Currently in Trove, we support full offline backups for DB2 which is the
|
||||
default backup mechanism for DB2. This enables users to take full backups of
|
||||
DB2 databases when no applications are connected to or using these databases.
|
||||
While useful for cases where this downtime is okay, it is not very useful in
|
||||
a production type environment were applications are expected to be online all
|
||||
the time. DB2 provides a way to do online backups and this spec outlines how
|
||||
we can implement a full online backup mechanism in Trove.
|
||||
|
||||
Launchpad Blueprint:
|
||||
https://blueprints.launchpad.net/trove/+spec/db2-online-backup
|
||||
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
|
||||
There are two types of backups we can perform using DB2 - offline and online.
|
||||
Offline backups require users and applications to be disconnected from the
|
||||
databases thereby requiring a certain down time. Online backups on the other
|
||||
hand, can be performed while users are still connected to databases. Trove
|
||||
currently supports full offline backups for DB2. In this spec, we explain
|
||||
how to enable full online backups for DB2 in Trove.
|
||||
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
||||
As mentioned above, the difference between offline and online backup is that
|
||||
in the former case all applications must disconnect from the database while
|
||||
in the latter, applications can still be connected to databases while backup
|
||||
is being taken. In order to enable online backups, archive logging needs to
|
||||
be enabled. This allows applications to use the database even while backups
|
||||
are being taken. Any updates made to the database is recorded in the logs.
|
||||
|
||||
Logging is a mechanism where every updates to a database is recorded to
|
||||
transaction logs (also called active logs) as the data is being modified.
|
||||
There are 2 types of logging mechanism - circular and archive logging. By
|
||||
default, circular logging is enabled which means that after the last active
|
||||
log is filled, the first active log is overwritten. With circular logging
|
||||
as the default logging mechanism, only offline backups can be taken.
|
||||
|
||||
Using archive logging, the logs are stored in sequence and as logs get full,
|
||||
they are archived. Archive logging allows users to restore databases by roll
|
||||
forwarding to a particular point in time or a point before failure. Hence,
|
||||
users can restore a database from a backup image and roll forward to a
|
||||
particular point using logs thus getting it to a consistent state. For an
|
||||
introduction to archive logging, please refer [1] and for an overview of how
|
||||
transactional logging works in DB2, please refer [2].
|
||||
|
||||
To enable full online backup for Trove, the following steps need to be taken:
|
||||
|
||||
1. Create a directory to store the backups and archive logs -
|
||||
/home/db2inst1/db2inst1/backup/ArchiveLogs
|
||||
2. When a database is created, if online backups is configured for DB2(through
|
||||
the backup_strategy config property), then configure the following database
|
||||
configuration parameter: 'LOGARCHMETH1' to point to a location where the
|
||||
archive logs can be stored using the command:
|
||||
|
||||
db2 update database configuration for <db> using LOGARCHMETH1
|
||||
'DISK:/home/db2inst1/backup/ArchiveLogs'
|
||||
|
||||
Setting this database configuration parameter enables archive logging and
|
||||
stores the archive logs in a location different from the primary/active
|
||||
logs.Once this change is made, a full offline backup needs to be taken
|
||||
before any connections can be made to the database (this is because the
|
||||
database goes into a 'BACKUP_PENDING' state when this configuration
|
||||
change is made).
|
||||
|
||||
db2 backup database <db> to /home/db2inst1/db2inst1/backup
|
||||
|
||||
This backup can be deleted from the backup directory after this operation.
|
||||
3. The online backup strategy for DB2 will comprise of the following actions:
|
||||
- Check if archive logging is enabled for each database
|
||||
- Check if there is enough space to store backups (which will include
|
||||
database and archived logs). The get_dbsize_info function that DB2
|
||||
provides can be used to get the size of the database directory. We can
|
||||
query the SYSIBMADM.LOG_UTILIZATION table to get the log space used for
|
||||
each database.
|
||||
- Do an online backup of the database and include the archive logs along
|
||||
with the backup image itself:
|
||||
|
||||
db2 backup database <db> ONLINE to
|
||||
/home/db2inst1/db2inst1/backup INCLUDE LOGS
|
||||
|
||||
4. Once the backup is complete, compress/encrypt the backup inside the volume
|
||||
storage.
|
||||
5. The compressed/encrypted file can then be stored in Swift under the
|
||||
database_backups container.
|
||||
6. Once the backups are stored in Swift, we can go ahead and delete the backup
|
||||
files and archive logs from the backup directory.
|
||||
|
||||
Restoring a database includes restoring from an online backup image and
|
||||
rolling forward the logs to the end of the backup to make sure the database is
|
||||
in a consistent state.
|
||||
|
||||
1. Create a directory where the backup files and archive logs can be stored
|
||||
2. Retrieve the backup from storage to the proper location
|
||||
3. Use the following commands to restore the database and ensure it is in a
|
||||
consistent state:
|
||||
- db2 RESTORE DATABASE <db> FROM <backup_dir> LOGTARGET <log_dir>
|
||||
|
||||
The above command restores the database from the specified online backup
|
||||
directory and extracts the archive log files into the log_dir
|
||||
|
||||
- db2 ROLLFORWARD DATABASE <db> TO END OF BACKLOG AND COMPLETE
|
||||
|
||||
This command will roll forward the restored database to a consistent state
|
||||
by applying the archive logs. To see more details on the ROLLFORWARD
|
||||
command, please refer [3].
|
||||
|
||||
Configuration
|
||||
-------------
|
||||
|
||||
The default configuration values of the following will need to change to
|
||||
correspond to the respective DB2 locations.
|
||||
|
||||
* backup_strategy
|
||||
* backup_namespace
|
||||
* restore_namespace
|
||||
|
||||
Database
|
||||
--------
|
||||
|
||||
None
|
||||
|
||||
Public API
|
||||
----------
|
||||
|
||||
None
|
||||
|
||||
Public API Security
|
||||
-------------------
|
||||
|
||||
None
|
||||
|
||||
Python API
|
||||
----------
|
||||
|
||||
None
|
||||
|
||||
CLI (python-troveclient)
|
||||
------------------------
|
||||
|
||||
None
|
||||
|
||||
Internal API
|
||||
------------
|
||||
|
||||
None
|
||||
|
||||
Guest Agent
|
||||
-----------
|
||||
|
||||
The DB2 guestagent will be modified to add support for full online backup
|
||||
and restore functionality. In addition, new strategies will also be added
|
||||
to support these new features.
|
||||
|
||||
The following guestagent files will be modified:
|
||||
- trove/guestagent/datastore/experimental/db2/service.py
|
||||
- trove/guestagent/datastore/experimental/db2/system.py
|
||||
|
||||
and the following new strategy files will be modified to add a new class
|
||||
for online backups:
|
||||
|
||||
- trove/guestagent/strategies/backup/experimental/db2_impl.py
|
||||
- trove/guestagent/strategies/restore/experimental/db2_impl.py
|
||||
|
||||
New classes will be added for online backup strategy and the existing
|
||||
class needs to be renamed from DB2Backup -> DB2OfflineBackup
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
None
|
||||
|
||||
|
||||
Dashboard Impact (UX)
|
||||
=====================
|
||||
|
||||
None
|
||||
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Mariam John (johnma)
|
||||
|
||||
|
||||
Milestones
|
||||
----------
|
||||
|
||||
Newton
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
There will only be one work item for this feature. This includes implementing
|
||||
the strategies for DB2 online backup and restore and make the necessary
|
||||
guestagent changes. This will also include test-cases necessary to test the
|
||||
new functionalities.
|
||||
|
||||
Upgrade Implications
|
||||
====================
|
||||
|
||||
None
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
* Add new test cases to test online backup and restore functionality for DB2:
|
||||
|
||||
- Add new unit tests to test the newly implemented functionality
|
||||
- Add a new DB2 helper class to the existing integration-test framework.
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
The datastore documentation should be updated to reflect the enabled features.
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
.. [1] https://www.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.admin.ha.doc/doc/c0051344.html
|
||||
|
||||
.. [2] http://www.ibm.com/developerworks/data/library/techarticle/0301kline/0301kline.html
|
||||
|
||||
.. [3] https://www.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.admin.cmd.doc/doc/r0001978.html
|
||||
|
||||
|
||||
Appendix
|
||||
========
|
||||
|
||||
None
|
||||
Loading…
Reference in New Issue
Block a user