[#TREP-278] Implement pluggable backup and restore operations - continuent.org JIRA

Issue Details (XML | Word)

Key:	TREP-278
Type:	New Feature
Status:	Resolved
Resolution:	Fixed
Priority:	Critical
Assignee:	Robert Hodges
Reporter:	Robert Hodges
Votes:	0
Watchers:	0

Operations

If you were logged in you would be able to see more operations.

Tungsten Replicator

Implement pluggable backup and restore operations

Created: 28/Apr/09 10:24 PM Updated: 23/Jul/09 05:40 PM

Component/s:

Core Framework

Affects Version/s:

None

Fix Version/s:

Tungsten Replicator 1.0.3

Original Estimate:

Unknown

Remaining Estimate:

Unknown

Time Spent:

Unknown

Environment:

All - mechanism must support multiple database mechanisms

Issue Links:

Dependants

This issue depends on:
	TREP-272 Expose underlying database URL and Dr...

Description

The Tungsten Replicator needs to support a pluggable backup mechanism that offer backup and restore operations as simple commands. The idea is that backing up a replicated server should be as simple as saying:

trepctl backup

and restoring the backup should be as simple as:

trepctl restore

Here's a more detailed list of requirements.

1.) It should be possible to backup with a single command.
2.) It should be possible to restore with a single command. The restore command by default will take the latest backup, but there should be an option to select an earlier backup.
3.) The backup mechanism should be pluggable and permit different backup types.
4.) It should be possible to configure the number of backups to retain. Once this number is exceeded the oldest backups are deleted automatically.
5.) It should be possible to list the currently available backups.

Finally, here's a full use case for S3 backups that describes how the implemented backup mechanism might work for the replicator working on MySQL.

1.) At installation time, the user selects the MySQL backup plugin and configures a default S3 bucket to store backup files as well as security information required to post backups. The user configures the backup type which could be either mysqldump or an LVM snapshot. In this case the user selects mysqldump.

2.) To backup the master, the user enters the following commands:

trepctl offline
trepctl backup
trepctl online

This backs up the entire server contents including the THL and stores them in S3. Since this is a master the backup process must ensure that databases are locked e.g., using FLUSH TABLES WITH READ LOCK to secure a transactionally consistent dump. The backup S3 name that allows it to sort from oldest to newest when listing backups. This name also encodes enough information to recognize the backup type.

3.) To restore the backup on a slave the user first configures the slave backup to use the same S3 repository as the master. He/she then issues the following commands:

trepctl offline
trepctl restore
trepctl online

The restore command automatically pulls the latest backup from the repository and based on the backup name selects the mysqldump plugin, which pipes the backup file through mysql to restore the data.

All

Comments

Work Log

Change History

Sort Order:

Comment by Robert Hodges [28/Apr/09 10:28 PM]

Based on our experience with Sequoia it will be necessary to introduce a feature to obtain information database host name, port, etc. in order to perform a backup. One way is to get the URL and parse it for the information required to run a backup. Another approach is to configure backup parameters independently.

Comment by Robert Hodges [10/Jun/09 02:46 PM]

Integrated backup needs to support auto-provisioning. This is where a slave automatically restores a backup the first time it starts up and then goes on-line. The idea is that you should be able to provision a new slave simply by turning it on.

This would be implemented by a start-up file somewhat along the lines of the PostgreSQL recovery.conf file (see http://www.postgresql.org/docs/8.3/interactive/continuous-archiving.html). Once we restore successfully we move the file to another name so that the replicator does not accidentally provision twice.

Comment by Robert Hodges [23/Jul/09 05:40 PM]

This is implemented, tested, and documented. (Currently lack full documentation of plug-ins, but that will come. )