Back up a cluster

This topic describes how to back up data in a KES cluster to Kingsoft Cloud Standard Storage Service (KS3) and how to restore data from backups. You can enable automatic backup on a daily basis in the KES console, or manually back up data by running scripts.

Backup and restoration are implemented based on the snapshot system.

Enable automatic backup

If automatic backup is enabled, the system creates a snapshot for the KES cluster as scheduled on a daily basis.

To enable automatic backup, perform the following operations:

1. Activate the KS3 service and create a bucket for storing backup data.
2. In the cluster list, find the cluster for which you want to enable automatic backup and choose Management > Data Backup in the Operation column to go to the backup management page.
3. Turn on Automatic Backup, set the daily backup time, and specify the destination bucket.
4. Click Submit.

Note: The system automatically creates a snapshot repository named SystemAutoBackup in KES. The first backup is a full backup, followed by incremental backups every day.

Restore data from automatic snapshots

The procedure of restoring data from an automatic snapshot is the same as that of restoring data from a manual snapshot. The difference lies in that the name of the automatic snapshot repository is SystemAutoBackup. Therefore, you can view the automatic snapshot list and select a snapshot of a specific day to restore data based on the date information in the snapshot name.

To view the automatic snapshot list, run the following command:

GET /_snapshot/SystemAutoBackup/_all

From the returned snapshot list, select snap_20191202_000543 with the time of 2019-12-02T0:05:43 and restore two indexes from it.

POST /_snapshot/SystemAutoBackup/snap_20191202_150405/_restore
{
  "indices": "my_index_1,my_index_2",
  "include_global_state": false,
  "rename_pattern": "(.+)",
  "rename_replacement": "restored_$1"   // Specify the new index names as restored_my_index_1 and restored_my_index_2.
}

Manually back up data

If automatic backup cannot meet your requirements, you can manually back up data. Automatic backup data and manual backup data are independent of each other.

Before manual backup, make sure that you have activated KS3 and created a bucket for storing backup data.

You can perform all the following operations by sending HTTP requests to KES.

Create a snapshot repository

Create a snapshot repository named my_repo.

PUT /_snapshot/my_repo
{
  "type": "s3",
  "settings": {
    "bucket": "my_bucket_name",
    "base_path": "my_path",
    "region": "xxxxx",
    "endpoint": "xxxxx.ksyun.com",
    "access_key": "xxxxxx",
    "secret_key": "xxxxxx"
  }
}

Notes:

• bucket: the name of the bucket that you created in KS3.
• base_path: the path for saving the backup data in the bucket.
• region: the region where your KES cluster resides. Use the region name in the KS3 endpoint.
• endpoint: the KS3 endpoint in the region where your KES cluster resides. Use the KS3 internal endpoint.
• access_key: the AccessKeyID of your Kingsoft Cloud account.
• secret_key: the SecretAccessKey of your Kingsoft Cloud account.

View snapshot repositories

View the repository list.

GET /_snapshot

View a specific repository.

GET /_snapshot/my_repo

Delete a snapshot repository

DELETE /_snapshot/my_repo

This operation does not delete the data in KS3.

Create a snapshot

This operation triggers a data backup of KES.

Create a snapshot of all indexes and name the snapshot my_snap1.

PUT /_snapshot/my_repo/my_snap1

Create a snapshot of some indexes and name the snapshot my_snap2.

PUT /_snapshot/my_repo/my_snap2
{
  "indices": "my_index_1,my_index_2"
}

Manage snapshots and restore data from snapshots

View snapshots

View the snapshot list.

GET /_snapshot/my_repo/_all

View a specific snapshot.

GET /_snapshot/my_repo/my_snap1

View snapshots that are being created

View snapshots that are being created.

GET /_snapshot/my_repo/_current

View the detailed progress of snapshot creation, including the shard backup progress.

GET /_snapshot/_status
GET /_snapshot/my_repo/_status
GET /_snapshot/my_repo/my_snap1/_status
GET /_snapshot/my_repo/my_snap1,my_snap2/_status

Delete a snapshot

Delete a completed snapshot or a snapshot that is being created. The data in KS3 is also deleted.

DELETE /_snapshot/my_repo/my_snap1

Restore data from a snapshot

When you restore data from a snapshot, KES imports the data from KS3 and create new indexes. If an index already exists in KES, the restoration fails.

Restore all data in my_snap1.

POST /_snapshot/my_repo/my_snap1/_restore

Restore a specified index and use a new index name.

POST /_snapshot/my_repo/my_snap1/_restore
{
  "indices": "my_index_1",
  "include_global_state": false,
  "rename_pattern": "(.+)",
  "rename_replacement": "restored_$1"   // Specify the new index name as restored_my_index_1.
}

View the restoration status

View the status and progress of index restoration.

GET /restored_my_index_1/_recovery

Cancel restoration

Delete the target index to cancel the restoration.

DELETE /restored_my_index_1