Skip to content

Commit b7592c1

Browse files
Added backup and restore sections (#5)
1 parent 7a5f291 commit b7592c1

File tree

10 files changed

+3659
-1044
lines changed

10 files changed

+3659
-1044
lines changed
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
title: "Database data protection"
3+
description: ""
4+
summary: ""
5+
date: 2024-09-24T14:22:10+03:00
6+
lastmod: 2024-09-24T14:22:10+03:00
7+
draft: false
8+
weight: 400
9+
toc: true
10+
sidebar:
11+
collapsed: true
12+
seo:
13+
title: "" # custom title (optional)
14+
description: "" # custom description (recommended)
15+
canonical: "" # custom canonical URL (optional)
16+
noindex: false # false (default) or true
17+
---
105 KB
Loading
79.1 KB
Loading
Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
---
2+
title: "Data protection overview"
3+
description: ""
4+
summary: ""
5+
date: 2024-09-24T14:22:10+03:00
6+
lastmod: 2024-09-24T14:22:10+03:00
7+
draft: false
8+
weight: 410
9+
toc: true
10+
seo:
11+
title: "" # custom title (optional)
12+
description: "" # custom description (recommended)
13+
canonical: "" # custom canonical URL (optional)
14+
noindex: false # false (default) or true
15+
---
16+
17+
NuoDBaaS safeguards customer data by automating the database backup process through user-defined policies that manage backup creation and retention.
18+
These backups can be used to create new databases or restore a database’s captured state and configuration. This section describes the backup and cloning mechanisms and offers insights into the internal architecture.
19+
20+
## Prerequisites
21+
22+
NuoDBaaS backup solution depends on several external components that must be provisioned in the Kubernetes cluster.
23+
It leverages [Container Storage Interface](https://kubernetes.io/docs/concepts/storage/volumes/#csi) (CSI) driver and [external snapshotter](https://github.com/kubernetes-csi/external-snapshotter) to backup block volumes.
24+
25+
{{< callout context="note" title="Note" icon="outline/info-circle" >}}
26+
Follow the documentation from your cloud provider to install and configure the CSI driver.
27+
For more information on Amazon Elastic Kubernetes Service (EKS) see [Store Kubernetes volumes with Amazon EBS](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html).
28+
{{< /callout >}}
29+
30+
### Compatibility
31+
32+
Refer to the table below for supported configurations and backup capabilities.
33+
34+
| Minimum NuoDB Control Plane version | Minimum NuoDB version | Minimum NuoDB Helm charts version | Capabilities |
35+
| ----------------------------|---------------|---------------------------|---------------|
36+
| [v2.4.1](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.4.1) | [v5.1.1](https://hub.docker.com/r/nuodb/nuodb) | [v3.8.2](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.8.2) | On-demand backup and clone for DBaaS databases. Snapshot-based backup using `fsfreeze` for databases with external journal enabled |
37+
| [v2.4.1](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.4.1) | [v6.0.2](https://hub.docker.com/r/nuodb/nuodb) | [v3.9.0](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.9.0) | Snapshot-based backup using `hotsnap` for databases with external journal enabled |
38+
| [v2.6.1](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.6.1) | [v6.0.2](https://hub.docker.com/r/nuodb/nuodb) | [v3.9.0](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.9.0) | Backup policies with scheduling and retention |
39+
| [v2.7.0](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.7.0) | [v6.0.2](https://hub.docker.com/r/nuodb/nuodb) | [v3.9.0](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.9.0) | Advanced backup retention configuration |
40+
41+
## Architecture
42+
43+
NuoDBaaS delivers backup support in a cloud-agnostic manner by leveraging several cloud-native components deployed in the Kubernetes cluster.
44+
45+
### CSI driver
46+
47+
Implements the [CSI interface](https://github.com/container-storage-interface/spec) which provisions volumes and snapshots in the cloud provider's storage system.
48+
49+
### External snapshot controller
50+
51+
Implements the CSI interface and reconciles `VolumeSnapshot` and `VolumeSnapshotContent` resources in the Kubernetes cluster.
52+
Communicates with a CSI driver configured in the `VolumeSnapshotClass` to manage snapshot lifecycle on the cloud.
53+
54+
### Volume snapshot class
55+
56+
Configures the CSI driver and specifies a snapshot class that is used to execute a volume snapshot.
57+
If there is no default `VolumeSnapshotClass` configured in the cluster, you must specify one using `--backup-volumesnapshot-class` option.
58+
59+
### Backup manager
60+
61+
A backup manager is a system that backs up Kubernetes applications including their data and configuration.
62+
It is responsible for creating, executing, storing backups, and managing the catalog of available backups.
63+
64+
NuoDBaaS provides an embedded backup manager with the following capabilities:
65+
- Creates `VolumeSnapshot` resources to capture snapshots of selected database volumes using the cloud provider's storage system
66+
- Uses `Secret` resources to capture database configuration and manage a backup catalog of the available backups
67+
- Executes HTTP backup hooks to prepare the database for backup operation
68+
69+
### Backup plugin
70+
71+
Implements the NuoDB backup interface (NBI) for the NuoDB Control Plane to communicate with the backup manager.
72+
The `DatabaseBackup` custom resource models a backup taken from a specific database in NuoDBaaS.
73+
The plugin decouples `DatabaseBackup` resource lifecycle (scheduling, deleting, and listing of backups) from the backup execution strategy (creating, storing, importing, and exporting backups).
74+
NuoDBaaS provides an embedded backup plugin that communicates with the embedded backup manager.
75+
76+
### Backup catalog
77+
78+
The target location where the backup configurations and metadata are stored.
79+
The backup catalog is not used for the _data_ backup as the snapshots are stored on the cloud provider's storage system.
80+
81+
{{< callout context="caution" title="Caution" icon="outline/alert-triangle" >}}
82+
The embedded backup manager uses `Secret` resources for a backup catalog bound to a single Kubernetes cluster.
83+
You must have a process that backs up the Kubernetes _etcd_ database to recover from a disaster including the loss of the Kubernetes cluster.
84+
{{< /callout >}}
85+
86+
### Backup hooks
87+
88+
Backup hooks are used within the NuoDB Helm charts and prepare the database for a backup.
89+
Responsible for storing metadata in the database volumes (such as backup ID) supplied later during restore.
90+
91+
A NuoDB database with an external journal enabled requires synchronization before taking snapshots.
92+
Backup hooks support several synchronization strategies (supplied via the `freezeMode` Helm value):
93+
- `hotsnap` (recommended) - Uses NuoDB product support for pausing archive writes.
94+
Supported from NuoDB v6.0.2 and above.
95+
- `fsfreeze` - Uses `fsfreeze` binary to suspend archive reads and writes.
96+
This mode requires a privileged sidecar container.
97+
- `suspend` - The SM process is suspended completely and then resumed using `kill -STOP` and `kill -CONT`.
98+
This is to enable usage in environments where `fsfreeze` cannot be invoked on the archive filesystem.
99+
Only for testing purposes.
100+
101+
NuoDBaaS will select the best synchronization strategy depending on the NuoDB product version and NuoDB Helm chart version installed.
102+
We recommend using the latest NuoDB version and `hotsnap` since it has several advantages such as:
103+
- Archive reads are not blocked so backup does not prevent cache misses from being fetched
104+
- Support automatic archive unfreeze with configurable timeout in case performing the snapshot backup takes longer than expected
105+
- Unintentional Storage Manager restart during backup unfreezes the archive volume
106+
107+
{{< callout context="note" title="Note" icon="outline/info-circle" >}}
108+
To minimize the impact on SQL application clients, the backup manager unfreezes volumes immediately after the snapshot is created on the cloud provider.
109+
Some cloud providers, like Outscale, consider a snapshot to be a consistent cut after it is reported _ready_ by the CSI driver.
110+
You must set `--backup-post-hooks-on-snapshot-ready=true` option when deploying NuoDBaaS in such environments.
111+
For more information about Outscale Block Storage Unit (BSU) snapshots see [About Snapshots](https://docs.outscale.com/en/userguide/About-Snapshots.html#_general_information).
112+
{{< /callout >}}
113+
114+
## Backup flow
115+
116+
A `BackupPolicy` custom resource defines the set of rules for performing backup scheduling and retention.
117+
A database must satisfy the set of rules for at least one backup policy to ensure backups are scheduled for it.
118+
A `DatabaseBackup` custom resource defines individual database backup requests and holds information for the backup (e.g. backup ID, phase, etc.).
119+
NuoDBaaS monitors backup policies and schedules backups by creating `DatabaseBackup` resources.
120+
Deleting a `BackupPolicy` does not delete or invalidate database backups that it has created.
121+
122+
NuoDBaaS prepares the database configuration and database PVCs for backup.
123+
The backup plugin collaborates with the backup manager to execute backup hooks, create `VolumeSnapshot` resources for the selected database volumes, and capture the database configuration.
124+
NuoDBaaS monitors the backup progress and updates the `DatabaseBackup` resource.
125+
126+
{{< picture src="backup-flow.png" alt="Database backup flow" >}}
127+
128+
## Restore (clone) flow
129+
130+
A `Database` custom resource may reference a `DatabaseBackup` as a restore source.
131+
NuoDBaaS validates the backup and merges the database _spec_ with the one stored in the configuration backup.
132+
The backup manager imports the `VolumeSnapshot` and `VolumeSnapshotContent` resources corresponding to the backup into the cluster so that the NuoDB Helm charts can utilize them as a source for the new database volumes.
133+
134+
{{< picture src="clone-flow.png" alt="Database clone flow" >}}
135+
136+
## Export backup
137+
138+
The NuoDBaaS administrator must export a backup taken on a database from another cluster manually.
139+
After import, this backup is used to restore a database in the target cluster.
140+
141+
List the available backups on the source cluster.
142+
143+
```sh
144+
kubectl --context source-cluster get secret -l backup.cp.nuodb.com/backup-id
145+
```
146+
147+
The backup `Secret` contains metadata about the actual backup and has a name with the format of `cp.nuodb.com.backup.<org>-<proj>-<db>-<name>`.
148+
149+
Copy the secret from the source cluster to the target cluster.
150+
151+
```sh
152+
kubectl --context source-cluster get secret \
153+
cp.nuodb.com.backup.acme-messaging-demo-20241009000000 -o yaml | kubectl --context target-cluster apply -f -
154+
```
155+
156+
For more information on how to import the backup, see [Import backup]({{< ref "../../configuration/backup/on-demand#import-a-backup" >}})
157+
158+
{{< callout context="caution" title="Caution" icon="outline/alert-triangle" >}}
159+
NuoDBaaS does not support exporting backups from the Kubernetes cluster across regions.
160+
{{< /callout >}}
161+
162+
## Configure backup controller
163+
164+
Embedded backup plugin and manager are enabled by default if `VolumeSnapshot` custom resource definition is available in the Kubernetes cluster.
165+
Administrators can disable backup by setting the `EmbeddedDatabaseBackupPlugin=false` feature gate when installing the NuoDBaaS operator.
166+
167+
The table below lists backup configuration options for the DBaaS operator.
168+
169+
| Option | Description | Default value |
170+
| ----------------------------|---------------|---------------------------|
171+
| `--database-backup-check-period` | The period for checking database backup progress for not completed backups | `10s` |
172+
| `--backup-metadata-namespace` | The namespace where the backup metadata is stored by the embedded backup plugin | DBaaS system namespace (if configured) or the `default` namespace |
173+
| `--backup-volumesnapshot-class` | The `VolumeSnapshotClass` name used for all `VolumeSnapshot` resources created by embedded backup plugin | Empty string (`""`) which indicate the default `VolumeSnapshotClass` |
174+
| `--backup-post-hooks-on-snapshot-ready` | Execute the post-backup hooks after the `VolumeSnapshot` resources are reported ready | `false` |
175+
| `--backup-max-retry-count` | The maximum number of retries performed by the embedded backup manager during backup execution | `4` |
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
title: "Database backup"
3+
description: ""
4+
summary: ""
5+
date: 2024-09-24T14:22:10+03:00
6+
lastmod: 2024-09-24T14:22:10+03:00
7+
draft: false
8+
weight: 330
9+
toc: true
10+
sidebar:
11+
collapsed: true
12+
seo:
13+
title: "" # custom title (optional)
14+
description: "" # custom description (recommended)
15+
canonical: "" # custom canonical URL (optional)
16+
noindex: false # false (default) or true
17+
---

0 commit comments

Comments
 (0)