|
| 1 | +--- |
| 2 | +title: "Data protection overview" |
| 3 | +description: "" |
| 4 | +summary: "" |
| 5 | +date: 2024-09-24T14:22:10+03:00 |
| 6 | +lastmod: 2024-09-24T14:22:10+03:00 |
| 7 | +draft: false |
| 8 | +weight: 410 |
| 9 | +toc: true |
| 10 | +seo: |
| 11 | + title: "" # custom title (optional) |
| 12 | + description: "" # custom description (recommended) |
| 13 | + canonical: "" # custom canonical URL (optional) |
| 14 | + noindex: false # false (default) or true |
| 15 | +--- |
| 16 | + |
| 17 | +NuoDBaaS safeguards customer data by automating the database backup process through user-defined policies that manage backup creation and retention. |
| 18 | +These backups can be used to create new databases or restore a database’s captured state and configuration. This section describes the backup and cloning mechanisms and offers insights into the internal architecture. |
| 19 | + |
| 20 | +## Prerequisites |
| 21 | + |
| 22 | +NuoDBaaS backup solution depends on several external components that must be provisioned in the Kubernetes cluster. |
| 23 | +It leverages [Container Storage Interface](https://kubernetes.io/docs/concepts/storage/volumes/#csi) (CSI) driver and [external snapshotter](https://github.com/kubernetes-csi/external-snapshotter) to backup block volumes. |
| 24 | + |
| 25 | +{{< callout context="note" title="Note" icon="outline/info-circle" >}} |
| 26 | +Follow the documentation from your cloud provider to install and configure the CSI driver. |
| 27 | +For more information on Amazon Elastic Kubernetes Service (EKS) see [Store Kubernetes volumes with Amazon EBS](https://docs.aws.amazon.com/eks/latest/userguide/ebs-csi.html). |
| 28 | +{{< /callout >}} |
| 29 | + |
| 30 | +### Compatibility |
| 31 | + |
| 32 | +Refer to the table below for supported configurations and backup capabilities. |
| 33 | + |
| 34 | +| Minimum NuoDB Control Plane version | Minimum NuoDB version | Minimum NuoDB Helm charts version | Capabilities | |
| 35 | +| ----------------------------|---------------|---------------------------|---------------| |
| 36 | +| [v2.4.1](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.4.1) | [v5.1.1](https://hub.docker.com/r/nuodb/nuodb) | [v3.8.2](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.8.2) | On-demand backup and clone for DBaaS databases. Snapshot-based backup using `fsfreeze` for databases with external journal enabled | |
| 37 | +| [v2.4.1](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.4.1) | [v6.0.2](https://hub.docker.com/r/nuodb/nuodb) | [v3.9.0](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.9.0) | Snapshot-based backup using `hotsnap` for databases with external journal enabled | |
| 38 | +| [v2.6.1](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.6.1) | [v6.0.2](https://hub.docker.com/r/nuodb/nuodb) | [v3.9.0](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.9.0) | Backup policies with scheduling and retention | |
| 39 | +| [v2.7.0](https://github.com/nuodb/nuodb-cp-releases/releases/tag/v2.7.0) | [v6.0.2](https://hub.docker.com/r/nuodb/nuodb) | [v3.9.0](https://github.com/nuodb/nuodb-helm-charts/releases/tag/v3.9.0) | Advanced backup retention configuration | |
| 40 | + |
| 41 | +## Architecture |
| 42 | + |
| 43 | +NuoDBaaS delivers backup support in a cloud-agnostic manner by leveraging several cloud-native components deployed in the Kubernetes cluster. |
| 44 | + |
| 45 | +### CSI driver |
| 46 | + |
| 47 | +Implements the [CSI interface](https://github.com/container-storage-interface/spec) which provisions volumes and snapshots in the cloud provider's storage system. |
| 48 | + |
| 49 | +### External snapshot controller |
| 50 | + |
| 51 | +Implements the CSI interface and reconciles `VolumeSnapshot` and `VolumeSnapshotContent` resources in the Kubernetes cluster. |
| 52 | +Communicates with a CSI driver configured in the `VolumeSnapshotClass` to manage snapshot lifecycle on the cloud. |
| 53 | + |
| 54 | +### Volume snapshot class |
| 55 | + |
| 56 | +Configures the CSI driver and specifies a snapshot class that is used to execute a volume snapshot. |
| 57 | +If there is no default `VolumeSnapshotClass` configured in the cluster, you must specify one using `--backup-volumesnapshot-class` option. |
| 58 | + |
| 59 | +### Backup manager |
| 60 | + |
| 61 | +A backup manager is a system that backs up Kubernetes applications including their data and configuration. |
| 62 | +It is responsible for creating, executing, storing backups, and managing the catalog of available backups. |
| 63 | + |
| 64 | +NuoDBaaS provides an embedded backup manager with the following capabilities: |
| 65 | +- Creates `VolumeSnapshot` resources to capture snapshots of selected database volumes using the cloud provider's storage system |
| 66 | +- Uses `Secret` resources to capture database configuration and manage a backup catalog of the available backups |
| 67 | +- Executes HTTP backup hooks to prepare the database for backup operation |
| 68 | + |
| 69 | +### Backup plugin |
| 70 | + |
| 71 | +Implements the NuoDB backup interface (NBI) for the NuoDB Control Plane to communicate with the backup manager. |
| 72 | +The `DatabaseBackup` custom resource models a backup taken from a specific database in NuoDBaaS. |
| 73 | +The plugin decouples `DatabaseBackup` resource lifecycle (scheduling, deleting, and listing of backups) from the backup execution strategy (creating, storing, importing, and exporting backups). |
| 74 | +NuoDBaaS provides an embedded backup plugin that communicates with the embedded backup manager. |
| 75 | + |
| 76 | +### Backup catalog |
| 77 | + |
| 78 | +The target location where the backup configurations and metadata are stored. |
| 79 | +The backup catalog is not used for the _data_ backup as the snapshots are stored on the cloud provider's storage system. |
| 80 | + |
| 81 | +{{< callout context="caution" title="Caution" icon="outline/alert-triangle" >}} |
| 82 | +The embedded backup manager uses `Secret` resources for a backup catalog bound to a single Kubernetes cluster. |
| 83 | +You must have a process that backs up the Kubernetes _etcd_ database to recover from a disaster including the loss of the Kubernetes cluster. |
| 84 | +{{< /callout >}} |
| 85 | + |
| 86 | +### Backup hooks |
| 87 | + |
| 88 | +Backup hooks are used within the NuoDB Helm charts and prepare the database for a backup. |
| 89 | +Responsible for storing metadata in the database volumes (such as backup ID) supplied later during restore. |
| 90 | + |
| 91 | +A NuoDB database with an external journal enabled requires synchronization before taking snapshots. |
| 92 | +Backup hooks support several synchronization strategies (supplied via the `freezeMode` Helm value): |
| 93 | +- `hotsnap` (recommended) - Uses NuoDB product support for pausing archive writes. |
| 94 | +Supported from NuoDB v6.0.2 and above. |
| 95 | +- `fsfreeze` - Uses `fsfreeze` binary to suspend archive reads and writes. |
| 96 | +This mode requires a privileged sidecar container. |
| 97 | +- `suspend` - The SM process is suspended completely and then resumed using `kill -STOP` and `kill -CONT`. |
| 98 | +This is to enable usage in environments where `fsfreeze` cannot be invoked on the archive filesystem. |
| 99 | +Only for testing purposes. |
| 100 | + |
| 101 | +NuoDBaaS will select the best synchronization strategy depending on the NuoDB product version and NuoDB Helm chart version installed. |
| 102 | +We recommend using the latest NuoDB version and `hotsnap` since it has several advantages such as: |
| 103 | +- Archive reads are not blocked so backup does not prevent cache misses from being fetched |
| 104 | +- Support automatic archive unfreeze with configurable timeout in case performing the snapshot backup takes longer than expected |
| 105 | +- Unintentional Storage Manager restart during backup unfreezes the archive volume |
| 106 | + |
| 107 | +{{< callout context="note" title="Note" icon="outline/info-circle" >}} |
| 108 | +To minimize the impact on SQL application clients, the backup manager unfreezes volumes immediately after the snapshot is created on the cloud provider. |
| 109 | +Some cloud providers, like Outscale, consider a snapshot to be a consistent cut after it is reported _ready_ by the CSI driver. |
| 110 | +You must set `--backup-post-hooks-on-snapshot-ready=true` option when deploying NuoDBaaS in such environments. |
| 111 | +For more information about Outscale Block Storage Unit (BSU) snapshots see [About Snapshots](https://docs.outscale.com/en/userguide/About-Snapshots.html#_general_information). |
| 112 | +{{< /callout >}} |
| 113 | + |
| 114 | +## Backup flow |
| 115 | + |
| 116 | +A `BackupPolicy` custom resource defines the set of rules for performing backup scheduling and retention. |
| 117 | +A database must satisfy the set of rules for at least one backup policy to ensure backups are scheduled for it. |
| 118 | +A `DatabaseBackup` custom resource defines individual database backup requests and holds information for the backup (e.g. backup ID, phase, etc.). |
| 119 | +NuoDBaaS monitors backup policies and schedules backups by creating `DatabaseBackup` resources. |
| 120 | +Deleting a `BackupPolicy` does not delete or invalidate database backups that it has created. |
| 121 | + |
| 122 | +NuoDBaaS prepares the database configuration and database PVCs for backup. |
| 123 | +The backup plugin collaborates with the backup manager to execute backup hooks, create `VolumeSnapshot` resources for the selected database volumes, and capture the database configuration. |
| 124 | +NuoDBaaS monitors the backup progress and updates the `DatabaseBackup` resource. |
| 125 | + |
| 126 | +{{< picture src="backup-flow.png" alt="Database backup flow" >}} |
| 127 | + |
| 128 | +## Restore (clone) flow |
| 129 | + |
| 130 | +A `Database` custom resource may reference a `DatabaseBackup` as a restore source. |
| 131 | +NuoDBaaS validates the backup and merges the database _spec_ with the one stored in the configuration backup. |
| 132 | +The backup manager imports the `VolumeSnapshot` and `VolumeSnapshotContent` resources corresponding to the backup into the cluster so that the NuoDB Helm charts can utilize them as a source for the new database volumes. |
| 133 | + |
| 134 | +{{< picture src="clone-flow.png" alt="Database clone flow" >}} |
| 135 | + |
| 136 | +## Export backup |
| 137 | + |
| 138 | +The NuoDBaaS administrator must export a backup taken on a database from another cluster manually. |
| 139 | +After import, this backup is used to restore a database in the target cluster. |
| 140 | + |
| 141 | +List the available backups on the source cluster. |
| 142 | + |
| 143 | +```sh |
| 144 | +kubectl --context source-cluster get secret -l backup.cp.nuodb.com/backup-id |
| 145 | +``` |
| 146 | + |
| 147 | +The backup `Secret` contains metadata about the actual backup and has a name with the format of `cp.nuodb.com.backup.<org>-<proj>-<db>-<name>`. |
| 148 | + |
| 149 | +Copy the secret from the source cluster to the target cluster. |
| 150 | + |
| 151 | +```sh |
| 152 | +kubectl --context source-cluster get secret \ |
| 153 | + cp.nuodb.com.backup.acme-messaging-demo-20241009000000 -o yaml | kubectl --context target-cluster apply -f - |
| 154 | +``` |
| 155 | + |
| 156 | +For more information on how to import the backup, see [Import backup]({{< ref "../../configuration/backup/on-demand#import-a-backup" >}}) |
| 157 | + |
| 158 | +{{< callout context="caution" title="Caution" icon="outline/alert-triangle" >}} |
| 159 | +NuoDBaaS does not support exporting backups from the Kubernetes cluster across regions. |
| 160 | +{{< /callout >}} |
| 161 | + |
| 162 | +## Configure backup controller |
| 163 | + |
| 164 | +Embedded backup plugin and manager are enabled by default if `VolumeSnapshot` custom resource definition is available in the Kubernetes cluster. |
| 165 | +Administrators can disable backup by setting the `EmbeddedDatabaseBackupPlugin=false` feature gate when installing the NuoDBaaS operator. |
| 166 | + |
| 167 | +The table below lists backup configuration options for the DBaaS operator. |
| 168 | + |
| 169 | +| Option | Description | Default value | |
| 170 | +| ----------------------------|---------------|---------------------------| |
| 171 | +| `--database-backup-check-period` | The period for checking database backup progress for not completed backups | `10s` | |
| 172 | +| `--backup-metadata-namespace` | The namespace where the backup metadata is stored by the embedded backup plugin | DBaaS system namespace (if configured) or the `default` namespace | |
| 173 | +| `--backup-volumesnapshot-class` | The `VolumeSnapshotClass` name used for all `VolumeSnapshot` resources created by embedded backup plugin | Empty string (`""`) which indicate the default `VolumeSnapshotClass` | |
| 174 | +| `--backup-post-hooks-on-snapshot-ready` | Execute the post-backup hooks after the `VolumeSnapshot` resources are reported ready | `false` | |
| 175 | +| `--backup-max-retry-count` | The maximum number of retries performed by the embedded backup manager during backup execution | `4` | |
0 commit comments