OADP Data protection test

The DataProtectionTest (DPT) is a custom resource (CR) that provides a framework to validate your OADP configuration. The DPT CR checks and reports information for the following parameters:

• The upload performance of the backups to the object storage.

• The CSI snapshot readiness for persistent volume claims.

• The storage bucket configuration, such as encryption and versioning.

Using this information in the DPT CR, you can ensure that your data protection environment is properly configured and performing according to the set configuration.

OADP DataProtectionTest CR specification fields

You can configure the following specification fields in the DataProtectionTest (DPT) custom resource (CR).

Table 1. DPT CR spec fields

Field	Type	Description
backupLocationName	string	Name of the BackupStorageLocation CR configured in the DataProtectionApplication (DPA) CR.
backupLocationSpec	object	Inline specification of the BackupStorageLocation CR.
uploadSpeedTestConfig	object	Configuration to run an upload speed test to the object storage.
csiVolumeSnapshotTestConfigs	list	List of persistent volume claims to take a snapshot of and to verify the snapshot readiness.
forceRun	boolean	Re-run the DPT CR even if status is Complete or Failed.
skipTLSVerify	boolean	Bypasses the TLS certificate validation if set to true.

OADP DataProtectionTest CR status fields

You can review the status of the DataProtectionTest (DPT) custom resource (CR) by using the following status fields:

Table 2. DPT CR status fields

Field	Type	Description
phase	string	Current phase of the DPT CR. Values are InProgress, Complete, or Failed.
lastTested	timestamp	The timestamp when the DPT CR was last run.
uploadTest	object	Results of the upload speed test.
bucketMetadata	object	Information about the storage bucket encryption and versioning.
snapshotTests	list	Snapshot test results for each persistent volume claim.
snapshotSummary	string	Aggregated pass/fail summary for snapshots. For example, 2/2 passed.
s3Vendor	string	AWS S3-compatible storage bucket vendors. For example, AWS, MinIO, Ceph.
errorMessage	string	Error message if the DPT CR fails.

Using the DataProtectionTest custom resource

You can configure the DataProtectionTest (DPT) custom resource (CR) and then run the DPT CR to verify the Container Storage Initiative (CSI) snapshot readiness and the data upload performance to the storage bucket.

Prerequisites

• You have logged in to the OpenShift Container Platform cluster as a user with the cluster-admin role.

• You have installed the OpenShift CLI (oc).

• You have installed the OADP Operator.

• You have created the DataProtectionApplication (DPA) CR.

• You have configured a backup storage location (BSL) to store the backups.

• You have an application with persistent volume claims (PVCs) running in a separate namespace.

Procedure

1. Create a manifest file for the DPT CR as shown in the example:

   apiVersion: oadp.openshift.io/v1alpha1
   kind: DataProtectionTest
   metadata:
     name: dpt-sample
     namespace: openshift-adp
   spec:
     backupLocationName: <bsl_name> 
     csiVolumeSnapshotTestConfigs: 
     - snapshotClassName: csi-gce-pd-vsc
       timeout: 90s
       volumeSnapshotSource:
         persistentVolumeClaimName: <pvc1_name> 
         persistentVolumeClaimNamespace: <pvc_namespace> 
     - snapshotClassName: csi-gce-pd-vsc
       timeout: 120s
       volumeSnapshotSource:
         persistentVolumeClaimName: <pvc2_name> 
         persistentVolumeClaimNamespace: <pvc_namespace>
     forceRun: false 
     uploadSpeedTestConfig: 
       fileSize: 200MB
       timeout: 120s

   1. Specify the name of the BSL.
   2. Specify a list for csiVolumeSnapshotTestConfigs. In this example, two PVCs are being tested.
   3. Specify the name of the first PVC.
   4. Specify the namespace of the PVC.
   5. Specify the name of the second PVC.
   6. Set the forceRun flag to false if you want to make the OADP controller skip re-running tests.
   7. Configure the uploadSpeedTestConfig object by setting the fileSize and timeout fields.

2. Create the DPT CR by running the following command:

   $ oc create -f <dpt_file_name> 

   1. Specify the file name of the DPT manifest.

Verification

1. Verify that the phase of the DPT CR is Complete by running the following command:

   $ oc get dpt dpt-sample

   The example output is as following:

   NAME         PHASE      LASTTESTED   UPLOADSPEED(MBPS)   ENCRYPTION   VERSIONING   SNAPSHOTS    AGE
   dpt-sample   Complete   17m          546                 AES256       Enabled      2/2 passed   17m

2. Verify that the CSI snapshots are ready and the data upload tests are successful by running the following command:

   $ oc get dpt dpt-sample -o yaml

   The example output is as following:

   apiVersion: oadp.openshift.io/v1alpha1
   kind: DataProtectionTest
   ....
   status:
     bucketMetadata: 
       encryptionAlgorithm: AES256
       versioningStatus: Enabled
     lastTested: "202...:47:51Z"
     phase: Complete
     s3Vendor: AWS 
     snapshotSummary: 2/2 passed 
     snapshotTests:
     - persistentVolumeClaimName: mysql-data
       persistentVolumeClaimNamespace: ocp-mysql
       readyDuration: 24s
       status: Ready
     - persistentVolumeClaimName: mysql-data1
       persistentVolumeClaimNamespace: ocp-mysql
       readyDuration: 40s
       status: Ready
     uploadTest: 
       duration: 3.071s
       speedMbps: 546
       success: true

   1. The bucket metadata information.
   2. The S3 bucket vendor.
   3. Summary of the CSI snapshot tests.
   4. The upload test details.

Running a data protection test by configuring a backup storage location specification

You can configure the DataProtectionTest (DPT) custom resource (CR) by specifying the backup storage location (BSL) specification instead of a BSL name. You then run the DPT CR to verify the Container Storage Initiative (CSI) snapshot readiness and the data upload performance to the storage bucket.

Prerequisites

• You have logged in to the OpenShift Container Platform cluster as a user with the cluster-admin role.

• You have installed the OpenShift CLI (oc).

• You have installed the OADP Operator.

• You have created the DataProtectionApplication (DPA) CR.

• You have configured a bucket to store the backups.

• You have created the Secret object to access the bucket storage.

• You have an application with persistent volume claims (PVCs) running in a separate namespace.

Procedure

1. Create a manifest file for the DPT CR as shown in the example:

   apiVersion: oadp.openshift.io/v1alpha1
   kind: DataProtectionTest
   metadata:
     name: dpt-sample
     namespace: openshift-adp
   spec:
     backupLocationSpec: 
       provider: aws
       default: true
       objectStorage:
         bucket: sample-bucket 
         prefix: velero
       config:
         region: us-east-1 
         profile: "default"
         insecureSkipTLSVerify: "true"
         s3Url: "https://s3.amazonaws.com/sample-bucket"
       credential: 
         name: cloud-credentials
         key: cloud
     uploadSpeedTestConfig: 
       fileSize: 50MB
       timeout: 120s
     csiVolumeSnapshotTestConfigs: 
       - volumeSnapshotSource:
           persistentVolumeClaimName: mongo
           persistentVolumeClaimNamespace: mongo-persistent
         snapshotClassName: csi-snapclass
         timeout: 2m
     forceRun: true
     skipTLSVerify: true 

   1. Configure the BSL spec by specifying details such as the cloud provider.
   2. Specify the bucket name. In this example, the bucket name is sample-bucket.
   3. Specify the cloud provider region.
   4. Specify the cloud credentials for the storage bucket.
   5. (Optional) Configure the uploadSpeedTestConfig object by setting the fileSize and timeout fields.
   6. Configure the csiVolumeSnapshotTestConfigs object.
   7. Set to true to skip the TLS certificate validation during the DPT CR run.

2. Create the DPT CR by running the following command:

   $ oc create -f <dpt_file_name> 

   1. Specify the file name of the DPT manifest.

Verification

1. Verify that the phase of the DPT CR is Complete by running the following command:

   $ oc get dpt dpt-sample

   The example output is as following:

   NAME         PHASE      LASTTESTED   UPLOADSPEED(MBPS)   ENCRYPTION   VERSIONING   SNAPSHOTS    AGE
   dpt-sample   Complete   17m          546                 AES256       Enabled      2/2 passed   17m

Running a data protection test on an Azure object storage

If you are using OADP on an Azure object storage, you need to specify the Azure STORAGE_ACCOUNT_ID as part of the secret object. Use the following procedure to run a DataProtectionTest (DPT) custom resource (CR) on an Azure cluster.

Prerequisites

• You have logged in to the Azure cluster as a user with the cluster-admin role.

• You have installed the OpenShift CLI (oc).

• You have installed the OADP Operator.

• You have configured a bucket to store the backups.

• You have an application with persistent volume claims (PVCs) running in a separate namespace.

Procedure

1. Add the Storage Blob Data Contributor role to Azure storageAccount object to avoid DPT run failure. Run the following command:

   $ az role assignment create \
   --assignee "$AZURE_CLIENT_ID" \
   --role "Storage Blob Data Contributor" \
   --scope "/subscriptions/$AZURE_SUBSCRIPTION_ID/resourceGroups/$AZURE_RESOURCE_GROUP/providers/Microsoft.Storage/storageAccounts/$AZURE_STORAGE_ACCOUNT_ID"

2. In your terminal, export the Azure parameters and create a secret credentials file with the parameters as shown in the following example.

   To run the DPT CR on Azure, you need to specify the STORAGE_ACCOUNT_ID parameter in the secret credentials file.

   AZURE_SUBSCRIPTION_ID=<subscription-id>
   AZURE_TENANT_ID=<tenant-id>
   AZURE_CLIENT_ID=<client-id>
   AZURE_CLIENT_SECRET=<client-secret>
   AZURE_RESOURCE_GROUP=<resource-group>
   AZURE_STORAGE_ACCOUNT_ID=<storage-account>

3. Create the Secret CR as shown in the following example:

   $ oc create secret generic cloud-credentials-azure -n openshift-adp --from-file cloud=<credentials_file_path>

4. Create the DataProtectionApplication (DPA) CR by using the configuration shown in the following example:

   apiVersion: oadp.openshift.io/v1alpha1
   kind: DataProtectionApplication
   metadata:
     name: ts-dpa
     namespace: openshift-adp
   spec:
     configuration:
       velero:
         defaultPlugins:
           - azure
           - openshift
     backupLocations:
       - velero:
           config:
             resourceGroup: oadp-....-b7q4-rg
             storageAccount: oadp...kb7q4
             subscriptionId: 53b8f5...fd54c8a
           credential:
             key: cloud
             name: cloud-credentials-azure 
           provider: azure
           default: true
           objectStorage:
             bucket: <bucket_name>
             prefix: velero

   1. Specify the name of the Secret object. In this example, the name is cloud-credentials-azure.

5. Create the DPT CR by specifying the name of backup storage location (BSL), VolumeSnapshotClass object, and the persistent volume claim details as shown in the following example:

   apiVersion: oadp.openshift.io/v1alpha1
   kind: DataProtectionTest
   metadata:
     name: dpt-sample
     namespace: openshift-adp
   spec:
     backupLocationName: <bsl_name> 
     uploadSpeedTestConfig:
       fileSize: 40MB
       timeout: 120s
     csiVolumeSnapshotTestConfigs:
       - snapshotClassName: csi-azuredisk-vsc 
         timeout: 90s
         volumeSnapshotSource:
           persistentVolumeClaimName: mysql-data 
           persistentVolumeClaimNamespace: ocp-mysql 
       - snapshotClassName: csi-azuredisk-vsc
         timeout: 120s
         volumeSnapshotSource:
           persistentVolumeClaimName: mysql-data1
           persistentVolumeClaimNamespace: ocp-mysql

   1. Specify the name of the BSL.
   2. The Azure snapshot class name.
   3. The name of the persistent volume claim.
   4. The name of the persistent volume claim namespace.

6. Run the DPT CR to verify the snapshot readiness.

Troubleshooting the DataProtectionTest custom resource

Use the following table to troubleshoot common issues when running the DataProtectionTest (DPT) custom resource (CR).

Table 3. DPT CR troubleshooting

Error	Reason	Solution
DPT stuck in InProgress state	Bucket credentials or bucket access failure	Check Secret object, bucket permissions, and logs.
Upload test failed	Incorrect Secret object or S3 endpoint	Check the BackupStorageLocation object config and the access keys.
Snapshot tests fail	Incorrect configuration of CSI snapshot controller	Check the VolumeSnapshotClass object availability and the CSI driver logs.
Bucket encryption or versioning not populated	Cloud provider limitations	Not all object storage providers expose these fields consistently.