Skip to content

Configuring the registry for bare metal

Configure image registry storage for bare-metal clusters after installation. Because bare-metal installations do not automatically provision storage, you must change the registry management state from Removed to Managed and configure persistent storage or use Red Hat OpenShift Data Foundation before the registry can store container images.

Image registry removed during installation

On platforms that do not provide shareable object storage, the OpenShift Image Registry Operator bootstraps itself as Removed. This allows openshift-installer to complete installations on these platform types.

After installation, you must edit the Image Registry Operator configuration to switch the managementState from Removed to Managed. When this has completed, you must configure storage.

Changing the image registry’s management state

To start the image registry, you must change the Image Registry Operator configuration’s managementState from Removed to Managed.

Procedure

  • Change managementState Image Registry Operator configuration from Removed to Managed. For example:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"managementState":"Managed"}}'

Image registry storage configuration

The Image Registry Operator is not initially available for platforms that do not provide default storage. After installation, you must configure your registry to use storage so that the Registry Operator is made available.

Configure a persistent volume, which is required for production clusters. Where applicable, you can configure an empty directory as the storage location for non-production clusters.

You can also allow the image registry to use block storage types by using the Recreate rollout strategy during upgrades.

Configuring registry storage for bare metal and other manual installations

To ensure the registry is fully operational, configure the registry to use storage immediately after the cluster installation. This configuration is a mandatory step to enable the registry to store data.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

  • You have a cluster that uses manually-provisioned Red Hat Enterprise Linux CoreOS (RHCOS) nodes, such as bare metal.

  • You have provisioned persistent storage for your cluster, such as Red Hat OpenShift Data Foundation.

    Important

    OpenShift Container Platform supports ReadWriteOnce access for image registry storage when you have only one replica. ReadWriteOnce access also requires that the registry uses the Recreate rollout strategy. To deploy an image registry that supports high availability with two or more replicas, ReadWriteMany access is required.

  • You must have a system with at least 100Gi capacity.

Procedure

  1. To configure your registry to use storage, change the spec.storage.pvc in the configs.imageregistry/cluster resource.

    Note

    When you use shared storage, review your security settings to prevent outside access.

  2. Verify that you do not have a registry pod:

    $ oc get pod -n openshift-image-registry -l docker-registry=default
    Example output
    No resources found in openshift-image-registry namespace

    Note

    If you do have a registry pod in your output, you do not need to continue with this procedure.

  3. Check the registry configuration:

    $ oc edit configs.imageregistry.operator.openshift.io
    Example output
    storage:
  pvc:
    claim:

    Leave the claim field blank to allow the automatic creation of an image-registry-storage PVC.

  4. Check the clusteroperator status:

    $ oc get clusteroperator image-registry
    Example output
    NAME             VERSION              AVAILABLE   PROGRESSING   DEGRADED   SINCE   MESSAGE
image-registry   4.19                 True        False         False      6h50m

  5. Ensure that your registry is set to managed to enable building and pushing of images.

    • Run:

      $ oc edit configs.imageregistry/cluster

      Then, change the line

      managementState: Removed

      to

      managementState: Managed

Configuring storage for the image registry in non-production clusters

You must configure storage for the Image Registry Operator. For non-production clusters, you can set the image registry to an empty directory. If you do so, all images are lost if you restart the registry.

Procedure

  • To set the image registry storage to an empty directory:

    $ oc patch configs.imageregistry.operator.openshift.io cluster --type merge --patch '{"spec":{"storage":{"emptyDir":{}}}}'

    Warning

    Configure this option only for non-production clusters.

    If you run this command before the Image Registry Operator initializes its components, the oc patch command fails with the following error:

    Error from server (NotFound): configs.imageregistry.operator.openshift.io "cluster" not found

    Wait a few minutes and run the command again.

Configuring block registry storage for bare metal

To allow the image registry to use block storage types during upgrades as a cluster administrator, you can use the Recreate rollout strategy.

Important

Block storage volumes, or block persistent volumes, are supported but not recommended for use with the image registry on production clusters. An installation where the registry is configured on block storage is not highly available because the registry cannot have more than one replica.

If you choose to use a block storage volume with the image registry, you must use a filesystem persistent volume claim (PVC).

Procedure

  1. Enter the following command to set the image registry storage as a block storage type, patch the registry so that it uses the Recreate rollout strategy, and runs with only one (1) replica:

    $ oc patch config.imageregistry.operator.openshift.io/cluster --type=merge -p '{"spec":{"rolloutStrategy":"Recreate","replicas":1}}'

  2. Provision the PV for the block storage device, and create a PVC for that volume. The requested block volume uses the ReadWriteOnce (RWO) access mode.

    1. Create a pvc.yaml file with the following contents to define a VMware vSphere PersistentVolumeClaim object:

      kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: image-registry-storage
  namespace: openshift-image-registry
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi

      where:

      name

      Specifies a unique name that represents the PersistentVolumeClaim object.

      namespace

      Specifies the namespace for the PersistentVolumeClaim object, which is openshift-image-registry.

      accessModes

      Specifies the access mode of the persistent volume claim. With ReadWriteOnce, the volume can be mounted with read and write permissions by a single node.

      storage

      The size of the persistent volume claim.

    2. Enter the following command to create the PersistentVolumeClaim object from the file:

      $ oc create -f pvc.yaml -n openshift-image-registry

  3. Enter the following command to edit the registry configuration so that it references the correct PVC:

    $ oc edit config.imageregistry.operator.openshift.io -o yaml
    Example output
    storage:
  pvc:
    claim:

    By creating a custom PVC, you can leave the claim field blank for the default automatic creation of an image-registry-storage PVC.

Configuring the Image Registry Operator to use Ceph RGW storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premise object storage

  • NooBaa, providing a Multicloud Object Gateway

Use the following, procedure to configure the image registry to use Ceph RGW storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

  • You have access to the OpenShift Container Platform web console.

  • You installed the oc CLI.

  • You installed the OpenShift Data Foundation Operator to provide object storage and Ceph RGW object storage.

Procedure

  1. Create the object bucket claim using the ocs-storagecluster-ceph-rgw storage class. For example:

    cat <<EOF | oc apply -f -
apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: rgwbucket
  namespace: openshift-storage
spec:
  storageClassName: ocs-storagecluster-ceph-rgw
  generateBucketName: rgwbucket
EOF

    Alternatively, you can use the openshift-image-registry for the namespace value.

  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage rgwbucket -o jsonpath='{.spec.bucketName}')

  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_ACCESS_KEY_ID}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage rgwbucket -o jsonpath='{.data.AWS_SECRET_ACCESS_KEY}' | base64 --decode)

  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry

  5. Get the route host by entering the following command:

    $ route_host=$(oc get route ocs-storagecluster-cephobjectstore -n openshift-storage --template='{{ .spec.host }}')

  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/$(oc get ingresscontroller -n openshift-ingress-operator default -o json | jq '.spec.defaultCertificate.name // "router-certs-default"' -r) -n openshift-ingress --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config

  7. Configure the image registry to use the Ceph RGW object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

Configuring the Image Registry Operator to use Noobaa storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premise object storage

  • NooBaa, providing a Multicloud Object Gateway

Use the following the procedure to configure the image registry to use Noobaa storage.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

  • You have access to the OpenShift Container Platform web console.

  • You installed the oc CLI.

  • You installed the OpenShift Data Foundation Operator to provide object storage and Noobaa object storage.

Procedure

  1. Create the object bucket claim using the openshift-storage.noobaa.io storage class. For example:

    cat <<EOF | oc apply -f -
apiVersion: objectbucket.io/v1alpha1
kind: ObjectBucketClaim
metadata:
  name: noobaatest
  namespace: openshift-storage
spec:
  storageClassName: openshift-storage.noobaa.io
  generateBucketName: noobaatest
EOF

    Alternatively, you can use the openshift-image-registry for the namespace value.

  2. Get the bucket name by entering the following command:

    $ bucket_name=$(oc get obc -n openshift-storage noobaatest -o jsonpath='{.spec.bucketName}')

  3. Get the AWS credentials by entering the following commands:

    $ AWS_ACCESS_KEY_ID=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_ACCESS_KEY_ID:" | head -n1 | awk '{print $2}' | base64 --decode)
    $ AWS_SECRET_ACCESS_KEY=$(oc get secret -n openshift-storage noobaatest -o yaml | grep -w "AWS_SECRET_ACCESS_KEY:" | head -n1 | awk '{print $2}' | base64 --decode)

  4. Create the secret image-registry-private-configuration-user with the AWS credentials for the new bucket under openshift-image-registry project by entering the following command:

    $ oc create secret generic image-registry-private-configuration-user --from-literal=REGISTRY_STORAGE_S3_ACCESSKEY=${AWS_ACCESS_KEY_ID} --from-literal=REGISTRY_STORAGE_S3_SECRETKEY=${AWS_SECRET_ACCESS_KEY} --namespace openshift-image-registry

  5. Get the route host by entering the following command:

    $ route_host=$(oc get route s3 -n openshift-storage -o=jsonpath='{.spec.host}')

  6. Create a config map that uses an ingress certificate by entering the following commands:

    $ oc extract secret/$(oc get ingresscontroller -n openshift-ingress-operator default -o json | jq '.spec.defaultCertificate.name // "router-certs-default"' -r) -n openshift-ingress --confirm
    $ oc create configmap image-registry-s3-bundle --from-file=ca-bundle.crt=./tls.crt  -n openshift-config

  7. Configure the image registry to use the Nooba object storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","s3":{"bucket":'\"${bucket_name}\"',"region":"us-east-1","regionEndpoint":'\"https://${route_host}\"',"virtualHostedStyle":false,"encrypt":false,"trustedCA":{"name":"image-registry-s3-bundle"}}}}}' --type=merge

Configuring the Image Registry Operator to use CephFS storage with Red Hat OpenShift Data Foundation

Red Hat OpenShift Data Foundation integrates multiple storage types that you can use with the OpenShift image registry:

  • Ceph, a shared and distributed file system and on-premise object storage

  • NooBaa, providing a Multicloud Object Gateway

Use the following procedure to configure the image registry to use CephFS storage.

Note

CephFS uses persistent volume claim (PVC) storage. It is not recommended to use PVCs for image registry storage if there are other options are available, such as Ceph RGW or Noobaa.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

  • You have access to the OpenShift Container Platform web console.

  • You installed the oc CLI.

  • You installed the OpenShift Data Foundation Operator to provide object storage and CephFS file storage.

Procedure

  1. Create a PVC to use the cephfs storage class. For example:

    cat <<EOF | oc apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
 name: registry-storage-pvc
 namespace: openshift-image-registry
spec:
 accessModes:
 - ReadWriteMany
 resources:
   requests:
     storage: 100Gi
 storageClassName: ocs-storagecluster-cephfs
EOF

  2. Configure the image registry to use the CephFS file system storage by entering the following command:

    $ oc patch config.image/cluster -p '{"spec":{"managementState":"Managed","replicas":2,"storage":{"managementState":"Unmanaged","pvc":{"claim":"registry-storage-pvc"}}}}' --type=merge

Additional resources