Configuring a shared VPC for ROSA clusters

You can create OpenShift Container Platform clusters in shared, centrally-managed AWS virtual private clouds (VPCs).

Note

Installing a new OpenShift Container Platform cluster into a VPC that was automatically created by the installer for a different cluster is not supported.

Note

This process requires two separate AWS accounts that belong to the same AWS organization. One account functions as the VPC-owning AWS account (VPC Owner), while the other account creates the cluster in the cluster-creating AWS account (Cluster Creator).

Prerequisites for the VPC Owner

• You have an AWS account with the proper permissions to create roles and share resources.

• The Cluster Creator’s AWS account is separate from the VPC Owner’s AWS account.

• Both AWS accounts belong to the same AWS organization.

• You enabled resource sharing from the management account for your organization.

• You have access to the AWS console.

Prerequisites for the Cluster Creator

• You installed the ROSA CLI (rosa) 1.2.26 or later.

• You created all of the required for creating a cluster.

• The Cluster Creator’s AWS account is separate from the VPC Owner’s AWS account.

• Both AWS accounts belong to the same AWS organization.

Note

Installing a cluster in a shared VPC is supported only for OpenShift 4.12.34 and later, 4.13.10 and later, and all future 4.y-streams.

Step One - VPC Owner: Configuring a VPC to share within your AWS organization

You can share subnets within a configured VPC with another AWS user account if that account is within your current AWS organization.

Procedure

1. Create or modify a VPC to your specifications in the VPC section of the AWS console.

2. Create a custom policy file to allow for necessary shared VPC permissions that uses the name SharedVPCPolicy:

   $ cat <<EOF > /tmp/shared-vpc-policy.json
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "route53:ChangeResourceRecordSets",
                   "route53:ListHostedZones",
                   "route53:ListHostedZonesByName",
                   "route53:ListResourceRecordSets",
                   "route53:ChangeTagsForResource",
                   "route53:GetAccountLimit",
                   "route53:GetChange",
                   "route53:GetHostedZone",
                   "route53:ListTagsForResource",
                   "route53:UpdateHostedZoneComment",
                   "tag:GetResources",
                   "tag:UntagResources"
               ],
               "Resource": "*"
           }
       ]
   }
   EOF

3. Create the policy in AWS:

   $ aws iam create-policy \
       --policy-name SharedVPCPolicy \
       --policy-document file:///tmp/shared-vpc-policy.json

   You will attach this policy to a role necessary for the shared VPC permissions.

4. Create a custom trust policy file that grants permission to assume roles:

   $ cat <<EOF > /tmp/shared-vpc-role.json
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Principal": {
                   "AWS": "arn:aws:iam::<Account-ID>:root"  
               },
               "Action": "sts:AssumeRole"
           }
       ]
   }
   EOF

   1. The principal will be scoped down after the Cluster Creator creates the necessary cluster roles. On creation, you must create a root user placeholder by using the Cluster Creator’s AWS account ID as arn:aws:iam::{Account}:root.

5. Create the IAM role:

   $ aws iam create-role --role-name <role_name> \  
       --assume-role-policy-document file:///tmp/shared-vpc-role.json

   1. Replace <role_name> with the name of the role you want to create.

6. Attach the custom SharedVPCPolicy permissions policy:

   $ aws iam attach-role-policy --role-name <role_name> --policy-arn \  
       arn:aws:iam::<AWS_account_ID>:policy/SharedVPCPolicy  

   1. Replace <role_name> with the name of the role you created.
   2. Replace <AWS_account_ID> with the VPC Owner’s AWS account ID.

7. Provide the SharedVPCRole ARN to the Cluster Creator to continue configuration.

Additional resources

• See the AWS documentation for sharing your AWS resources.

Step Two - Cluster Creator: Reserving your DNS and creating cluster operator roles

After the VPC Owner creates a virtual private cloud, subnets, and an IAM role for sharing the VPC resources, reserve an openshiftapps.com DNS domain and create Operator roles to communicate back to the VPC Owner.

Note

For shared VPC clusters, you can choose to create the Operator roles after the cluster creation steps. The cluster will be in a waiting state until the Ingress Operator role ARN is added to the shared VPC role trusted relationships.

Prerequisites

• You have the SharedVPCRole ARN for the IAM role from the VPC Owner.

Procedure

1. Reserve an openshiftapps.com DNS domain with the following command:

   $ rosa create dns-domain

   The command creates a reserved openshiftapps.com DNS domain.

   I: DNS domain '14eo.p1.openshiftapps.com' has been created.
   I: To view all DNS domains, run 'rosa list dns-domains'

2. Create an OIDC configuration.

   Review this article for more information on the OIDC configuration process. The following command produces the OIDC configuration ID that you need:

   $ rosa create oidc-config

   You receive confirmation that the command created an OIDC configuration:

   I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice:
   	rosa create operator-roles --prefix <user-defined> --oidc-config-id 25tu67hq45rto1am3slpf5lq6jargg

3. Create the Operator roles by entering the following command:

   $ rosa create operator-roles --oidc-config-id <oidc-config-ID> 
       --installer-role-arn <Installer_Role> 
       --shared-vpc-role-arn <Created_VPC_Role_Arn> 
       --prefix <operator-prefix> 

   1. Provide the OIDC configuration ID that you created in the previous step.
   2. Provide your installer ARN that was created as part of the rosa create account-roles process.
   3. Provide the ARN for the role that the VPC Owner created.
   4. Provide a prefix for the Operator roles.

   Note

   The Installer account role and the shared VPC role must have a one-to-one relationship. If you want to create multiple shared VPC roles, you should create one set of account roles per shared VPC role.

4. After creating the Operator roles, share the following information with the VPC Owner to proceed with the configuration:

   • The full domain name, <intended_cluster_domain_prefix>.<reserved_dns_domain>

   • The ARN for your Ingress Operator Cloud Credentials role.

   • The ARN for your Installer role.

The shared information resembles these examples:

• my-rosa-cluster.14eo.p1.openshiftapps.com

• arn:aws:iam::111122223333:role/ManagedOpenShift-Installer-Role

• arn:aws:iam::111122223333:role/my-rosa-cluster-openshift-ingress-operator-cloud-credentials

Step Three - VPC Owner: Updating the shared VPC role and creating hosted zones

After the Cluster Creator provides the DNS domain and the IAM roles, create a private hosted zone and update the trust policy on the IAM role that was created for sharing the VPC.

Prerequisites

• You have the full domain name from the Cluster Creator.

• You have the Ingress Operator Cloud Credentials role’s ARN from the Cluster Creator.

• You have the Installer role’s ARN from the Cluster Creator.

Procedure

1. In the Resource Access Manager of the AWS console, create a resource share that shares the previously created public and private subnets with the Cluster Creator’s AWS account ID.

2. Update the VPC sharing IAM role and add the Installer and Ingress Operator Cloud Credentials roles to the principal section of the trust policy.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
   	  "Sid": "Statement1",
   	  "Effect": "Allow",
   	  "Principal": {
   	  	"AWS": [
             "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-ingress-operator-cloud-credentials",
             "arn:aws:iam::<Cluster-Creator's-AWS-Account-ID>:role/<prefix>-Installer-Role"
           ]
   	  },
   	  "Action": "sts:AssumeRole"
   	}
     ]
   }

3. Create a private hosted zone in the Route 53 section of the AWS console. Configure the hosted zone following these guidelines:

   • Express the domain name as <cluster_domain_prefix>.<reserved_dns_domain>

   • Associate the private hosted zone with the created VPC.

4. After the hosted zone is created and associated with the VPC, provide the following to the Cluster Creator to continue configuration:

   • Hosted zone ID

   • AWS region

   • Subnet IDs

Step Four - Cluster Creator: Creating your cluster in a shared VPC

To create a cluster in a shared VPC, complete the following steps.

Note

Installing a cluster in a shared VPC is supported only for OpenShift 4.12.34 and later, 4.13.10 and later, and all future 4.y-streams.

Prerequisites

• You have the hosted zone ID from the VPC Owner.

• You have the AWS region from the VPC Owner.

• You have the subnet IDs from the VPC Owner.

• You have the SharedVPCRole ARN from the VPC Owner.

Procedure

• In a terminal, enter the following command to create the shared VPC:

  rosa create cluster --cluster-name <cluster_name> --sts --operator-roles-prefix <prefix> --oidc-config-id <oidc_config_id> --region us-east-1 --subnet-ids <subnet_ids> --private-hosted-zone-id <hosted_zone_ID> --shared-vpc-role-arn <vpc-role-arn> --base-domain <dns-domain>

Note

If your cluster name is longer than 15 characters, it will contain an autogenerated domain prefix as a sub-domain for your provisioned cluster on *.openshiftapps.com.

To customize the subdomain, use the --domain-prefix flag. The domain prefix cannot be longer than 15 characters, must be unique, and cannot be changed after cluster creation.