AWS prerequisites for ROSA

OpenShift Container Platform (ROSA) provides a model that allows Red Hat to deploy clusters into a customer’s existing Amazon Web Service (AWS) account.

You must ensure that the prerequisites are met before installing ROSA. This requirements document does not apply to AWS Security Token Service (STS). If you are using STS, see the STS-specific requirements.

Tip

AWS Security Token Service (STS) is the recommended credential mode for installing and interacting with clusters on OpenShift Container Platform because it provides enhanced security.

Customer Requirements

OpenShift Container Platform (ROSA) clusters must meet several prerequisites before they can be deployed.

Note

In order to create the cluster, the user must be logged in as an IAM user and not an assumed role or STS user.

Account

The customer ensures that the AWS limits are sufficient to support OpenShift Container Platform provisioned within the customer’s AWS account.
The customer’s AWS account should be in the customer’s AWS Organizations with the applicable service control policy (SCP) applied.

Note

It is not a requirement that the customer’s account be within the AWS Organizations or for the SCP to be applied, however Red Hat must be able to perform all the actions listed in the SCP without restriction.
The customer’s AWS account should not be transferable to Red Hat.
The customer may not impose AWS usage restrictions on Red Hat activities. Imposing restrictions will severely hinder Red Hat’s ability to respond to incidents.
The customer may deploy native AWS services within the same AWS account.

Note

Customers are encouraged, but not mandated, to deploy resources in a Virtual Private Cloud (VPC) separate from the VPC hosting OpenShift Container Platform and other Red Hat supported services.

Access requirements

To appropriately manage the OpenShift Container Platform service, Red Hat must have the AdministratorAccess policy applied to the administrator role at all times. This requirement does not apply if you are using AWS Security Token Service (STS).

Note

This policy only provides Red Hat with permissions and capabilities to change resources in the customer-provided AWS account.
Red Hat must have AWS console access to the customer-provided AWS account. This access is protected and managed by Red Hat.
The customer must not utilize the AWS account to elevate their permissions within the OpenShift Container Platform cluster.
Actions available in the OpenShift Container Platform (ROSA) CLI, rosa, or OpenShift Cluster Manager console must not be directly performed in the customer’s AWS account.

Support requirements

Red Hat recommends that the customer have at least Business Support from AWS.
Red Hat has authority from the customer to request AWS support on their behalf.
Red Hat has authority from the customer to request AWS resource limit increases on the customer’s account.
Red Hat manages the restrictions, limitations, expectations, and defaults for all OpenShift Container Platform clusters in the same manner, unless otherwise specified in this requirements section.

Security requirements

Volume snapshots will remain within the customer’s AWS account and customer-specified region.
Red Hat must have ingress access to EC2 hosts and the API server from allow-listed IP addresses.
Red Hat must have egress allowed to forward system and audit logs to a Red Hat managed central logging stack.

Required customer procedure

Complete these steps before deploying OpenShift Container Platform (ROSA).

Procedure

If you, as the customer, are utilizing AWS Organizations, then you must use an AWS account within your organization or create a new one.
To ensure that Red Hat can perform necessary actions, you must either create a service control policy (SCP) or ensure that none is applied to the AWS account.
Attach the SCP to the AWS account.
Follow the ROSA procedures for setting up the environment.

Minimum set of effective permissions for service control policies (SCP)

Service control policies (SCP) are a type of organization policy that manages permissions within your organization. SCPs ensure that accounts within your organization stay within your defined access control guidelines. These policies are maintained in AWS Organizations and control the services that are available within the attached AWS accounts. SCP management is the responsibility of the customer.

Note

The minimum SCP requirement does not apply when using AWS Security Token Service (STS). For more information about STS, see AWS prerequisites for ROSA with STS.

Verify that your service control policy (SCP) does not restrict any of these required permissions.

Service

Actions

Effect

Required

Amazon EC2

All

Allow

Amazon EC2 Auto Scaling

All

Allow

Amazon S3

All

Allow

Identity And Access Management

All

Allow

Elastic Load Balancing

All

Allow

Elastic Load Balancing V2

All

Allow

Amazon CloudWatch

All

Allow

Amazon CloudWatch Events

All

Allow

Amazon CloudWatch Logs

All

Allow

AWS EC2 Instance Connect

SendSerialConsoleSSHPublicKey

Allow

AWS Support

All

Allow

AWS Key Management Service

All

Allow

AWS Security Token Service

All

Allow

AWS Tiro

CreateQuery

GetQueryAnswer

GetQueryExplanation

Allow

AWS Marketplace

Subscribe

Unsubscribe

View Subscriptions

Allow

AWS Resource Tagging

All

Allow

AWS Route53 DNS

All

Allow

AWS Service Quotas

ListServices

GetRequestedServiceQuotaChange

GetServiceQuota

RequestServiceQuotaIncrease

ListServiceQuotas

Allow

Optional

AWS Billing

ViewAccount

Viewbilling

ViewUsage

Allow

AWS Cost and Usage Report

All

Allow

AWS Cost Explorer Services

All

Allow

Additional resources

Red Hat managed IAM references for AWS

Red Hat is responsible for creating and managing the following Amazon Web Services (AWS) resources: IAM policies, IAM users, and IAM roles.

IAM Policies

Note

IAM policies are subject to modification as the capabilities of OpenShift Container Platform change.

The AdministratorAccess policy is used by the administration role. This policy provides Red Hat the access necessary to administer the OpenShift Container Platform (ROSA) cluster in the customer’s AWS account.
```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "*",
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}
```

IAM users

The osdManagedAdmin user is created immediately after installing ROSA into the customer’s AWS account.

Provisioned AWS Infrastructure

This is an overview of the provisioned Amazon Web Services (AWS) components on a deployed OpenShift Container Platform cluster.

EC2 instances

AWS EC2 instances are required to deploy the control plane and data plane functions for OpenShift Container Platform. Instance types can vary for control plane and infrastructure nodes, depending on the worker node count.

At a minimum, the following EC2 instances are deployed:

Three m5.2xlarge control plane nodes
Two r5.xlarge infrastructure nodes
Two m5.xlarge worker nodes

The instance type shown for worker nodes is the default value, but you can customize the instance type for worker nodes according to the needs of your workload.

Amazon Elastic Block Store storage

Amazon Elastic Block Store (Amazon EBS) block storage is used for both local node storage and persistent volume storage. By default, the following storage is provisioned for each EC2 instance:

Control Plane Volume
- Size: 350GB
- Type: gp3
- Input/Output Operations Per Second: 1000
Infrastructure Volume
- Size: 300GB
- Type: gp3
- Input/Output Operations Per Second: 900
Worker Volume
- Default size: 300 GiB (adjustable at creation time)
- Minimum size: 128GB
- Type: gp3
- Input/Output Operations Per Second: 900

Note

Clusters deployed before the release of OpenShift Container Platform 4.11 use gp2 type storage by default.

Elastic Load Balancing

Each cluster can use up to two Classic Load Balancers for application router and up to two Network Load Balancers for API. For more information, see the ELB documentation for AWS.

S3 storage

The image registry is backed by AWS S3 storage. Resources are pruned regularly to optimize S3 usage and cluster performance.

Note

Two buckets are required with a typical size of 2TB each.

VPC

Configure your VPC according to the following requirements:

Subnets: Every cluster requires a minimum of one private subnet for every availability zone. For example, 1 private subnet is required for a single-zone cluster, and 3 private subnets are required for a cluster with 3 availability zones.

If your cluster needs direct access to a network that is external to the cluster, including the public internet, you require at least one public subnet.

Red Hat strongly recommends using unique subnets for each cluster. Sharing subnets between multiple clusters is not recommended.

Note

A public subnet connects directly to the internet through an internet gateway.

A private subnet connects to the internet through a network address translation (NAT) gateway.
Route tables: One route table per private subnet, and one additional table per cluster.
Internet gateways: One Internet Gateway per cluster.
NAT gateways: One NAT Gateway per public subnet.

VPC Reference Architecture — Figure 1. Sample VPC Architecture

Security groups

AWS security groups provide security at the protocol and port access level; they are associated with EC2 instances and Elastic Load Balancing (ELB) load balancers. Each security group contains a set of rules that filter traffic coming in and out of one or more EC2 instances.

Ensure that the ports required for cluster installation and operation are open on your network and configured to allow access between hosts. The requirements for the default security groups are listed in Required ports for default security groups.

Table 1. Required ports for default security groups
Group	Type	IP Protocol	Port range
MasterSecurityGroup	`AWS::EC2::SecurityGroup`	`icmp`	`0`
		`tcp`	`22`
		`tcp`	`6443`
		`tcp`	`22623`
WorkerSecurityGroup	`AWS::EC2::SecurityGroup`	`icmp`	`0`
WorkerSecurityGroup	`AWS::EC2::SecurityGroup`	`tcp`	`22`
BootstrapSecurityGroup	`AWS::EC2::SecurityGroup`	`tcp`	`22`
BootstrapSecurityGroup	`AWS::EC2::SecurityGroup`	`tcp`	`19531`

Networking prerequisites

The following sections detail the requirements to create your cluster.

Minimum bandwidth

During cluster deployment, OpenShift Container Platform requires a minimum bandwidth of 120 Mbps between cluster infrastructure and the public internet or private network locations that provide deployment artifacts and resources. When network connectivity is slower than 120 Mbps (for example, when connecting through a proxy) the cluster installation process times out and deployment fails.

After cluster deployment, network requirements are determined by your workload. However, a minimum bandwidth of 120 Mbps helps to ensure timely cluster and operator upgrades.