Configuring an AWS account

Before you can install OKD, you must configure an Amazon Web Services (AWS) account.

Configuring Route 53

To install OKD, the Amazon Web Services (AWS) account you use must have a dedicated public hosted zone in your Route 53 service. This zone must be authoritative for the domain. The Route 53 service provides cluster DNS resolution and name lookup for external connections to the cluster.

Procedure

1. Identify your domain, or subdomain, and registrar. You can transfer an existing domain and registrar or obtain a new one through AWS or another source.

   If you purchase a new domain through AWS, it takes time for the relevant DNS changes to propagate. For more information about purchasing domains through AWS, see Registering Domain Names Using Amazon Route 53 in the AWS documentation.

2. If you are using an existing domain and registrar, migrate its DNS to AWS. See Making Amazon Route 53 the DNS Service for an Existing Domain in the AWS documentation.

3. Create a public hosted zone for your domain or subdomain. See Creating a Public Hosted Zone in the AWS documentation.

   Use an appropriate root domain, such as openshiftcorp.com, or subdomain, such as clusters.openshiftcorp.com.

4. Extract the new authoritative name servers from the hosted zone records. See Getting the Name Servers for a Public Hosted Zone in the AWS documentation.

5. Update the registrar records for the AWS Route 53 name servers that your domain uses. For example, if you registered your domain to a Route 53 service in a different accounts, see the following topic in the AWS documentation: Adding or Changing Name Servers or Glue Records.

6. If you are using a subdomain, add its delegation records to the parent domain. This gives Amazon Route 53 responsibility for the subdomain. Follow the delegation procedure outlined by the DNS provider of the parent domain. See Creating a subdomain that uses Amazon Route 53 as the DNS service without migrating the parent domain in the AWS documentation for an example high level procedure.

Ingress Operator endpoint configuration for AWS Route 53

If you install in either Amazon Web Services (AWS) GovCloud (US) US-West or US-East region, the Ingress Operator uses us-gov-west-1 region for Route53 and tagging API clients.

The Ingress Operator uses [https://tagging.us-gov-west-1.amazonaws.com](https://tagging.us-gov-west-1.amazonaws.com) as the tagging API endpoint if a tagging custom endpoint is configured that includes the string ‘us-gov-east-1’.

For more information on AWS GovCloud (US) endpoints, see the Service Endpoints in the AWS documentation about GovCloud (US).

Example Route 53 configuration

platform:
  aws:
    region: us-gov-west-1
    serviceEndpoints:
    - name: ec2
      url: https://ec2.us-gov-west-1.amazonaws.com
    - name: elasticloadbalancing
      url: https://elasticloadbalancing.us-gov-west-1.amazonaws.com
    - name: route53
      url: https://route53.us-gov.amazonaws.com (1)
    - name: tagging
      url: https://tagging.us-gov-west-1.amazonaws.com (2)

AWS account limits

The OKD cluster uses a number of Amazon Web Services (AWS) components, and the default Service Limits affect your ability to install OKD clusters. If you use certain cluster configurations, deploy your cluster in certain AWS regions, or run multiple clusters from your account, you might need to request additional resources for your AWS account.

The following table summarizes the AWS components whose limits can impact your ability to install and run OKD clusters.

Required AWS permissions for the IAM user

When you attach the AdministratorAccess policy to the IAM user that you create in Amazon Web Services (AWS), you grant that user all of the required permissions. To deploy all components of an OKD cluster, the IAM user requires the following permissions:

Required EC2 permissions for installation

• ec2:AuthorizeSecurityGroupEgress

• ec2:AuthorizeSecurityGroupIngress

• ec2:CopyImage

• ec2:CreateNetworkInterface

• ec2:AttachNetworkInterface

• ec2:CreateSecurityGroup

• ec2:CreateTags

• ec2:CreateVolume

• ec2:DeleteSecurityGroup

• ec2:DeleteSnapshot

• ec2:DeleteTags

• ec2:DeregisterImage

• ec2:DescribeAccountAttributes

• ec2:DescribeAddresses

• ec2:DescribeAvailabilityZones

• ec2:DescribeDhcpOptions

• ec2:DescribeImages

• ec2:DescribeInstanceAttribute

• ec2:DescribeInstanceCreditSpecifications

• ec2:DescribeInstances

• ec2:DescribeInstanceTypes

• ec2:DescribeInternetGateways

• ec2:DescribeKeyPairs

• ec2:DescribeNatGateways

• ec2:DescribeNetworkAcls

• ec2:DescribeNetworkInterfaces

• ec2:DescribePrefixLists

• ec2:DescribeRegions

• ec2:DescribeRouteTables

• ec2:DescribeSecurityGroups

• ec2:DescribeSubnets

• ec2:DescribeTags

• ec2:DescribeVolumes

• ec2:DescribeVpcAttribute

• ec2:DescribeVpcClassicLink

• ec2:DescribeVpcClassicLinkDnsSupport

• ec2:DescribeVpcEndpoints

• ec2:DescribeVpcs

• ec2:GetEbsDefaultKmsKeyId

• ec2:ModifyInstanceAttribute

• ec2:ModifyNetworkInterfaceAttribute

• ec2:RevokeSecurityGroupEgress

• ec2:RevokeSecurityGroupIngress

• ec2:RunInstances

• ec2:TerminateInstances

Required permissions for creating network resources during installation

• ec2:AllocateAddress

• ec2:AssociateAddress

• ec2:AssociateDhcpOptions

• ec2:AssociateRouteTable

• ec2:AttachInternetGateway

• ec2:CreateDhcpOptions

• ec2:CreateInternetGateway

• ec2:CreateNatGateway

• ec2:CreateRoute

• ec2:CreateRouteTable

• ec2:CreateSubnet

• ec2:CreateVpc

• ec2:CreateVpcEndpoint

• ec2:ModifySubnetAttribute

• ec2:ModifyVpcAttribute

Required Elastic Load Balancing permissions (ELB) for installation

• elasticloadbalancing:AddTags

• elasticloadbalancing:ApplySecurityGroupsToLoadBalancer

• elasticloadbalancing:AttachLoadBalancerToSubnets

• elasticloadbalancing:ConfigureHealthCheck

• elasticloadbalancing:CreateLoadBalancer

• elasticloadbalancing:CreateLoadBalancerListeners

• elasticloadbalancing:DeleteLoadBalancer

• elasticloadbalancing:DeregisterInstancesFromLoadBalancer

• elasticloadbalancing:DescribeInstanceHealth

• elasticloadbalancing:DescribeLoadBalancerAttributes

• elasticloadbalancing:DescribeLoadBalancers

• elasticloadbalancing:DescribeTags

• elasticloadbalancing:ModifyLoadBalancerAttributes

• elasticloadbalancing:RegisterInstancesWithLoadBalancer

• elasticloadbalancing:SetLoadBalancerPoliciesOfListener

Required Elastic Load Balancing permissions (ELBv2) for installation

• elasticloadbalancing:AddTags

• elasticloadbalancing:CreateListener

• elasticloadbalancing:CreateLoadBalancer

• elasticloadbalancing:CreateTargetGroup

• elasticloadbalancing:DeleteLoadBalancer

• elasticloadbalancing:DeregisterTargets

• elasticloadbalancing:DescribeListeners

• elasticloadbalancing:DescribeLoadBalancerAttributes

• elasticloadbalancing:DescribeLoadBalancers

• elasticloadbalancing:DescribeTargetGroupAttributes

• elasticloadbalancing:DescribeTargetHealth

• elasticloadbalancing:ModifyLoadBalancerAttributes

• elasticloadbalancing:ModifyTargetGroup

• elasticloadbalancing:ModifyTargetGroupAttributes

• elasticloadbalancing:RegisterTargets

Required IAM permissions for installation

• iam:AddRoleToInstanceProfile

• iam:CreateInstanceProfile

• iam:CreateRole

• iam:DeleteInstanceProfile

• iam:DeleteRole

• iam:DeleteRolePolicy

• iam:GetInstanceProfile

• iam:GetRole

• iam:GetRolePolicy

• iam:GetUser

• iam:ListInstanceProfilesForRole

• iam:ListRoles

• iam:ListUsers

• iam:PassRole

• iam:PutRolePolicy

• iam:RemoveRoleFromInstanceProfile

• iam:SimulatePrincipalPolicy

• iam:TagRole

Required Route 53 permissions for installation

• route53:ChangeResourceRecordSets

• route53:ChangeTagsForResource

• route53:CreateHostedZone

• route53:DeleteHostedZone

• route53:GetChange

• route53:GetHostedZone

• route53:ListHostedZones

• route53:ListHostedZonesByName

• route53:ListResourceRecordSets

• route53:ListTagsForResource

• route53:UpdateHostedZoneComment

Required S3 permissions for installation

• s3:CreateBucket

• s3:DeleteBucket

• s3:GetAccelerateConfiguration

• s3:GetBucketAcl

• s3:GetBucketCors

• s3:GetBucketLocation

• s3:GetBucketLogging

• s3:GetBucketPolicy

• s3:GetBucketObjectLockConfiguration

• s3:GetBucketReplication

• s3:GetBucketRequestPayment

• s3:GetBucketTagging

• s3:GetBucketVersioning

• s3:GetBucketWebsite

• s3:GetEncryptionConfiguration

• s3:GetLifecycleConfiguration

• s3:GetReplicationConfiguration

• s3:ListBucket

• s3:PutBucketAcl

• s3:PutBucketTagging

• s3:PutEncryptionConfiguration

S3 permissions that cluster Operators require

• s3:DeleteObject

• s3:GetObject

• s3:GetObjectAcl

• s3:GetObjectTagging

• s3:GetObjectVersion

• s3:PutObject

• s3:PutObjectAcl

• s3:PutObjectTagging

Required permissions to delete base cluster resources

• autoscaling:DescribeAutoScalingGroups

• ec2:DeletePlacementGroup

• ec2:DeleteNetworkInterface

• ec2:DeleteVolume

• elasticloadbalancing:DeleteTargetGroup

• elasticloadbalancing:DescribeTargetGroups

• iam:DeleteAccessKey

• iam:DeleteUser

• iam:ListAttachedRolePolicies

• iam:ListInstanceProfiles

• iam:ListRolePolicies

• iam:ListUserPolicies

• s3:DeleteObject

• s3:ListBucketVersions

• tag:GetResources

Required permissions to delete network resources

• ec2:DeleteDhcpOptions

• ec2:DeleteInternetGateway

• ec2:DeleteNatGateway

• ec2:DeleteRoute

• ec2:DeleteRouteTable

• ec2:DeleteSubnet

• ec2:DeleteVpc

• ec2:DeleteVpcEndpoints

• ec2:DetachInternetGateway

• ec2:DisassociateRouteTable

• ec2:ReleaseAddress

• ec2:ReplaceRouteTableAssociation

Required permissions to delete a cluster with shared instance roles

• iam:UntagRole

Additional IAM and S3 permissions that are required to create manifests

• iam:DeleteAccessKey

• iam:DeleteUser

• iam:DeleteUserPolicy

• iam:GetUserPolicy

• iam:ListAccessKeys

• iam:PutUserPolicy

• iam:TagUser

• s3:PutBucketPublicAccessBlock

• s3:GetBucketPublicAccessBlock

• s3:PutLifecycleConfiguration

• s3:HeadBucket

• s3:ListBucketMultipartUploads

• s3:AbortMultipartUpload

Optional permissions for instance and quota checks for installation

• ec2:DescribeInstanceTypeOfferings

• servicequotas:ListAWSDefaultServiceQuotas

Creating an IAM user

Each Amazon Web Services (AWS) account contains a root user account that is based on the email address you used to create the account. This is a highly-privileged account, and it is recommended to use it for only initial account and billing configuration, creating an initial set of users, and securing the account.

Before you install OKD, create a secondary IAM administrative user. As you complete the Creating an IAM User in Your AWS Account procedure in the AWS documentation, set the following options:

Procedure

1. Specify the IAM user name and select Programmatic access.

2. Attach the AdministratorAccess policy to ensure that the account has sufficient permission to create the cluster. This policy provides the cluster with the ability to grant credentials to each OKD component. The cluster grants the components only the credentials that they require.

   While it is possible to create a policy that grants the all of the required AWS permissions and attach it to the user, this is not the preferred option. The cluster will not have the ability to grant additional credentials to individual components, so the same credentials are used by all components.

3. Optional: Add metadata to the user by attaching tags.

4. Confirm that the user name that you specified is granted the AdministratorAccess policy.

5. Record the access key ID and secret access key values. You must use these values when you configure your local machine to run the installation program.

   You cannot use a temporary session token that you generated while using a multi-factor authentication device to authenticate to AWS when you deploy a cluster. The cluster continues to use your current AWS credentials to create AWS resources for the entire life of the cluster, so you must use key-based, long-lived credentials.

Additional resources

• See Manually creating IAM for AWS for steps to set the Cloud Credential Operator (CCO) to manual mode prior to installation. Use this mode in environments where the cloud identity and access management (IAM) APIs are not reachable, or if you prefer not to store an administrator-level credential secret in the cluster kube-system project.

IAM Policies and AWS authentication

By default, the installation program creates instance profiles for the bootstrap, control plane, and compute instances with the necessary permissions for the cluster to operate.

However, you can create your own IAM roles and specify them as part of the installation process. You might need to specify your own roles to deploy the cluster or to manage the cluster after installation. For example:

• Your organization’s security policies require that you use a more restrictive set of permissions to install the cluster.

• After the installation, the cluster is configured with an Operator that requires access to additional services.

If you choose to specify your own IAM roles, you can take the following steps:

• Begin with the default policies and adapt as required. For more information, see “Default permissions for IAM instance profiles”.

• Use the AWS Identity and Access Management Access Analyzer (IAM Access Analyzer) to create a policy template that is based on the cluster’s activity. For more information see, “Using AWS IAM Analyzer to create policy templates”.

Default permissions for IAM instance profiles

By default, the installation program creates IAM instance profiles for the bootstrap, control plane and worker instances with the necessary permissions for the cluster to operate.

The following lists specify the default permissions for control plane and compute machines:

Default IAM role permissions for control plane instance profiles

• ec2:AttachVolume

• ec2:AuthorizeSecurityGroupIngress

• ec2:CreateSecurityGroup

• ec2:CreateTags

• ec2:CreateVolume

• ec2:DeleteSecurityGroup

• ec2:DeleteVolume

• ec2:Describe*

• ec2:DetachVolume

• ec2:ModifyInstanceAttribute

• ec2:ModifyVolume

• ec2:RevokeSecurityGroupIngress

• elasticloadbalancing:AddTags

• elasticloadbalancing:AttachLoadBalancerToSubnets

• elasticloadbalancing:ApplySecurityGroupsToLoadBalancer

• elasticloadbalancing:CreateListener

• elasticloadbalancing:CreateLoadBalancer

• elasticloadbalancing:CreateLoadBalancerPolicy

• elasticloadbalancing:CreateLoadBalancerListeners

• elasticloadbalancing:CreateTargetGroup

• elasticloadbalancing:ConfigureHealthCheck

• elasticloadbalancing:DeleteListener

• elasticloadbalancing:DeleteLoadBalancer

• elasticloadbalancing:DeleteLoadBalancerListeners

• elasticloadbalancing:DeleteTargetGroup

• elasticloadbalancing:DeregisterInstancesFromLoadBalancer

• elasticloadbalancing:DeregisterTargets

• elasticloadbalancing:Describe*

• elasticloadbalancing:DetachLoadBalancerFromSubnets

• elasticloadbalancing:ModifyListener

• elasticloadbalancing:ModifyLoadBalancerAttributes

• elasticloadbalancing:ModifyTargetGroup

• elasticloadbalancing:ModifyTargetGroupAttributes

• elasticloadbalancing:RegisterInstancesWithLoadBalancer

• elasticloadbalancing:RegisterTargets

• elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer

• elasticloadbalancing:SetLoadBalancerPoliciesOfListener

• kms:DescribeKey

Default IAM role permissions for compute instance profiles

• ec2:DescribeInstances

• ec2:DescribeRegions

Specifying an existing IAM role

Instead of allowing the installation program to create IAM instance profiles with the default permissions, you can use the install-config.yaml file to specify an existing IAM role for control plane and compute instances.

Prerequisites

• You have an existing install-config.yaml file.

Procedure

1. Update compute.platform.aws.iamRole with an existing role for the control plane machines.

   Sample install-config.yaml file with an IAM role for compute instances

   compute:
   - hyperthreading: Enabled
     name: worker
     platform:
       aws:
         iamRole: ExampleRole

2. Update controlPlane.platform.aws.iamRole with an existing role for the compute machines.

   Sample install-config.yaml file with an IAM role for control plane instances

   controlPlane:
     hyperthreading: Enabled
     name: master
     platform:
       aws:
         iamRole: ExampleRole

3. Save the file and reference it when installing the OKD cluster.

Additional resources

• See Deploying the cluster.

Using AWS IAM Analyzer to create policy templates

The minimal set of permissions that the control plane and compute instance profiles require depends on how the cluster is configured for its daily operation.

One way to determine which permissions the cluster instances require is to use the AWS Identity and Access Management Access Analyzer (IAM Access Analyzer) to create a policy template:

• A policy template contains the permissions the cluster has used over a specified period of time.

• You can then use the template to create policies with fine-grained permissions.

Procedure

The overall process could be:

1. Ensure that CloudTrail is enabled. CloudTrail records all of the actions and events in your AWS account, including the API calls that are required to create a policy template. For more information, see the AWS documentation for working with CloudTrail.

2. Create an instance profile for control plane instances and an instance profile for compute instances. Be sure to assign each role a permissive policy, such as PowerUserAccess. For more information, see the AWS documentation for creating instance profile roles.

3. Install the cluster in a development environment and configure it as required. Be sure to deploy all of applications the cluster will host in a production environment.

4. Test the cluster thoroughly. Testing the cluster ensures that all of the required API calls are logged.

5. Use the IAM Access Analyzer to create a policy template for each instance profile. For more information, see the AWS documentation for generating policies based on the CloudTrail logs.

6. Create and add a fine-grained policy to each instance profile.

7. Remove the permissive policy from each instance profile.

8. Deploy a production cluster using the existing instance profiles with the new policies.

Supported AWS Marketplace regions

Installing an OKD cluster using an AWS Marketplace image is available to customers who purchase the offer in North America.

While the offer must be purchased in North America, you can deploy the cluster to any of the following supported paritions:

• Public

• GovCloud

Supported AWS regions

You can deploy an OKD cluster to the following regions.

AWS public regions

The following AWS public regions are supported:

• af-south-1 (Cape Town)

• ap-east-1 (Hong Kong)

• ap-northeast-1 (Tokyo)

• ap-northeast-2 (Seoul)

• ap-northeast-3 (Osaka)

• ap-south-1 (Mumbai)

• ap-southeast-1 (Singapore)

• ap-southeast-2 (Sydney)

• ap-southeast-3 (Jakarta)

• ca-central-1 (Central)

• eu-central-1 (Frankfurt)

• eu-north-1 (Stockholm)

• eu-south-1 (Milan)

• eu-west-1 (Ireland)

• eu-west-2 (London)

• eu-west-3 (Paris)

• me-south-1 (Bahrain)

• sa-east-1 (São Paulo)

• us-east-1 (N. Virginia)

• us-east-2 (Ohio)

• us-west-1 (N. California)

• us-west-2 (Oregon)

AWS GovCloud regions

The following AWS GovCloud regions are supported:

• us-gov-west-1

• us-gov-east-1

AWS SC2S and C2S secret regions

The following AWS secret regions are supported:

• us-isob-east-1 Secret Commercial Cloud Services (SC2S)

• us-iso-east-1 Commercial Cloud Services (C2S)

AWS China regions

The following AWS China regions are supported:

• cn-north-1 (Beijing)

• cn-northwest-1 (Ningxia)

Next steps

• Install an OKD cluster:

  • Quickly install a cluster with default options on installer-provisioned infrastructure

  • Install a cluster with cloud customizations on installer-provisioned infrastructure

  • Install a cluster with network customizations on installer-provisioned infrastructure

  • Installing a cluster on user-provisioned infrastructure in AWS by using CloudFormation templates