Configuring an AWS account
Before you can install OKD, you must configure an Amazon Web Services (AWS) account.
Configuring Route 53
To install OKD, the Amazon Web Services (AWS) account you use must have a dedicated public hosted zone in your Route 53 service. This zone must be authoritative for the domain. The Route 53 service provides cluster DNS resolution and name lookup for external connections to the cluster.
Procedure
Identify your domain, or subdomain, and registrar. You can transfer an existing domain and registrar or obtain a new one through AWS or another source.
If you purchase a new domain through AWS, it takes time for the relevant DNS changes to propagate. For more information about purchasing domains through AWS, see Registering Domain Names Using Amazon Route 53 in the AWS documentation.
If you are using an existing domain and registrar, migrate its DNS to AWS. See Making Amazon Route 53 the DNS Service for an Existing Domain in the AWS documentation.
Create a public hosted zone for your domain or subdomain. See Creating a Public Hosted Zone in the AWS documentation.
Use an appropriate root domain, such as
openshiftcorp.com
, or subdomain, such asclusters.openshiftcorp.com
.Extract the new authoritative name servers from the hosted zone records. See Getting the Name Servers for a Public Hosted Zone in the AWS documentation.
Update the registrar records for the AWS Route 53 name servers that your domain uses. For example, if you registered your domain to a Route 53 service in a different accounts, see the following topic in the AWS documentation: Adding or Changing Name Servers or Glue Records.
If you are using a subdomain, add its delegation records to the parent domain. This gives Amazon Route 53 responsibility for the subdomain. Follow the delegation procedure outlined by the DNS provider of the parent domain. See Creating a subdomain that uses Amazon Route 53 as the DNS service without migrating the parent domain in the AWS documentation for an example high level procedure.
Ingress Operator endpoint configuration for AWS Route 53
If you install in either Amazon Web Services (AWS) GovCloud (US) US-West or US-East region, the Ingress Operator uses us-gov-west-1
region for Route53 and tagging API clients.
The Ingress Operator uses [https://tagging.us-gov-west-1.amazonaws.com](https://tagging.us-gov-west-1.amazonaws.com)
as the tagging API endpoint if a tagging custom endpoint is configured that includes the string ‘us-gov-east-1’.
For more information on AWS GovCloud (US) endpoints, see the Service Endpoints in the AWS documentation about GovCloud (US).
Private, disconnected installations are not supported for AWS GovCloud when you install in the |
Example Route 53 configuration
platform:
aws:
region: us-gov-west-1
serviceEndpoints:
- name: ec2
url: https://ec2.us-gov-west-1.amazonaws.com
- name: elasticloadbalancing
url: https://elasticloadbalancing.us-gov-west-1.amazonaws.com
- name: route53
url: https://route53.us-gov.amazonaws.com (1)
- name: tagging
url: https://tagging.us-gov-west-1.amazonaws.com (2)
1 | Route 53 defaults to https://route53.us-gov.amazonaws.com for both AWS GovCloud (US) regions. |
2 | Only the US-West region has endpoints for tagging. Omit this parameter if your cluster is in another region. |
AWS account limits
The OKD cluster uses a number of Amazon Web Services (AWS) components, and the default Service Limits affect your ability to install OKD clusters. If you use certain cluster configurations, deploy your cluster in certain AWS regions, or run multiple clusters from your account, you might need to request additional resources for your AWS account.
The following table summarizes the AWS components whose limits can impact your ability to install and run OKD clusters.
Component | Number of clusters available by default | Default AWS limit | Description | ||
---|---|---|---|---|---|
Instance Limits | Varies | Varies | By default, each cluster creates the following instances:
These instance type counts are within a new account’s default limit. To deploy more worker nodes, enable autoscaling, deploy large workloads, or use a different instance type, review your account limits to ensure that your cluster can deploy the machines that you need. In most regions, the bootstrap and worker machines uses an | ||
Elastic IPs (EIPs) | 0 to 1 | 5 EIPs per account | To provision the cluster in a highly available configuration, the installation program creates a public and private subnet for each availability zone within a region. Each private subnet requires a NAT Gateway, and each NAT gateway requires a separate elastic IP. Review the AWS region map to determine how many availability zones are in each region. To take advantage of the default high availability, install the cluster in a region with at least three availability zones. To install a cluster in a region with more than five availability zones, you must increase the EIP limit.
| ||
Virtual Private Clouds (VPCs) | 5 | 5 VPCs per region | Each cluster creates its own VPC. | ||
Elastic Load Balancing (ELB/NLB) | 3 | 20 per region | By default, each cluster creates internal and external network load balancers for the master API server and a single classic elastic load balancer for the router. Deploying more Kubernetes | ||
NAT Gateways | 5 | 5 per availability zone | The cluster deploys one NAT gateway in each availability zone. | ||
Elastic Network Interfaces (ENIs) | At least 12 | 350 per region | The default installation creates 21 ENIs and an ENI for each availability zone in your region. For example, the Additional ENIs are created for additional machines and elastic load balancers that are created by cluster usage and deployed workloads. | ||
VPC Gateway | 20 | 20 per account | Each cluster creates a single VPC Gateway for S3 access. | ||
S3 buckets | 99 | 100 buckets per account | Because the installation process creates a temporary bucket and the registry component in each cluster creates a bucket, you can create only 99 OKD clusters per AWS account. | ||
Security Groups | 250 | 2,500 per account | Each cluster creates 10 distinct security groups. |
Required AWS permissions
When you attach the AdministratorAccess
policy to the IAM user that you create in Amazon Web Services (AWS), you grant that user all of the required permissions. To deploy all components of an OKD cluster, the IAM user requires the following permissions:
Required EC2 permissions for installation
tag:TagResources
tag:UntagResources
ec2:AllocateAddress
ec2:AssociateAddress
ec2:AuthorizeSecurityGroupEgress
ec2:AuthorizeSecurityGroupIngress
ec2:CopyImage
ec2:CreateNetworkInterface
ec2:AttachNetworkInterface
ec2:CreateSecurityGroup
ec2:CreateTags
ec2:CreateVolume
ec2:DeleteSecurityGroup
ec2:DeleteSnapshot
ec2:DeleteTags
ec2:DeregisterImage
ec2:DescribeAccountAttributes
ec2:DescribeAddresses
ec2:DescribeAvailabilityZones
ec2:DescribeDhcpOptions
ec2:DescribeImages
ec2:DescribeInstanceAttribute
ec2:DescribeInstanceCreditSpecifications
ec2:DescribeInstances
ec2:DescribeInternetGateways
ec2:DescribeKeyPairs
ec2:DescribeNatGateways
ec2:DescribeNetworkAcls
ec2:DescribeNetworkInterfaces
ec2:DescribePrefixLists
ec2:DescribeRegions
ec2:DescribeRouteTables
ec2:DescribeSecurityGroups
ec2:DescribeSubnets
ec2:DescribeTags
ec2:DescribeVolumes
ec2:DescribeVpcAttribute
ec2:DescribeVpcClassicLink
ec2:DescribeVpcClassicLinkDnsSupport
ec2:DescribeVpcEndpoints
ec2:DescribeVpcs
ec2:GetEbsDefaultKmsKeyId
ec2:ModifyInstanceAttribute
ec2:ModifyNetworkInterfaceAttribute
ec2:ReleaseAddress
ec2:RevokeSecurityGroupEgress
ec2:RevokeSecurityGroupIngress
ec2:RunInstances
ec2:TerminateInstances
Required permissions for creating network resources during installation
ec2:AssociateDhcpOptions
ec2:AssociateRouteTable
ec2:AttachInternetGateway
ec2:CreateDhcpOptions
ec2:CreateInternetGateway
ec2:CreateNatGateway
ec2:CreateRoute
ec2:CreateRouteTable
ec2:CreateSubnet
ec2:CreateVpc
ec2:CreateVpcEndpoint
ec2:ModifySubnetAttribute
ec2:ModifyVpcAttribute
If you use an existing VPC, your account does not require these permissions for creating network resources. |
Required Elastic Load Balancing permissions for installation
elasticloadbalancing:AddTags
elasticloadbalancing:ApplySecurityGroupsToLoadBalancer
elasticloadbalancing:AttachLoadBalancerToSubnets
elasticloadbalancing:ConfigureHealthCheck
elasticloadbalancing:CreateListener
elasticloadbalancing:CreateLoadBalancer
elasticloadbalancing:CreateLoadBalancerListeners
elasticloadbalancing:CreateTargetGroup
elasticloadbalancing:DeleteLoadBalancer
elasticloadbalancing:DeregisterInstancesFromLoadBalancer
elasticloadbalancing:DeregisterTargets
elasticloadbalancing:DescribeInstanceHealth
elasticloadbalancing:DescribeListeners
elasticloadbalancing:DescribeLoadBalancerAttributes
elasticloadbalancing:DescribeLoadBalancers
elasticloadbalancing:DescribeTags
elasticloadbalancing:DescribeTargetGroupAttributes
elasticloadbalancing:DescribeTargetHealth
elasticloadbalancing:ModifyLoadBalancerAttributes
elasticloadbalancing:ModifyTargetGroup
elasticloadbalancing:ModifyTargetGroupAttributes
elasticloadbalancing:RegisterInstancesWithLoadBalancer
elasticloadbalancing:RegisterTargets
elasticloadbalancing:SetLoadBalancerPoliciesOfListener
Required IAM permissions for installation
iam:AddRoleToInstanceProfile
iam:CreateInstanceProfile
iam:CreateRole
iam:DeleteInstanceProfile
iam:DeleteRole
iam:DeleteRolePolicy
iam:GetInstanceProfile
iam:GetRole
iam:GetRolePolicy
iam:GetUser
iam:ListInstanceProfilesForRole
iam:ListRoles
iam:ListUsers
iam:PassRole
iam:PutRolePolicy
iam:RemoveRoleFromInstanceProfile
iam:SimulatePrincipalPolicy
iam:TagRole
If you have not created an elastic load balancer (ELB) in your AWS account, the IAM user also requires the |
Required Route 53 permissions for installation
route53:ChangeResourceRecordSets
route53:ChangeTagsForResource
route53:CreateHostedZone
route53:DeleteHostedZone
route53:GetChange
route53:GetHostedZone
route53:ListHostedZones
route53:ListHostedZonesByName
route53:ListResourceRecordSets
route53:ListTagsForResource
route53:UpdateHostedZoneComment
Required S3 permissions for installation
s3:CreateBucket
s3:DeleteBucket
s3:GetAccelerateConfiguration
s3:GetBucketAcl
s3:GetBucketCors
s3:GetBucketLocation
s3:GetBucketLogging
s3:GetBucketObjectLockConfiguration
s3:GetBucketReplication
s3:GetBucketRequestPayment
s3:GetBucketTagging
s3:GetBucketVersioning
s3:GetBucketWebsite
s3:GetEncryptionConfiguration
s3:GetLifecycleConfiguration
s3:GetReplicationConfiguration
s3:ListBucket
s3:PutBucketAcl
s3:PutBucketTagging
s3:PutEncryptionConfiguration
S3 permissions that cluster Operators require
s3:DeleteObject
s3:GetObject
s3:GetObjectAcl
s3:GetObjectTagging
s3:GetObjectVersion
s3:PutObject
s3:PutObjectAcl
s3:PutObjectTagging
Required permissions to delete base cluster resources
autoscaling:DescribeAutoScalingGroups
ec2:DeleteNetworkInterface
ec2:DeleteVolume
elasticloadbalancing:DeleteTargetGroup
elasticloadbalancing:DescribeTargetGroups
iam:DeleteAccessKey
iam:DeleteUser
iam:ListAttachedRolePolicies
iam:ListInstanceProfiles
iam:ListRolePolicies
iam:ListUserPolicies
s3:DeleteObject
s3:ListBucketVersions
tag:GetResources
Required permissions to delete network resources
ec2:DeleteDhcpOptions
ec2:DeleteInternetGateway
ec2:DeleteNatGateway
ec2:DeleteRoute
ec2:DeleteRouteTable
ec2:DeleteSubnet
ec2:DeleteVpc
ec2:DeleteVpcEndpoints
ec2:DetachInternetGateway
ec2:DisassociateRouteTable
ec2:ReplaceRouteTableAssociation
If you use an existing VPC, your account does not require these permissions to delete network resources. |
Additional IAM and S3 permissions that are required to create manifests
iam:CreateAccessKey
iam:CreateUser
iam:DeleteAccessKey
iam:DeleteUser
iam:DeleteUserPolicy
iam:GetUserPolicy
iam:ListAccessKeys
iam:PutUserPolicy
iam:TagUser
iam:GetUserPolicy
iam:ListAccessKeys
s3:PutBucketPublicAccessBlock
s3:GetBucketPublicAccessBlock
s3:PutLifecycleConfiguration
s3:HeadBucket
s3:ListBucketMultipartUploads
s3:AbortMultipartUpload
Optional permission for quota checks for installation
servicequotas:ListAWSDefaultServiceQuotas
Creating an IAM user
Each Amazon Web Services (AWS) account contains a root user account that is based on the email address you used to create the account. This is a highly-privileged account, and it is recommended to use it for only initial account and billing configuration, creating an initial set of users, and securing the account.
Before you install OKD, create a secondary IAM administrative user. As you complete the Creating an IAM User in Your AWS Account procedure in the AWS documentation, set the following options:
Procedure
Specify the IAM user name and select
Programmatic access
.Attach the
AdministratorAccess
policy to ensure that the account has sufficient permission to create the cluster. This policy provides the cluster with the ability to grant credentials to each OKD component. The cluster grants the components only the credentials that they require.While it is possible to create a policy that grants the all of the required AWS permissions and attach it to the user, this is not the preferred option. The cluster will not have the ability to grant additional credentials to individual components, so the same credentials are used by all components.
Optional: Add metadata to the user by attaching tags.
Confirm that the user name that you specified is granted the
AdministratorAccess
policy.Record the access key ID and secret access key values. You must use these values when you configure your local machine to run the installation program.
You cannot use a temporary session token that you generated while using a multi-factor authentication device to authenticate to AWS when you deploy a cluster. The cluster continues to use your current AWS credentials to create AWS resources for the entire life of the cluster, so you must use key-based, long-lived credentials.
Additional resources
- See Manually creating IAM for AWS for steps to set the Cloud Credential Operator (CCO) to manual mode prior to installation. Use this mode in environments where the cloud identity and access management (IAM) APIs are not reachable, or if you prefer not to store an administrator-level credential secret in the cluster
kube-system
project.
Supported AWS regions
You can deploy an OKD cluster to the following public regions:
af-south-1
(Cape Town)ap-east-1
(Hong Kong)ap-northeast-1
(Tokyo)ap-northeast-2
(Seoul)ap-south-1
(Mumbai)ap-southeast-1
(Singapore)ap-southeast-2
(Sydney)ca-central-1
(Central)eu-central-1
(Frankfurt)eu-north-1
(Stockholm)eu-south-1
(Milan)eu-west-1
(Ireland)eu-west-2
(London)eu-west-3
(Paris)me-south-1
(Bahrain)sa-east-1
(São Paulo)us-east-1
(N. Virginia)us-east-2
(Ohio)us-west-1
(N. California)us-west-2
(Oregon)
The following AWS GovCloud regions are supported:
us-gov-west-1
us-gov-east-1
Next steps
Install an OKD cluster:
Quickly install a cluster with default options on installer-provisioned infrastructure
Install a cluster with cloud customizations on installer-provisioned infrastructure
Install a cluster with network customizations on installer-provisioned infrastructure
Installing a cluster on user-provisioned infrastructure in AWS by using CloudFormation templates