Setting up the Amazon Cloud Provider

When using the Amazon cloud provider, you can leverage the following capabilities:

  • Load Balancers: Launches an AWS Elastic Load Balancer (ELB) when choosing Layer-4 Load Balancer in Port Mapping or when launching a Service with type: LoadBalancer.
  • Persistent Volumes: Allows you to use AWS Elastic Block Stores (EBS) for persistent volumes.

See cloud-provider-aws README for all information regarding the Amazon cloud provider.

To set up the Amazon cloud provider,

  1. Create an IAM role and attach to the instances
  2. Configure the ClusterID

1. Create an IAM Role and attach to the instances

All nodes added to the cluster must be able to interact with EC2 so that they can create and remove resources. You can enable this interaction by using an IAM role attached to the instance. See Amazon documentation: Creating an IAM Role how to create an IAM role. There are two example policies:

  • The first policy is for the nodes with the controlplane role. These nodes have to be able to create/remove EC2 resources. The following IAM policy is an example, please remove any unneeded permissions for your use case.
  • The second policy is for the nodes with the etcd or worker role. These nodes only have to be able to retrieve information from EC2.

While creating an Amazon EC2 cluster, you must fill in the IAM Instance Profile Name (not ARN) of the created IAM role when creating the Node Template.

While creating a Custom cluster, you must manually attach the IAM role to the instance(s).

IAM Policy for nodes with the controlplane role:

  1. {
  2. "Version": "2012-10-17",
  3. "Statement": [
  4.   {
  5.     "Effect": "Allow",
  6.     "Action": [
  7.       "autoscaling:DescribeAutoScalingGroups",
  8.       "autoscaling:DescribeLaunchConfigurations",
  9.       "autoscaling:DescribeTags",
  10.       "ec2:DescribeInstances",
  11.       "ec2:DescribeRegions",
  12.       "ec2:DescribeRouteTables",
  13.       "ec2:DescribeSecurityGroups",
  14.       "ec2:DescribeSubnets",
  15.       "ec2:DescribeVolumes",
  16.       "ec2:CreateSecurityGroup",
  17.       "ec2:CreateTags",
  18.       "ec2:CreateVolume",
  19.       "ec2:ModifyInstanceAttribute",
  20.       "ec2:ModifyVolume",
  21.       "ec2:AttachVolume",
  22.       "ec2:AuthorizeSecurityGroupIngress",
  23.       "ec2:CreateRoute",
  24.       "ec2:DeleteRoute",
  25.       "ec2:DeleteSecurityGroup",
  26.       "ec2:DeleteVolume",
  27.       "ec2:DetachVolume",
  28.       "ec2:RevokeSecurityGroupIngress",
  29.       "ec2:DescribeVpcs",
  30.       "elasticloadbalancing:AddTags",
  31.       "elasticloadbalancing:AttachLoadBalancerToSubnets",
  32.       "elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
  33.       "elasticloadbalancing:CreateLoadBalancer",
  34.       "elasticloadbalancing:CreateLoadBalancerPolicy",
  35.       "elasticloadbalancing:CreateLoadBalancerListeners",
  36.       "elasticloadbalancing:ConfigureHealthCheck",
  37.       "elasticloadbalancing:DeleteLoadBalancer",
  38.       "elasticloadbalancing:DeleteLoadBalancerListeners",
  39.       "elasticloadbalancing:DescribeLoadBalancers",
  40.       "elasticloadbalancing:DescribeLoadBalancerAttributes",
  41.       "elasticloadbalancing:DetachLoadBalancerFromSubnets",
  42.       "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
  43.       "elasticloadbalancing:ModifyLoadBalancerAttributes",
  44.       "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
  45.       "elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",
  46.       "elasticloadbalancing:AddTags",
  47.       "elasticloadbalancing:CreateListener",
  48.       "elasticloadbalancing:CreateTargetGroup",
  49.       "elasticloadbalancing:DeleteListener",
  50.       "elasticloadbalancing:DeleteTargetGroup",
  51.       "elasticloadbalancing:DescribeListeners",
  52.       "elasticloadbalancing:DescribeLoadBalancerPolicies",
  53.       "elasticloadbalancing:DescribeTargetGroups",
  54.       "elasticloadbalancing:DescribeTargetHealth",
  55.       "elasticloadbalancing:ModifyListener",
  56.       "elasticloadbalancing:ModifyTargetGroup",
  57.       "elasticloadbalancing:RegisterTargets",
  58.       "elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
  59.       "iam:CreateServiceLinkedRole",
  60.       "kms:DescribeKey"
  61.     ],
  62.     "Resource": [
  63.       "*"
  64.     ]
  65.   }
  66. ]
  67. }

IAM policy for nodes with the etcd or worker role:

  1. {
  2. "Version": "2012-10-17",
  3. "Statement": [
  4.     {
  5.         "Effect": "Allow",
  6.         "Action": [
  7.             "ec2:DescribeInstances",
  8.             "ec2:DescribeRegions",
  9.             "ecr:GetAuthorizationToken",
  10.             "ecr:BatchCheckLayerAvailability",
  11.             "ecr:GetDownloadUrlForLayer",
  12.             "ecr:GetRepositoryPolicy",
  13.             "ecr:DescribeRepositories",
  14.             "ecr:ListImages",
  15.             "ecr:BatchGetImage"
  16.         ],
  17.         "Resource": "*"
  18.     }
  19. ]
  20. }

2. Configure the ClusterID

The following resources need to tagged with a ClusterID:

  • Nodes: All hosts added in Rancher.
  • Subnet: The subnet used for your cluster.
  • Security Group: The security group used for your cluster.

Note: Do not tag multiple security groups. Tagging multiple groups generates an error when creating an Elastic Load Balancer (ELB).

When you create an Amazon EC2 Cluster, the ClusterID is automatically configured for the created nodes. Other resources still need to be tagged manually.

Use the following tag:

Key = kubernetes.io/cluster/CLUSTERID Value = owned

CLUSTERID can be any string you like, as long as it is equal across all tags set.

Setting the value of the tag to owned tells the cluster that all resources with this tag are owned and managed by this cluster. If you share resources between clusters, you can change the tag to:

Key = kubernetes.io/cluster/CLUSTERID Value = shared.

Using Amazon Elastic Container Registry (ECR)

The kubelet component has the ability to automatically obtain ECR credentials, when the IAM profile mentioned in Create an IAM Role and attach to the instances is attached to the instance(s). When using a Kubernetes version older than v1.15.0, the Amazon cloud provider needs be configured in the cluster. Starting with Kubernetes version v1.15.0, the kubelet can obtain ECR credentials without having the Amazon cloud provider configured in the cluster.