Installing Kubernetes Addons

With kops you manage addons by using kubectl.

(For a description of the addon-manager, please see addon_manager.md.)

Addons in Kubernetes are traditionally done by copying files to /etc/kubernetes/addons on the master. But thisdoesn’t really make sense in HA master configurations. We also have kubectl available, and addons are just a thinwrapper over calling kubectl.

The command kops create cluster does not support specifying addons to be added to the cluster when it is created. Instead they can be added after cluster creation using kubectl. Alternatively when creating a cluster from a yaml manifest, addons can be specified using spec.addons.

  1. spec:
  2.   addons:
  3.   - kubernetes-dashboard
  4.   - s3://kops-addons/addon.yaml

This document describes how to install some common addons and how to create your own custom ones.

Custom addons

The docs about the addon manager describe in more detail how to define a addon resource with regards to versioning.Here is a minimal example of an addon manifest that would install two different addons.

  1. kind: Addons
  2. metadata:
  3.   name: example
  4. spec:
  5.   addons:
  6.   - name: foo.addons.org.io
  7.     version: 0.0.1
  8.     selector:
  9.       k8s-addon: foo.addons.org.io
  10.     manifest: foo.addons.org.io/v0.0.1.yaml
  11.   - name: bar.addons.org.io
  12.     version: 0.0.1
  13.     selector:
  14.       k8s-addon: bar.addons.org.io
  15.     manifest: bar.addons.org.io/v0.0.1.yaml

In this this example the folder structure should look like this;

  1. addon.yaml
  2.   foo.addons.org.io
  3.     v0.0.1.yaml
  4.   bar.addons.org.io
  5.     v0.0.1.yaml

The yaml files in the foo/bar folders can be any kubernetes resource. Typically this file structure would be pushed to S3 or another of the supported backends and then referenced as above in spec.addons. In order for master nodes to be able to access the S3 bucket containing the addon manifests, one might have to add additional iam policies to the master nodes using spec.additionalPolicies, like so;

  1. spec:
  2.   additionalPolicies:
  3.     master: |
  4.       [
  5.         {
  6.           "Effect": "Allow",
  7.           "Action": [
  8.             "s3:GetObject"
  9.           ],
  10.           "Resource": ["arn:aws:s3:::kops-addons/*"]
  11.         },
  12.         {
  13.           "Effect": "Allow",
  14.           "Action": [
  15.             "s3:GetBucketLocation",
  16.             "s3:ListBucket"
  17.           ],
  18.           "Resource": ["arn:aws:s3:::kops-addons"]
  19.         }
  20.       ]

The masters will poll for changes changes in the bucket and keep the addons up to date.

Dashboard

The dashboard project provides a nice administrative UI:

Install using:

  1. kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/kubernetes-dashboard/v1.8.3.yaml

And then navigate to https://api.<clustername>/ui

(/ui is an alias to https://<clustername>/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard)

The login credentials are:

  • Username: admin
  • Password: get by running kops get secrets kube --type secret -oplaintext or kubectl config view --minify

RBAC

For k8s version > 1.6 and RBAC enabled it’s necessary to add your own permission to the dashboard. Please read the RBAC docs before applying permissions.

Below you see an example giving full access to the dashboard.

  1. apiVersion: rbac.authorization.k8s.io/v1beta1
  2. kind: ClusterRoleBinding
  3. metadata:
  4.   name: kubernetes-dashboard
  5.   labels:
  6.     k8s-app: kubernetes-dashboard
  7. roleRef:
  8.   apiGroup: rbac.authorization.k8s.io
  9.   kind: ClusterRole
  10.   name: cluster-admin
  11. subjects:
  12. - kind: ServiceAccount
  13.   name: kubernetes-dashboard
  14.   namespace: kube-system

Monitoring with Heapster - Standalone

Monitoring supports the horizontal pod autoscaler.

Install using:

  1. kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/monitoring-standalone/v1.11.0.yaml

Please note that heapster is deprecated. Consider using metrics-server and a third party metrics pipeline to gather Prometheus-format metrics instead.

Monitoring with Prometheus Operator + kube-prometheus

The Prometheus Operator makes the Prometheus configuration Kubernetes native and manages and operates Prometheus and Alertmanager clusters. It is a piece of the puzzle regarding full end-to-end monitoring.

kube-prometheus combines the Prometheus Operator with a collection of manifests to help getting started with monitoring Kubernetes itself and applications running on top of it.

  1. kubectl apply -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/prometheus-operator/v0.19.0.yaml

Route53 Mapper

Please note that kops installs a Route53 DNS controller automatically (it is required for cluster discovery).The functionality of the route53-mapper overlaps with the dns-controller, but some users will prefer touse one or the other.README for the included dns-controller

route53-mapper automates creation and updating of entries on Route53 with A records pointingto ELB-backed LoadBalancer services created by Kubernetes. Install using:

The project is created by wearemolecule, and maintained atwearemolecule/route53-kubernetes.Usage instructions

  1. kubectl apply -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/route53-mapper/v1.3.0.yml