Significant changes

  • flannel now has a backend property in the manifest, which can be either udp or vxlan. udp is not recommended, but will be the default value for existing clusters or clusters created via manifests. kops create cluster with —networking flannel will use vxlan, —networking flannel-vxlan or —networking flannel-udp can be specified to explicitly choose a backend mode.

  • IAM lockdown on new clusters: we define the existing policy as legacy, it defaults to true for existing clusters; new clusters will have legacy: false which will mean that only IAM policies needed by kops / k8s are guaranteed to be set. If you are using IAM credentials for your application workload, please either set legacy: true, or use your own IAM roles (direct credentials or kube2iam)

  • New AWS instance types: P3, C5, M5, H1. Please note that NVME volumes are not supported on the default jessie image, so masters will not boot on M5 and C5 instance types unless a stretch image is chosen (change jessie to stretch in the image name). Also note that kubernetes will not support mounting persistent volumes on NVME instances until Kubernetes v1.9.

  • While Aggregated API Servers are supported, there are known issues in kubernetes such as (#55022)[https://github.com/kubernetes/kubernetes/issues/55022]. Note that this includes metrics-server and kopeio authentication. Please consider waiting for 1.8.5 / 1.9.0 before deploying into production.

  • Includes fix for kube-dns CVE-2017-14491 (was also included in kops 1.7.1)

Required Actions

  • Existing Calico users on clusters that were created prior to kops 1.8.0 need to be updated for the new "DefaultDeny" behavior for Kubernetes NetworkPolicies. See the Changes to k8s-policy section in the Calico release notes for help.
  • Due to ThirdPartyResources becoming fully deprecated in Kubernetes v1.8 (replaced by CustomResourceDefinitions), existing Canal users upgrading their Clusters to Kubernetes v1.8 must follow the below TPR->CRD migration steps:
  • Run: kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v2.6.2/upgrade/v2.5/manifests/upgrade-job.yaml
  • Retrieve the pod name from describing the job: kubectl describe job/calico-upgrade-v2.5
  • Validate the last log line from the pod reports that it completed successfully: kubectl logs calico-upgrade-v2.5-<random-id>
  • Update the KubernetesVersion within your ClusterSpec to v1.8 (or above), performing an update & rolling-update to all nodes (will involve downtime)
  • Confirm cluster is back up and all canal pods are running successfully: kops validate cluster (this may take a few minutes for the cluster to fully validate)
  • Delete the upgrade job as it is no longer required: kubectl delete job calico-upgrade-v2.5 (you can also safely delete the clusterrole, clusterrolebinding and serviceaccount resources that were created by the above manifest file)

Highlighted changes

  • Support for etcd3 for new clusters, also allow etcd TLS to be enabled for new clusters. etcd peer port is also locked down.

  • Support for custom metrics. Please exercise caution enabling before kubernetes 1.8.5 due to aggregation known issues.

  • Add kops create secret dockerconfig

  • kops replace —force will now replace-or-create, which is useful for CI / automated workflows

  • —watch-ingress flag on dns-controller can now be configured through cluster.spec.externalDns.watchIngress: true

  • kubelet security can be enabled with cluster.spec.kubelet.anonymousAuth: true. Will likely be default in kops 1.9

  • Improved logic around when a rolling-update is needed

  • Better support and documentation for node resources

  • Enhanced cluster hooks support

  • Support for clusters where network access must use an HTTP proxy

  • We now automatically add a default NodeLabel with the InstanceGroup name

  • Addons: added external-dns, kube-state-metrics addon. Updates for autoscaler, dashboard, heapster,

  • Networking: initial support for kube-router & romana. Updates for weave, kopeio-networking, flannel, canal, calico.

  • Docker: Docker 1.13.1 will be used with kubernetes 1.8 (overrides for 17.03.2 and 17.09 possible).

  • Debian 9 (stretch) now supported. AMIs updated with 4.4.102 kernel. A stretch based AMI is available, but jessieremains the default. We will likely change the default to stretch in kops 1.9 or kops 1.10.

  • CoreOS: logrotate support & docker fixes

  • Don't store unneeded secrets on the node

  • ExperimentalCriticalPodAnnotation now enabled by default. Updated critical pod annotations to avoid eviction of system pods

  • Ensure iptables forwarding is enabled, avoiding breaking CNI plugins if Docker or the OS sets a different default.

AWS:

  • New instance types: P3, C5, M5, H1. Please note that NVME volumes are not supported on the default jessieimage, so masters will not boot on M5 and C5 instance types unless a stretch image is chosen (change jessie to stretch in the image name).Also kubernetes will not support mounting persistent volumes on NVME instances until Kubernetes v1.9.
  • Support for root provisioned IOPS.
  • Properly tag public and private subnets for ELB creation in advanced network topologies
  • Use SSL in ELB API server health check

GCE:

  • Checks that networks are in auto mode, not legacy mode. You can either switch your network (gcloud compute networks switch-mode)or specify a different network (current using —vpc flag)
  • Supports rolling updates and the containerized mounter.
  • Sets bucket permissions, so your state bucket and compute can be in different projects.

Early support for:

  • DigitalOcean
  • OpenStack
  • Templating with kops toolbox template
  • cloud-controller-manager
  • encryption-at-rest for the kube-apiserver
  • Mirroring assets to a private S3 bucket, for airgapped installs
  • Mirroring configuration so that kops-state store need not be cluster-accessible (for use with kops-server)
  • Phases, to allow separation of networking, security & compute management
  • Audit Policy
  • CA keypair rotation
  • Additional Subject Alternate Names
  • building code using bazel

All PRs

to beta.1

beta.1 to beta.2

beta.2 to 1.8.0