Kubernetes 完全教程

Kubernetes 的 Pod 放置放置策略以及应用案例

王渊命 @jolestar

bg

Agenda

  1. Kubernetes 的 Pod 放置策略
    • 如何放置 Pod 到 Node
    • Affinity 以及 anti-affinity
    • Taints 和 Tolerations 机制
  2. Kubernetes 应用开发
    • Java Spring Cloud 案例
    • PHP Wordpress

如何放置 Pod 到 Node — 从手动编排过度到 Kubernets

Pod 的 nodeName 和 nodeSelector 属性

  1. kubectl label nodes <nodename> disktype=ssd
  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4. name: db
  5. spec:
  6. containers:
  7. - name: db
  8. image: mysql
  9. nodeSelector:
  10.   disktype: ssd

内置的 node labels

  1. kubectl get nodes --show-labels
  1. #cloud-provider
  2. kubernetes.io/hostname
  3. failure-domain.beta.kubernetes.io/zone
  4. failure-domain.beta.kubernetes.io/region
  5. beta.kubernetes.io/instance-type
  6. #kubelet
  7. beta.kubernetes.io/os
  8. beta.kubernetes.io/arch
  9. #kubeadm
  10. node-role.kubernetes.io/master

Affinity 以及 anti-affinity

为什么需要 Affinity 机制

  1. nodeSelector 选择表达式不够强大
  2. nodeSelector 不能区分强制要求和优先配置
  3. 无法避免同一个应用的多个 pod 调度到同一个 node 上

Affinity

  • nodeAffinity
  • podAffinity
  • podAntiAffinity

为什么没有 nodeAntiAffinity?

nodeAffinity

  • preferredDuringSchedulingIgnoredDuringExecution: PreferredSchedulingTerm array
    • preference: nodeSelectorTerms
    • weight: 0-100
  • requiredDuringSchedulingIgnoredDuringExecution: NodeSelector

NodeSelector

参看第四章API Spec 以及安全机制 — Label 和 Selector

  • nodeSelectorTerms
    • matchExpressions array
      • key
      • operator: In, NotIn, Exists, DoesNotExist. Gt, Lt
      • values: string array
  1.   matchExpressions:
  2.     - {key: disktype, operator: In, values: [ssd]}
  3.     - {key: environment, operator: NotIn, values: [dev]}

nodeAffinity

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4.   name: with-node-affinity
  5. spec:
  6.   affinity:
  7.     nodeAffinity:
  8.       requiredDuringSchedulingIgnoredDuringExecution:
  9.         nodeSelectorTerms:
  10.         - matchExpressions:
  11.           - key: failure-domain.beta.kubernetes.io/zone
  12.             operator: In
  13.             values:
  14.             - az1
  15.             - az2
  16.   containers:
  17.   - name: with-node-affinity
  18.     image: example

nodeAffinity

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4.   name: with-node-affinity
  5. spec:
  6.   affinity:
  7.     nodeAffinity:
  8.       preferredDuringSchedulingIgnoredDuringExecution:
  9.       - weight: 1
  10.         preference:
  11.           matchExpressions:
  12.           - key: disktype
  13.             operator: In
  14.             values:
  15.             - ssd
  16.   containers:
  17.   - name: with-node-affinity
  18.     image: example

podAffinity/podAntiAffinity

  • preferredDuringSchedulingIgnoredDuringExecution: WeightedPodAffinityTerm array
    • podAffinityTerm： PodAffinityTerm
    • weight： 0-100
  • requiredDuringSchedulingIgnoredDuringExecution: PodAffinityTerm array
    • labelSelector： LabelSelector
    • namespaces： string array
    • topologyKey

podAffinity

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4.   name: with-pod-affinity
  5. spec:
  6.   affinity:
  7.     podAffinity:
  8.       requiredDuringSchedulingIgnoredDuringExecution:
  9.       - labelSelector:
  10.           matchExpressions:
  11.           - key: security
  12.             operator: In
  13.             values:
  14.             - S1
  15.         topologyKey: failure-domain.beta.kubernetes.io/zone
  16.   containers:
  17.   - name: with-pod-affinity
  18.     image: example

podAntiAffinity

  1. apiVersion: v1
  2. kind: Pod
  3. metadata:
  4.   name: zk
  5.   lables:
  6.     app: zk
  7. spec:
  8.   affinity:
  9.     podAntiAffinity:
  10.       requiredDuringSchedulingIgnoredDuringExecution:
  11.         - labelSelector:
  12.             matchExpressions:
  13.             - key: "app"
  14.               operator: In
  15.               values:
  16.               - zk
  17.           topologyKey: "kubernetes.io/hostname"
  18.   containers:
  19.   - name: zk
  20.     image: zk

为什么要有 Taints 和 Tolerations

  1. Taints 隔离 Node
    • 上线新 Node 验证
    • 下线 Node 维护
    • 专用 Node
  2. Tolerations Pod 对 Taints Node 的容忍
    • 验证 Taints Node 是否工作正常
    • 表明 Pod 有资格使用专用 Node
    • 每个节点必须运行的基础 Pod

Taints 和 Tolerations

  1. kubectl taint nodes node1 key=value:NoSchedule
  • tolerations
    • operator: Exists,Equal (default)
    • effect: NoSchedule,PreferNoSchedule,NoExecute
      ```yaml
      tolerations:
  • key: “key”
    operator: “Equal”
    value: “value”
    effect: “NoSchedule”
    tolerationSeconds: 3600
    ```

Tolerations

  1. apiVersion: extensions/v1beta1
  2. kind: DaemonSet
  3. metadata:
  4.   name: kube-proxy
  5. spec:
  6.   template:
  7.     spec:
  8.       containers:
  9.       - name: kube-proxy
  10.         image: gcr.io/google_containers/hyperkube-amd64:1.7.9
  11.       tolerations:
  12.       - key: "CriticalAddonsOnly"
  13.         operator: "Exists"
  14.       - key: "dedicated"
  15.         operator: "Exists"
  16.       - key: "node-role.kubernetes.io/master"
  17.         effect: NoSchedule
  18.       - key: node.cloudprovider.kubernetes.io/uninitialized
  19.         effect: NoSchedule
  20.         value: "true"

Taint based Evictions

  1. kube-controller-manager --feature-gates=TaintBasedEvictions=true
内置的 taints
  1. node.kubernetes.io/not-ready
  2. node.alpha.kubernetes.io/unreachable
  3. node.kubernetes.io/out-of-disk
  4. node.kubernetes.io/memory-pressure
  5. node.kubernetes.io/disk-pressure
  6. node.kubernetes.io/network-unavailable
  7. node.cloudprovider.kubernetes.io/uninitialized
DaemonSet Tolerations
  1. node.alpha.kubernetes.io/unreachable
  2. node.kubernetes.io/not-ready

Kubernetes 应用开发案例

  1. Java SpringCloud
  2. PHP Wordpress

Kubernetes SpringCloud 案例

  1. spring-cloud-kubernetes
  2. fabric8

spring-cloud-kubernetes

https://github.com/spring-cloud-incubator/spring-cloud-kubernetes

  • DiscoveryClient
  • ConfigMap/Secrets PropertySource
  • Pod Health Indicator
  • Ribbon/Zipkin discovery

fabric8

Integrated Development Platform for Kubernetes

https://fabric8.io/

fabric8-maven-plugin

  • fabric8:run
  • fabric8:build
  • fabric8:push
  • fabric8:deploy

SpringCloud 案例演示

准备 minikube 环境

  1. #第一次启动要翻墙
  2. minikube start
  3. #没开启一个新的 terminal 都需要重新执行一下
  4. eval $(minikube docker-env)
  5. sudo route -n add 10.0.0.0/24 $(minikube ip)

spring-cloud-kubernetes-examples

  • kubernetes-hello-world-example
    1.   mvn fabric8:run -Pkubernetes
    2.      kubectl get service
    3.   curl http://$clusterip
    4.   #修改代码 重新执行上面的命令看结果
  • kubernetes-reload-example
    1.   kubectl apply -f config-map.yml
    2.   mvn fabric8:run -Pkubernetes
    3.   kubectl logs -f $spring-cloud-reload-podid
    4.   kubectl edit configmap reload-example
    5.   # 可看到修改后的 configmap 被重新加载

spring-cloud-kubernetes-examples

  • kubernetes-circuitbreaker-ribbon-example
    1.   cd name-service; mvn fabric8:run -Pkubernetes
    2.   cd greeting-service; mvn fabric8:run -Pkubernetes
    3.   curl $greeting-service-clusterIP/greeting
    4.   kubectl scale --replicas=2 deployment name-service
    5.   curl $greeting-service-clusterIP/greeting
    6.   # 可以看到 ribbon 的效果

PHP Wordpress

Kubernetes 上部署 Wordpress

  • LoadBalancer
  • PersistentVolumeClaim
  • Secret
  • MySQL

总结

  • 对 Kubernetes 的 Pod 放置策略有一个整体的了解
  • 了解在 Kubernetes 上开发应用的流程以及应用如何利用 Kubernetes 的特性

作业

  1. 思考自己所在公司的服务如果全部部署到 Kubernetes 中，需要如何规划 Node 的 Label，以及那些节点需要通过 taints 机制来专用，那些服务需要使用到 Affinity 机制。
  2. 通过 Kubernetes 去部署一个更复杂的微服务应用（根据自己熟悉的语言来选择）
  3. 以 Wordpress 为例，思考如果要做高可用，需要做哪些方面的改造（提示: 图片存储，数据库）。

关于我

个人博客: http://jolestar.com
课程 Github：https://github.com/jolestar/kubernetes-complete-course

课程 QQ 群: 451252952

about

第五课：Kubernetes 的 Pod 放置放置策略以及应用案例 - 图3