kubeadm

Kubernetes 一键部署脚本（使用 docker 运行时）

  1. # on master
  2. git clone https://github.com/feiskyer/ops
  3. cd ops
  4. kubernetes/install-kubernetes.sh
  5. # 记住控制台输出的 TOEKN 和 MASTER 地址，在其他 Node 安装时会用到
  7. # on node
  8. git clone https://github.com/feiskyer/ops
  9. cd ops
  10. export TOKEN=xxxxx
  11. export MASTER_IP=xx.xx.xx.xx
  12. kubernetes/add-docker-node.sh

以下是详细的安装步骤。

初始化系统

所有机器都需要初始化 docker 和 kubelet。

ubuntu

  1. # for ubuntu 16.04+
  2. apt-get update && apt-get install -y apt-transport-https
  3. curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
  4. cat <<EOF> /etc/apt/sources.list.d/kubernetes.list
  5. deb http://apt.kubernetes.io/ kubernetes-xenial main
  6. EOF
  8. apt-get update
  10. # Install docker if you don't have it already.
  11. apt-get install -y docker.io
  12. apt-get install -y kubelet kubeadm kubectl kubernetes-cni
  13. systemctl enable docker && systemctl start docker
  14. systemctl enable kubelet

centos

  1. cat <<EOF> /etc/yum.repos.d/kubernetes.repo
  2. [kubernetes]
  3. name=Kubernetes
  4. baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64
  5. enabled=1
  6. gpgcheck=1
  7. repo_gpgcheck=1
  8. gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
  9.        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
  10. EOF
  12. setenforce 0
  13. yum install -y docker kubelet kubeadm kubectl kubernetes-cni
  14. systemctl enable docker && systemctl start docker
  15. systemctl enable kubelet

国内用户也可以使用阿里云的镜像来安装

  1. cat <<EOF> /etc/yum.repos.d/kubernetes.repo
  2. [kubernetes]
  3. name=Kubernetes
  4. baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
  5. enabled=1
  6. gpgcheck=0
  7. EOF

安装 master

  1. # --api-advertise-addresses <ip-address>
  2. # for flannel, setup --pod-network-cidr 10.244.0.0/16
  3. kubeadm init --pod-network-cidr 10.244.0.0/16 --kubernetes-version latest
  5. # enable schedule pods on the master
  6. export KUBECONFIG=/etc/kubernetes/admin.conf
  7. # for v1.5-, use kubectl taint nodes --all dedicated-
  8. kubectl taint nodes --all node-role.kubernetes.io/master:NoSchedule-

如果需要修改 kubernetes 服务的配置选项，则需要创建一个 MasterConfiguration 配置文件，其格式为

  1. apiVersion: kubeadm.k8s.io/v1alpha1
  2. kind: MasterConfiguration
  3. api:
  4.   advertiseAddress: <address|string>
  5.   bindPort: <int>
  6. etcd:
  7.   endpoints:
  8.   - <endpoint1|string>
  9.   - <endpoint2|string>
  10.   caFile: <path|string>
  11.   certFile: <path|string>
  12.   keyFile: <path|string>
  13. networking:
  14.   dnsDomain: <string>
  15.   serviceSubnet: <cidr>
  16.   podSubnet: <cidr>
  17. kubernetesVersion: <string>
  18. cloudProvider: <string>
  19. authorizationModes:
  20. - <authorizationMode1|string>
  21. - <authorizationMode2|string>
  22. token: <string>
  23. tokenTTL: <time duration>
  24. selfHosted: <bool>
  25. apiServerExtraArgs:
  26.   <argument>: <value|string>
  27.   <argument>: <value|string>
  28. controllerManagerExtraArgs:
  29.   <argument>: <value|string>
  30.   <argument>: <value|string>
  31. schedulerExtraArgs:
  32.   <argument>: <value|string>
  33.   <argument>: <value|string>
  34. apiServerCertSANs:
  35. - <name1|string>
  36. - <name2|string>
  37. certificatesDir: <string>

比如

  1. # cat kubeadm.yml
  2. kind: MasterConfiguration
  3. apiVersion: kubeadm.k8s.io/v1alpha1
  4. kubernetesVersion: "stable"
  5. apiServerCertSANs: []
  6. controllerManagerExtraArgs:
  7.   horizontal-pod-autoscaler-use-rest-clients: "true"
  8.   horizontal-pod-autoscaler-sync-period: "10s"
  9.   node-monitor-grace-period: "10s"
  10.   feature-gates: "AllAlpha=true"
  11.   enable-dynamic-provisioning: "true"
  12. apiServerExtraArgs:
  13.   runtime-config: "api/all=true"
  14.   feature-gates: "AllAlpha=true"
  15. networking:
  16.   podSubnet: "10.244.0.0/16"

然后，在初始化 master 的时候指定 kubeadm.yml 的路径：

  1. kubeadm init --config ./kubeadm.yaml

配置 Network plugin

CNI bridge

  1. mkdir -p /etc/cni/net.d
  2. cat >/etc/cni/net.d/10-mynet.conf <<-EOF
  3. {
  4.     "cniVersion": "0.3.0",
  5.     "name": "mynet",
  6.     "type": "bridge",
  7.     "bridge": "cni0",
  8.     "isGateway": true,
  9.     "ipMasq": true,
  10.     "ipam": {
  11.         "type": "host-local",
  12.         "subnet": "10.244.0.0/16",
  13.         "routes": [
  14.             {"dst": "0.0.0.0/0"}
  15.         ]
  16.     }
  17. }
  18. EOF
  19. cat >/etc/cni/net.d/99-loopback.conf <<-EOF
  20. {
  21.     "cniVersion": "0.3.0",
  22.     "type": "loopback"
  23. }
  24. EOF

flannel

注意：需要 kubeadm init 时设置 --pod-network-cidr=10.244.0.0/16

  1. kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml

weave

  1. kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d'\n')"

calico

注意：需要 kubeadm init 时设置 --pod-network-cidr=192.168.0.0/16

  1. kubectl apply -f https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/kubeadm/1.7/calico.yaml

添加 Node

  1. token=$(kubeadm token list | grep authentication,signing | awk '{print $1}')
  2. kubeadm join --token $token ${master_ip}

跟 Master 一样，添加 Node 的时候也可以自定义 Kubernetes 服务的选项，格式为

  1. apiVersion: kubeadm.k8s.io/v1alpha1
  2. kind: NodeConfiguration
  3. caCertPath: <path|string>
  4. discoveryFile: <path|string>
  5. discoveryToken: <string>
  6. discoveryTokenAPIServers:
  7. - <address|string>
  8. - <address|string>
  9. tlsBootstrapToken: <string>

在把 Node 加入集群的时候，指定 NodeConfiguration 配置文件的路径

  1. kubeadm join --config ./nodeconfig.yml --token $token ${master_ip}

删除安装

  1. kubeadm reset

动态升级

kubeadm v1.8 开始支持动态升级，升级步骤为

  • 首先上传 kubeadm 配置，如 kubeadm config upload from-flags [flags]（使用命令行参数）或 kubeadm config upload from-file --config [config]（使用配置文件）
  • 在 master 上检查新版本 kubeadm upgrade plan， 当有新版本（如 v1.8.0）时，执行 kubeadm upgrade apply v1.8.0 升级控制平面
  • 手动 升级 CNI 插件（如果有新版本的话）
  • 添加自动证书回滚的 RBAC 策略 kubectl create clusterrolebinding kubeadm:node-autoapprove-certificate-rotation --clusterrole=system:certificates.k8s.io:certificatesigningrequests:selfnodeclient --group=system:nodes
  • 最后升级 kubelet
  1. $ kubectl drain $HOST --ignore-daemonsets
  3. # 升级软件包
  4. $ apt-get update
  5. $ apt-get upgrade
  6. # CentOS 上面执行 yum 升级
  7. # $ yum update
  9. $ kubectl uncordon $HOST

手动升级

kubeadm v1.7 以及以前的版本不支持动态升级，但可以手动升级。

升级 Master

假设你已经有一个使用 kubeadm 部署的 Kubernetes v1.6 集群，那么升级到 v1.7 的方法为：

  1. 升级安装包 apt-get upgrade && apt-get update
  2. 重启 kubelet systemctl restart kubelet
  3. 删除 kube-proxy DaemonSet KUBECONFIG=/etc/kubernetes/admin.conf kubectl delete daemonset kube-proxy -n kube-system
  4. kubeadm init —skip-preflight-checks —kubernetes-version v1.7.1
  5. 更新 CNI 插件

升级 Node

  1. 升级安装包 apt-get upgrade && apt-get update
  2. 重启 kubelet systemctl restart kubelet

安全选项

默认情况下，kubeadm 会开启 Node 客户端证书的自动批准，如果不需要的话可以选择关闭，关闭方法为

  1. $ kubectl delete clusterrole kubeadm:node-autoapprove-bootstrap

关闭后，增加新的 Node 时，kubeadm join 会阻塞等待管理员手动批准，匹配方法为

  1. $ kubectl get csr
  2. NAME                                                   AGE       REQUESTOR                 CONDITION
  3. node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ   18s       system:bootstrap:878f07   Pending
  5. $ kubectl certificate approve node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ
  6. certificatesigningrequest "node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ" approved
  8. $ kubectl get csr
  9. NAME                                                   AGE       REQUESTOR                 CONDITION
  10. node-csr-c69HXe7aYcqkS1bKmH4faEnHAWxn6i2bHZ2mD04jZyQ   1m        system:bootstrap:878f07   Approved,Issued

参考文档