部署 ETCD

为 ETCD 签发证书

这里证书可以只创建一次，所有 etcd 实例都公用这里创建的证书:

  1. cat > etcd-csr.json <<EOF
  2. {
  3.     "CN": "etcd",
  4.     "hosts": [
  5.       "127.0.0.1",
  6.       "10.200.16.79",
  7.       "10.200.17.6",
  8.       "10.200.16.70"
  9.     ],
  10.     "key": {
  11.         "algo": "rsa",
  12.         "size": 2048
  13.     },
  14.     "names": [
  15.         {
  16.             "C": "CN",
  17.             "ST": "SiChuan",
  18.             "L": "Chengdu",
  19.             "O": "etcd",
  20.             "OU": "etcd"
  21.         }
  22.     ]
  23. }
  24. EOF
  26. cfssl gencert \
  27.   -ca=ca.pem \
  28.   -ca-key=ca-key.pem \
  29.   -config=ca-config.json \
  30.   -profile=kubernetes \
  31.   etcd-csr.json | cfssljson -bare etcd

hosts 需要包含 etcd 每个实例所在节点的内网 IP

会生成下面两个重要的文件:

  • etcd-key.pem: kube-apiserver 证书密钥
  • etcd.pem: kube-apiserver 证书

下载安装 ETCD

下载 release 包:

  1. wget -q --show-progress --https-only --timestamping \
  2.   "https://github.com/etcd-io/etcd/releases/download/v3.4.1/etcd-v3.4.1-linux-amd64.tar.gz"

解压安装 etcd 和 etcdctl 到 PATH:

  1. tar -xvf etcd-v3.4.1-linux-amd64.tar.gz
  2. sudo mv etcd-v3.4.1-linux-amd64/etcd* /usr/local/bin/

配置

创建配置相关目录，放入证书文件:

  1. sudo mkdir -p /etc/etcd /var/lib/etcd
  2. sudo cp ca.pem etcd-key.pem etcd.pem /etc/etcd/

etcd 集群每个成员都需要一个名字，这里第一个成员名字用 infra0，第二个可以用 infra1，以此类推，你也可以直接用节点的 hostname:

  1. NAME=infra0

记当前部署 ETCD 的节点的内网 IP 为 INTERNAL_IP:

  1. INTERNAL_IP=10.200.16.79

记所有 ETCD 成员的名称和成员间通信的 https 监听地址为 ETCD_SERVERS (注意是 2380 端口，不是 2379):

  1. ETCD_SERVERS="infra0=https://10.200.16.79:2380,infra1=https://10.200.17.6:2380,infra2=https://10.200.16.70:2380"

创建 systemd 配置:

  1. cat <<EOF | sudo tee /etc/systemd/system/etcd.service
  2. [Unit]
  3. Description=etcd
  4. Documentation=https://github.com/coreos
  6. [Service]
  7. Type=notify
  8. ExecStart=/usr/local/bin/etcd \\
  9.   --name ${NAME} \\
  10.   --cert-file=/etc/etcd/etcd.pem \\
  11.   --key-file=/etc/etcd/etcd-key.pem \\
  12.   --peer-cert-file=/etc/etcd/etcd.pem \\
  13.   --peer-key-file=/etc/etcd/etcd-key.pem \\
  14.   --trusted-ca-file=/etc/etcd/ca.pem \\
  15.   --peer-trusted-ca-file=/etc/etcd/ca.pem \\
  16.   --peer-client-cert-auth \\
  17.   --client-cert-auth \\
  18.   --initial-advertise-peer-urls https://${INTERNAL_IP}:2380 \\
  19.   --listen-peer-urls https://${INTERNAL_IP}:2380 \\
  20.   --listen-client-urls https://${INTERNAL_IP}:2379,https://127.0.0.1:2379 \\
  21.   --advertise-client-urls https://${INTERNAL_IP}:2379 \\
  22.   --initial-cluster-token etcd-cluster-0 \\
  23.   --initial-cluster ${ETCD_SERVERS} \\
  24.   --initial-cluster-state new \\
  25.   --data-dir=/var/lib/etcd
  26. Restart=on-failure
  27. RestartSec=5
  29. [Install]
  30. WantedBy=multi-user.target
  31. EOF

启动

  1. sudo systemctl daemon-reload
  2. sudo systemctl enable etcd
  3. sudo systemctl start etcd

验证

等所有 etcd 成员安装启动成功后，来验证下是否可用:

  1. sudo ETCDCTL_API=3 etcdctl member list \
  2.   --endpoints=https://127.0.0.1:2379 \
  3.   --cacert=/etc/etcd/ca.pem \
  4.   --cert=/etc/etcd/etcd.pem \
  5.   --key=/etc/etcd/etcd-key.pem

输出:

  1. a7f995caeeaf7a59, started, infra1, https://10.200.17.6:2380, https://10.200.17.6:2379, false
  2. b90901a06e9aec53, started, infra2, https://10.200.16.70:2380, https://10.200.16.70:2379, false
  3. ba126eb695f5ba71, started, infra0, https://10.200.16.79:2380, https://10.200.16.79:2379, false