09-5.部署 EFK 插件

EFK 对应的目录:kubernetes/cluster/addons/fluentd-elasticsearch

  1. $ cd /opt/k8s/kubernetes/cluster/addons/fluentd-elasticsearch
  2. $ ls *.yaml
  3. es-service.yaml es-statefulset.yaml fluentd-es-configmap.yaml fluentd-es-ds.yaml kibana-deployment.yaml kibana-service.yaml

修改定义文件

  1. $ cp es-statefulset.yaml{,.orig}
  2. $ diff es-statefulset.yaml{,.orig}
  3. 76c76
  4. < - image: longtds/elasticsearch:v5.6.4
  5. ---
  6. > - image: k8s.gcr.io/elasticsearch:v5.6.4
  7. $ cp fluentd-es-ds.yaml{,.orig}
  8. $ diff fluentd-es-ds.yaml{,.orig}
  9. 79c79
  10. < image: netonline/fluentd-elasticsearch:v2.0.4
  11. ---
  12. > image: k8s.gcr.io/fluentd-elasticsearch:v2.0.4

给 Node 设置标签

DaemonSet fluentd-es 只会调度到设置了标签 beta.kubernetes.io/fluentd-ds-ready=true 的 Node,需要在期望运行 fluentd 的 Node 上设置该标签;

  1. $ kubectl get nodes
  2. NAME STATUS ROLES AGE VERSION
  3. kube-node1 Ready <none> 3d v1.10.4
  4. kube-node2 Ready <none> 3d v1.10.4
  5. kube-node3 Ready <none> 3d v1.10.4
  6. $ kubectl label nodes kube-node3 beta.kubernetes.io/fluentd-ds-ready=true
  7. node "kube-node3" labeled

执行定义文件

  1. $ pwd
  2. /opt/k8s/kubernetes/cluster/addons/fluentd-elasticsearch
  3. $ ls *.yaml
  4. es-service.yaml es-statefulset.yaml fluentd-es-configmap.yaml fluentd-es-ds.yaml kibana-deployment.yaml kibana-service.yaml
  5. $ kubectl create -f .

检查执行结果

  1. $ kubectl get pods -n kube-system -o wide|grep -E 'elasticsearch|fluentd|kibana'
  2. elasticsearch-logging-0 1/1 Running 0 5m 172.30.81.7 kube-node1
  3. elasticsearch-logging-1 1/1 Running 0 2m 172.30.39.8 kube-node3
  4. fluentd-es-v2.0.4-hntfp 1/1 Running 0 5m 172.30.39.6 kube-node3
  5. kibana-logging-7445dc9757-pvpcv 1/1 Running 0 5m 172.30.39.7 kube-node3
  6. $ kubectl get service -n kube-system|grep -E 'elasticsearch|kibana'
  7. elasticsearch-logging ClusterIP 10.254.50.198 <none> 9200/TCP 5m
  8. kibana-logging ClusterIP 10.254.255.190 <none> 5601/TCP 5m

kibana Pod 第一次启动时会用较长时间(0-20分钟)来优化和 Cache 状态页面,可以 tailf 该 Pod 的日志观察进度:

  1. [k8s@kube-node1 fluentd-elasticsearch]$ kubectl logs kibana-logging-7445dc9757-pvpcv -n kube-system -f
  2. {"type":"log","@timestamp":"2018-06-16T11:36:18Z","tags":["info","optimize"],"pid":1,"message":"Optimizing and caching bundles for graph, ml, kibana, stateSessionStorageRedirect, timelion and status_page. This may take a few minutes"}
  3. {"type":"log","@timestamp":"2018-06-16T11:40:03Z","tags":["info","optimize"],"pid":1,"message":"Optimization of bundles for graph, ml, kibana, stateSessionStorageRedirect, timelion and status_page complete in 224.57 seconds"}

注意:只有当的 Kibana pod 启动完成后,才能查看 kibana dashboard,否则会提示 refuse。

访问 kibana

  1. 通过 kube-apiserver 访问:

    1. $ kubectl cluster-info|grep -E 'Elasticsearch|Kibana'
    2. Elasticsearch is running at https://172.27.129.105:6443/api/v1/namespaces/kube-system/services/elasticsearch-logging/proxy
    3. Kibana is running at https://172.27.129.105:6443/api/v1/namespaces/kube-system/services/kibana-logging/proxy

    浏览器访问 URL: https://172.27.129.105:6443/api/v1/namespaces/kube-system/services/kibana-logging/proxy 对于 virtuabox 做了端口映射: http://127.0.0.1:8080/api/v1/namespaces/kube-system/services/kibana-logging/proxy

  2. 通过 kubectl proxy 访问:

    创建代理

    1. $ kubectl proxy --address='172.27.129.105' --port=8086 --accept-hosts='^*$'
    2. Starting to serve on 172.27.129.80:8086

    浏览器访问 URL:http://172.27.129.105:8086/api/v1/namespaces/kube-system/services/kibana-logging/proxy 对于 virtuabox 做了端口映射: http://127.0.0.1:8086/api/v1/namespaces/kube-system/services/kibana-logging/proxy

在 Settings -> Indices 页面创建一个 index(相当于 mysql 中的一个 database),选中 Index contains time-based events,使用默认的 logstash-* pattern,点击 Create ;

es-setting

创建 Index 后,稍等几分钟就可以在 Discover 菜单下看到 ElasticSearch logging 中汇聚的日志;

es-home