部署 EFK 插件

官方文件目录：kubernetes/cluster/addons/fluentd-elasticsearch

  1. $ cd /opt/k8s/work/kubernetes/cluster/addons/fluentd-elasticsearch
  2. $ ls *.yaml
  3. es-service.yaml  es-statefulset.yaml  fluentd-es-configmap.yaml  fluentd-es-ds.yaml  kibana-deployment.yaml  kibana-service.yaml

给 Node 设置标签

DaemonSet fluentd-es-v1.22 只会调度到设置了标签 beta.kubernetes.io/fluentd-ds-ready=true 的 Node，需要在期望运行 fluentd 的 Node 上设置该标签；

  1. $ kubectl label nodes m7-demo-136003 beta.kubernetes.io/fluentd-ds-ready=true
  2. node "m7-demo-136003" labeled

执行定义文件

  1. cd /opt/k8s/work/kubernetes/cluster/addons/fluentd-elasticsearch
  2. kubectl create -f .

检查执行结果

  1. $ kubectl get deployment -n kube-system|grep kibana
  2. kibana-logging         1         1         1            0           2m
  4. $ kubectl get pods -n kube-system|grep -E 'elasticsearch|fluentd|kibana'
  5. elasticsearch-logging-v1-kwc9w          1/1       Running   0          4m
  6. elasticsearch-logging-v1-ws9mk          1/1       Running   0          4m
  7. fluentd-es-v1.22-g76x0                  1/1       Running   0          4m
  8. kibana-logging-324921636-ph7sn          1/1       Running   0          4m
  10. $ kubectl get service  -n kube-system|grep -E 'elasticsearch|kibana'
  11. elasticsearch-logging   ClusterIP   10.254.254.203   <none>        9200/TCP        4m
  12. kibana-logging          ClusterIP   10.254.105.54    <none>        5601/TCP        4m

kibana Pod 第一次启动时会用较长时间(1-10分钟)来优化和 Cache 状态页面，可以 tailf 该 Pod 的日志观察进度：

  1. $ kubectl logs kibana-logging-69bc5bf6cc-kz27l -n kube-system -f
  2. {"type":"log","@timestamp":"2018-07-04T08:34:32Z","tags":["info","optimize"],"pid":1,"message":"Optimizing and caching bundles for graph, ml, kibana, stateSessionStorageRedirect, timelion and status_page. This may take a few minutes"}
  3. {"type":"log","@timestamp":"2018-07-04T08:37:33Z","tags":["info","optimize"],"pid":1,"message":"Optimization of bundles for graph, ml, kibana, stateSessionStorageRedirect, timelion and status_page complete in 180.78 seconds"}
  4. {"type":"log","@timestamp":"2018-07-04T08:37:33Z","tags":["status","plugin:kibana@5.5.1","info"],"pid":1,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}

注意：只有当的 Kibana pod 启动完成后，才能查看 kibana dashboard，否则会提示 refuse。

访问 kibana

  1. 通过 kube-apiserver 访问：

    1.  $ kubectl cluster-info|grep -E 'Elasticsearch|Kibana'
    2.  Elasticsearch is running at https://172.27.128.254:8443/api/v1/namespaces/kube-system/services/elasticsearch-logging/proxy
    3.  Kibana is running at https://172.27.128.254:8443/api/v1/namespaces/kube-system/services/kibana-logging/proxy

    浏览器访问 URL： https://172.27.128.254:8443/api/v1/namespaces/kube-system/services/kibana-logging/proxy

  2. 通过 kubectl proxy 访问：

    创建代理

    1.  $ kubectl proxy --address='172.27.128.254' --port=8086 --accept-hosts='^*$'
    2.  Starting to serve on 172.27.128.254:8086

    浏览器访问 URL：http://172.27.128.254:8086/api/v1/namespaces/kube-system/services/kibana-logging/proxy

在 Settings -> Indices 页面创建一个 index（相当于 mysql 中的一个 database），选中 Index contains time-based events，使用默认的 logstash-* pattern，点击 Create ;

es-setting

创建 Index 后，稍等几分钟就可以在 Discover 菜单下看到 ElasticSearch logging 中汇聚的日志；

es-home