metrics监控

采集运行态 Metrics 指标并接入 Prometheus、Grafana 系统

Dubbo 支持采集运行态 Metrics 指标并接入 Prometheus、Grafana 系统，实现对微服务集群的可视化监控，以下是一个具体的使用示例，可查看 示例完整源码。

Contents

  • server/main.go - is the main definition of the service, handler and rpc server
  • client/main.go - is the rpc client
  • proto - contains the protobuf definition of the API

How to run

Run server

  1. go run ./go-server/cmd/main.go

test server work as expected:

  1. curl \
  2.     --header "Content-Type: application/json" \
  3.     --data '{"name": "Dubbo"}' \
  4.     http://localhost:20000/greet.GreetService/Greet

Run client

  1. go run ./go-client/cmd/main.go

deploy to local

install prometheus and open prometheus config file prometheus.yml, write the config like this

  1. global:
  2.   evaluation_interval: 15s
  3.   scrape_interval: 15s
  4. scrape_configs:
  5. - job_name: dubbo-provider
  6.   scrape_interval: 15s
  7.   scrape_timeout: 5s
  8.   metrics_path: /prometheus
  9.   static_configs:
  10.     - targets: ['localhost:9099']
  11. - job_name: dubbo-consumer
  12.   scrape_interval: 15s
  13.   scrape_timeout: 5s
  14.   metrics_path: /prometheus
  15.   static_configs:
  16.     - targets: ['localhost:9097']

install grafana and open grafana web page like localhost:3000

open: 【Home / Connections / Data sources】

click 【Add new data source】

select Prometheus

enter 【Prometheus server URL】 like http://localhost:9090 and click 【Save & test】

datasource.png

open 【Home / Dashboards 】click 【New】【import】and enter 19294 click Load

import

if your grafana can’t access internet you can open https://grafana.com/grafana/dashboards/19294-dubbo-observability/ and click 【Download JSON】

paste the JSON

json.png

datasource.png

click 【Import】button and you will see the Dubbo Observability dashboard,enjoy it

databoard

Deploy to Kubernetes

kube-prometheus

install prometheus in k8s kube-prometheus

Set prometheus-service.yaml type to NodePort

  1. add dubboPodMoitor.yaml to kube-prometheus manifests dir, The content is as follows
  1. apiVersion: monitoring.coreos.com/v1
  2. kind: PodMonitor
  3. metadata:
  4.  name: podmonitor
  5.  labels:
  6.    app: podmonitor
  7.  namespace: monitoring
  8. spec:
  9.  namespaceSelector:
  10.    matchNames:
  11.      - dubbo-system
  12.  selector:
  13.    matchLabels:
  14.      app-type: dubbo
  15.  podMetricsEndpoints:
  16.    - port: metrics # ref to dubbo-app port name metrics
  17.      path: /prometheus
  18. ---
  19. # rbac
  20. apiVersion: rbac.authorization.k8s.io/v1
  21. kind: Role
  22. metadata:
  23.  namespace: dubbo-system
  24.  name: pod-reader
  25. rules:
  26.  - apiGroups: [""]
  27.    resources: ["pods"]
  28.    verbs: ["get", "list", "watch"]
  30. ---
  31. # rbac
  32. apiVersion: rbac.authorization.k8s.io/v1
  33. kind: RoleBinding
  34. metadata:
  35.  name: pod-reader-binding
  36.  namespace: dubbo-system
  37. roleRef:
  38.  apiGroup: rbac.authorization.k8s.io
  39.  kind: Role
  40.  name: pod-reader
  41. subjects:
  42.  - kind: ServiceAccount
  43.    name: prometheus-k8s
  44.    namespace: monitoring
  1. kubectl apply -f Deployment.yaml
  2. open prometheus web page such as http://localhost:9090/targets podmonitor.png

最后修改 September 13, 2024: Refactor website structure (#2860) (1a4b998f54b)