Profiling Karmada

Enable profiling

To profile Karmada components running inside a Kubernetes pod, set —enable-pprof flag to true in the yaml of Karmada components. The default profiling address is 127.0.0.1:6060, and it can be configured via --profiling-bind-address. The components which are compiled by the Karmada source code support the flag above, including Karmada-agent, Karmada-aggregated-apiserver, Karmada-controller-manager, Karmada-descheduler, Karmada-search, Karmada-scheduler, Karmada-scheduler-estimator, Karmada-webhook.

  1. --enable-pprof                                                                                                                                                                
  2.                 Enable profiling via web interface host:port/debug/pprof/.
  3. --profiling-bind-address string                                                                                                                                               
  4.                 The TCP address for serving profiling(e.g. 127.0.0.1:6060, :6060). This is only applicable if profiling is enabled. (default ":6060")

Expose the endpoint at the local port

You can get at the application in the pod by port forwarding with kubectl, for example:

  1. $ kubectl -n karmada-system get pod
  2. NAME                                          READY   STATUS    RESTARTS   AGE
  3. karmada-controller-manager-7567b44b67-8kt59   1/1     Running   0          19s
  4. ...
  1. $ kubectl -n karmada-system port-forward karmada-controller-manager-7567b44b67-8kt59 6060
  2. Forwarding from 127.0.0.1:6060 -> 6060
  3. Forwarding from [::1]:6060 -> 6060

The HTTP endpoint will now be available as a local port.

Generate the data

You can then generate the file for the memory profile with curl and pipe the data to a file:

  1. $ curl http://localhost:6060/debug/pprof/heap  > heap.pprof

Generate the file for the CPU profile with curl and pipe the data to a file (7200 seconds is two hours):

  1. curl "http://localhost:6060/debug/pprof/profile?seconds=7200" > cpu.pprof

Analyze the data

To analyze the data:

  1. go tool pprof heap.pprof

Read more about profiling

  1. Profiling Golang Programs on Kubernetes
  2. Official Go blog