How-To: Set up Jaeger for distributed tracing

Set up Jaeger for distributed tracing

Dapr supports the Zipkin protocol. Since Jaeger is compatible with Zipkin, the Zipkin protocol can be used to communication with Jaeger.

Configure self hosted mode

Setup

The simplest way to start Jaeger is to use the pre-built all-in-one Jaeger image published to DockerHub:

  1. docker run -d --name jaeger \
  2.   -e COLLECTOR_ZIPKIN_HOST_PORT=:9412 \
  3.   -p 16686:16686 \
  4.   -p 9412:9412 \
  5.   jaegertracing/all-in-one:1.22

Next, create the following YAML files locally:

  • config.yaml: Note that because we are using the Zipkin protocol to talk to Jaeger, we specify the zipkin section of tracing configuration set the endpointAddress to address of the Jaeger instance.
  1. apiVersion: dapr.io/v1alpha1
  2. kind: Configuration
  3. metadata:
  4.   name: tracing
  5.   namespace: default
  6. spec:
  7.   tracing:
  8.     samplingRate: "1"
  9.     zipkin:
  10.       endpointAddress: "http://localhost:9412/api/v2/spans"

To launch the application referring to the new YAML file, you can use --config option:

  1. dapr run --app-id mynode --app-port 3000 node app.js --config config.yaml

Viewing Traces

To view traces, in your browser go to http://localhost:16686 to see the Jaeger UI.

Configure Kubernetes

The following steps shows you how to configure Dapr to send distributed tracing data to Jaeger running as a container in your Kubernetes cluster, how to view them.

Setup

First create the following YAML file to install Jaeger, file name is jaeger-operator.yaml

Development and test

By default, the allInOne Jaeger image uses memory as the backend storage and it is not recommended to use this in a production environment.

  1. apiVersion: jaegertracing.io/v1
  2. kind: "Jaeger"
  3. metadata:
  4.   name: jaeger
  5. spec:
  6.   strategy: allInOne
  7.   ingress:
  8.     enabled: false
  9.   allInOne:
  10.     image: jaegertracing/all-in-one:1.22
  11.     options:
  12.       query:
  13.         base-path: /jaeger

Production

Jaeger uses Elasticsearch as the backend storage, and you can create a secret in k8s cluster to access Elasticsearch server with access control.

  1. kubectl create secret generic jaeger-secret --from-literal=ES_PASSWORD='xxx' --from-literal=ES_USERNAME='xxx' -n ${NAMESPACE}
  1. apiVersion: jaegertracing.io/v1
  2. kind: "Jaeger"
  3. metadata:
  4.   name: jaeger
  5. spec:
  6.   strategy: production
  7.   query:
  8.     options:
  9.       log-level: info
  10.       query:
  11.         base-path: /jaeger
  12.   collector:
  13.     maxReplicas: 5
  14.     resources:
  15.       limits:
  16.         cpu: 500m
  17.         memory: 516Mi
  18.   storage:
  19.     type: elasticsearch
  20.     esIndexCleaner:
  21.       enabled: false                                ## turn the job deployment on and off
  22.       numberOfDays: 7                               ## number of days to wait before deleting a record
  23.       schedule: "55 23 * * *"                       ## cron expression for it to run
  24.       image: jaegertracing/jaeger-es-index-cleaner  ## image of the job
  25.     secretName: jaeger-secret
  26.     options:
  27.       es:
  28.         server-urls: http://elasticsearch:9200

The pictures are as follows, include Elasticsearch and Grafana tracing data:

jaeger-storage-es

grafana

Now, use the above YAML file to install Jaeger

  1. # Install Jaeger
  2. helm repo add jaegertracing https://jaegertracing.github.io/helm-charts
  3. helm install jaeger-operator jaegertracing/jaeger-operator
  4. kubectl apply -f jaeger-operator.yaml
  6. # Wait for Jaeger to be up and running
  7. kubectl wait deploy --selector app.kubernetes.io/name=jaeger --for=condition=available

Next, create the following YAML file locally:

  • tracing.yaml
  1. apiVersion: dapr.io/v1alpha1
  2. kind: Configuration
  3. metadata:
  4.   name: tracing
  5.   namespace: default
  6. spec:
  7.   tracing:
  8.     samplingRate: "1"
  9.     zipkin:
  10.       endpointAddress: "http://jaeger-collector.default.svc.cluster.local:9411/api/v2/spans"

Finally, deploy the the Dapr component and configuration files:

  1. kubectl apply -f tracing.yaml

In order to enable this configuration for your Dapr sidecar, add the following annotation to your pod spec template:

  1. annotations:
  2.   dapr.io/config: "tracing"

That’s it! Your Dapr sidecar is now configured for use with Jaeger.

Viewing Tracing Data

To view traces, connect to the Jaeger Service and open the UI:

  1. kubectl port-forward svc/jaeger-query 16686

In your browser, go to http://localhost:16686 and you will see the Jaeger UI.

jaeger

References

Last modified June 23, 2022: Merge pull request #2550 from ItalyPaleAle/cosmosdb-harcoded-dapr-version (cf03237)