Overview

Overview

Distributed tracing enables users to track a request through mesh that is distributed across multiple services.This allows a deeper understanding about request latency, serialization and parallelism via visualization.

Istio leverages Envoy’s distributed tracing featureto provide tracing integration out of the box. Specifically, Istio provides options to install various tracing backendand configure proxies to send trace spans to them automatically.See Zipkin, Jaeger and LightStep task docs about how Istio works with those tracing systems.

Trace context propagation

Although Istio proxies are able to automatically send spans, they need some hints to tie together the entire trace.Applications need to propagate the appropriate HTTP headers so that when the proxies send span information,the spans can be correlated correctly into a single trace.

To do this, an application needs to collect and propagate the following headers from the incoming request to any outgoing requests:

x-request-id
x-b3-traceid
x-b3-spanid
x-b3-parentspanid
x-b3-sampled
x-b3-flags
x-ot-span-context

Additionally, tracing integrations based on OpenCensus (e.g. Stackdriver) propagate the following headers:

x-cloud-trace-context
traceparent
grpc-trace-bin

If you look at the sample Python productpage service, for example,you see that the application extracts the required headers from an HTTP requestusing OpenTracing libraries:

def getForwardHeaders(request):
    headers = {}
    # x-b3-*** headers can be populated using the opentracing span
    span = get_current_span()
    carrier = {}
    tracer.inject(
        span_context=span.context,
        format=Format.HTTP_HEADERS,
        carrier=carrier)
    headers.update(carrier)
    # ...
    incoming_headers = ['x-request-id']
    # ...
    for ihdr in incoming_headers:
        val = request.headers.get(ihdr)
        if val is not None:
            headers[ihdr] = val
    return headers

The reviews application (Java) does something similar:

@GET
@Path("/reviews/{productId}")
public Response bookReviewsById(@PathParam("productId") int productId,
                            @HeaderParam("end-user") String user,
                            @HeaderParam("x-request-id") String xreq,
                            @HeaderParam("x-b3-traceid") String xtraceid,
                            @HeaderParam("x-b3-spanid") String xspanid,
                            @HeaderParam("x-b3-parentspanid") String xparentspanid,
                            @HeaderParam("x-b3-sampled") String xsampled,
                            @HeaderParam("x-b3-flags") String xflags,
                            @HeaderParam("x-ot-span-context") String xotspan) {
  if (ratings_enabled) {
    JsonObject ratingsResponse = getRatings(Integer.toString(productId), user, xreq, xtraceid, xspanid, xparentspanid, xsampled, xflags, xotspan);

When you make downstream calls in your applications, make sure to include these headers.

Trace sampling

Istio captures a trace for all requests by default when installing with the demo profile.For example, when using the Bookinfo sample application above, every time you access/productpage you see a corresponding trace in thedashboard. This sampling rate is suitable for a test or low trafficmesh. For a high traffic mesh you can lower the trace samplingpercentage in one of two ways:

During the mesh setup, use the option values.pilot.traceSampling toset the percentage of trace sampling. See theInstalling with istioctl documentation fordetails on setting options.
In a running mesh, edit the istio-pilot deployment andchange the environment variable with the following steps:
- To open your text editor with the deployment configuration fileloaded, run the following command:

$ kubectl -n istio-system edit deploy istio-pilot

Find the PILOT_TRACE_SAMPLING environment variable, and changethe value: to your desired percentage.

In both cases, valid values are from 0.0 to 100.0 with a precision of 0.01.

Overview

Overview

Trace context propagation

Trace sampling

See also