Inject and Extract

Introduction

In order to trace across process boundaries and RPC calls in distributed systems, spanContext needs to propagated over the wire. The OpenTracing Java API provides two methods in the Tracer interface to do just that, inject(SpanContext, format, carrier) and extract(format, carrier).

Format Options and Carriers

The format parameter refers to one of the three standard encodings the OpenTracing API defines:

  1. TEXT_MAP where spanContext is encoded as a collection of string key-value pairs,
  2. BINARY where spanContext is encoded as an opaque byte array,
  3. HTTP_HEADERS, which is similar to TEXT_MAP except that the keys must be safe to be used as HTTP headers.

The carrier is an abstraction over the underlying RPC framework. For example, a carrier for TEXT_MAP format is an interface that allows the tracer to write key-value pairs via put(key, value) method, while a carrier for BINARY format is simply a ByteBuffer.

Injecting and Extracting HTTP

When your service calls a downstream service, it’s useful to pass down the SpanContext, so that Spans generated by this service could join the Spans from our service in a single trace. To do that, our service needs to Inject the SpanContext into the payload. On the other side of the connection, the downstream service needs then to Extract the SpanContext before it creates any Spans.

Note that manually dealing with the Inject and Extract operations is not common in Java: there are OpenTracing instrumentation libraries that take care of this for you, such as the java-web-servlet-filter for the Extract operation, or one of the following for the Inject operation:

You are still encouraged to read on, to understand what happens behind the scenes.

Injecting the spanContext to HTTP

In order to pass a spanContext over the HTTP request, the developer needs to call the tracer.inject before building the HTTP request, like so:

  1. Tags.SPAN_KIND.set(tracer.activeSpan(), Tags.SPAN_KIND_CLIENT);
  2. Tags.HTTP_METHOD.set(tracer.activeSpan(), "GET");
  3. Tags.HTTP_URL.set(tracer.activeSpan(), url.toString());
  4. tracer.inject(tracer.activeSpan().context(), Format.Builtin.HTTP_HEADERS, new RequestBuilderCarrier(requestBuilder));

Notice that a couple additional tags have been added to the span with some metadata about the HTTP request, following the OpenTracing Semantic Conventions. In this case the carrier is HTTP request headers object, which can be adapted to the carrier API by wrapping it in a helper class as follows:

  1. public class RequestBuilderCarrier implements io.opentracing.propagation.TextMap {
  2.     private final Request.Builder builder;
  4.     RequestBuilderCarrier(Request.Builder builder) {
  5.         this.builder = builder;
  6.     }
  8.     @Override
  9.     public Iterator<Map.Entry<String, String>> iterator() {
  10.         throw new UnsupportedOperationException("carrier is write-only");
  11.     }
  13.     @Override
  14.     public void put(String key, String value) {
  15.         builder.addHeader(key, value);
  16.     }
  17. }

Extracting the Span’s Context from Incoming Request

The logic on the client side instrumentation is similar, the only difference is that tracer.extract is used and the span is tagged as span.kind=server.

  1. public static Scope startServerSpan(Tracer tracer, javax.ws.rs.core.HttpHeaders httpHeaders,
  2.         String operationName) {
  3.     // format the headers for extraction
  4.     MultivaluedMap<String, String> rawHeaders = httpHeaders.getRequestHeaders();
  5.     final HashMap<String, String> headers = new HashMap<String, String>();
  6.     for (String key : rawHeaders.keySet()) {
  7.         headers.put(key, rawHeaders.get(key).get(0));
  8.     }
  10.     Tracer.SpanBuilder spanBuilder;
  11.     try {
  12.         SpanContext parentSpan = tracer.extract(Format.Builtin.HTTP_HEADERS, new TextMapExtractAdapter(headers));
  13.         if (parentSpan == null) {
  14.             spanBuilder = tracer.buildSpan(operationName);
  15.         } else {
  16.             spanBuilder = tracer.buildSpan(operationName).asChildOf(parentSpan);
  17.         }
  18.     } catch (IllegalArgumentException e) {
  19.         spanBuilder = tracer.buildSpan(operationName);
  20.     }
  21.     return spanBuilder.withTag(Tags.SPAN_KIND.getKey(), Tags.SPAN_KIND_SERVER).startActive(true);
  22. }

In the above example, instead of using a dedicated adapter class to convert JAX-RS HttpHeaders type into io.opentracing.propagation.TextMap, the headers are copied to a plain HashMap<String, String> and converted using a standard adapter TextMapExtractAdapter.

The complete tutorial is available here.

Injecting and Extracting TextMap

The process of injecting and extracting TextMap is similar to that of HTTP. Given below are some elaborated code examples of injecting and extracting tracing information using TextMap format.

  1. protected void attachTraceInfo(Tracer tracer, Span span, final Invocation inv) {
  2.     tracer.inject(span.context(), Format.Builtin.TEXT_MAP, new TextMap() {
  4.         @Override
  5.         public void put(String key, String value) {
  6.             inv.getAttachments().put(key, value);
  7.         }
  9.         @Override
  10.         public Iterator<Map.Entry<String, String>> iterator() {
  11.             throw new UnsupportedOperationException("TextMapInjectAdapter should only be used with Tracer.inject()");
  12.         }
  13.     });
  14. }

The above method injects the spanContext into the carrier on the consumer side.

  1. protected Span extractTraceInfo(Tracer tracer, Invoker<?> invoker, Invocation inv) {
  3.     Tracer.SpanBuilder span = tracer.buildSpan(“Operation_Name_Here”);
  4.     try {
  5.         SpanContext spanContext = tracer.extract(Format.Builtin.TEXT_MAP, new TextMapExtractAdapter(inv.getAttachments()));
  6.         if (spanContext != null) {
  7.             //if spanContext is extracted, the spanContext is propagated to the new span
  9.             span.asChildOf(spanContext);
  10.         }
  11.     } catch (Exception e) {
  12.         span.withTag("Error", "extract from request fail, error msg:" + e.getMessage());
  13.     }
  14.     return span.startManual();
  15. }

On the provider side, tracer.extract() is used to extract the spanContext. The logic is the reverse operation of the tracer.inject().

Injecting and Extracting BINARY

BINARY format is used for injecting/extracting spanContext encoded in an opaque byte array. Opaque means that the inner representation of the array is not known.

Injecting and extracting BINARY can be implemented as follows:

  1. //Inject binary
  2. public void byteBufferInjection() throws Exception {
  3.   byte[] key = "foo".getBytes(ByteBufferContext.CHARSET), value = "bar".getBytes(ByteBufferContext.CHARSET);
  4.   ByteBuffer byteBuffer = ByteBuffer.allocate(2 + 2 * 4 + key.length + value.length);
  5.   tracer.inject(spanContext, Format.Builtin.BINARY, byteBuffer);
  7. }
  9. //Extract binary
  10. public void byteBufferExtraction(ByteBuffer byteBuffer) throws Exception {
  11.   SpanContext spanContext = tracer.extract(Format.Builtin.BINARY, byteBuffer);
  12.   //do something with spanContext...
  13. }

The complete code is available here.