gRPC-JSON transcoder

This is a filter which allows a RESTful JSON API client to send requests to Envoy over HTTP and get proxied to a gRPC service. The HTTP mapping for the gRPC service has to be defined by custom options.

JSON mapping

The protobuf to JSON mapping is defined here. For gRPC stream request parameters, Envoy expects an array of messages, and it returns an array of messages for stream response parameters.

How to generate proto descriptor set

Envoy has to know the proto descriptor of your gRPC service in order to do the transcoding.

To generate a protobuf descriptor set for the gRPC service, you’ll also need to clone the googleapis repository from GitHub before running protoc, as you’ll need annotations.proto in your include path, to define the HTTP mapping.

  1. git clone https://github.com/googleapis/googleapis
  2. GOOGLEAPIS_DIR=<your-local-googleapis-folder>

Then run protoc to generate the descriptor set from bookstore.proto:

  1. protoc -I$(GOOGLEAPIS_DIR) -I. --include_imports --include_source_info \
  2. --descriptor_set_out=proto.pb test/proto/bookstore.proto

If you have more than one proto source files, you can pass all of them in one command.

Route configs for transcoded requests

The route configs to be used with the gRPC-JSON transcoder should be identical to the gRPC route. The requests processed by the transcoder filter will have /./ path and POST method. The route configs for those requests should match on /./, not the incoming request path. This allows the routes to be used for both gRPC requests and gRPC-JSON transcoded requests.

For example, with the following proto example, the router will process /helloworld.Greeter/SayHello as the path, so the route config prefix /say won’t match requests to SayHello. If you want to match the incoming request path, set match_incoming_request_route to true.

  1. package helloworld;
  2. // The greeting service definition.
  3. service Greeter {
  4. // Sends a greeting
  5. rpc SayHello (HelloRequest) returns (HelloReply) {
  6. option (google.api.http) = {
  7. get: "/say"
  8. };
  9. }
  10. }

Sending arbitrary content

By default, when transcoding occurs, gRPC-JSON encodes the message output of a gRPC service method into JSON and sets the HTTP response Content-Type header to application/json. To send arbitrary content, a gRPC service method can use google.api.HttpBody as its output message type. The implementation needs to set content_type (which sets the value of the HTTP response Content-Type header) and data (which sets the HTTP response body) accordingly. Multiple google.api.HttpBody can be send by the gRPC server in the server streaming case. In this case, HTTP response header Content-Type will use the content-type from the first google.api.HttpBody https://github.com/googleapis/googleapis/blob/master/google/api/httpbody.proto.

Headers

gRPC-JSON forwards the following headers to the gRPC server:

  • x-envoy-original-path, containing the value of the original path of HTTP request

  • x-envoy-original-method, containing the value of the original method of HTTP request

Sample Envoy configuration

Here’s a sample Envoy configuration that proxies to a gRPC server running on localhost:50051. Port 51051 proxies gRPC requests and uses the gRPC-JSON transcoder filter to provide the RESTful JSON mapping. I.e., you can make either gRPC or RESTful JSON requests to localhost:51051.

  1. admin:
  2. access_log_path: /tmp/admin_access.log
  3. address:
  4. socket_address: { address: 0.0.0.0, port_value: 9901 }
  5. static_resources:
  6. listeners:
  7. - name: listener1
  8. address:
  9. socket_address: { address: 0.0.0.0, port_value: 51051 }
  10. filter_chains:
  11. - filters:
  12. - name: envoy.filters.network.http_connection_manager
  13. typed_config:
  14. "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  15. stat_prefix: grpc_json
  16. codec_type: AUTO
  17. route_config:
  18. name: local_route
  19. virtual_hosts:
  20. - name: local_service
  21. domains: ["*"]
  22. routes:
  23. # NOTE: by default, matching happens based on the gRPC route, and not on the incoming request path.
  24. # Reference: https://www.envoyproxy.io/docs/envoy/latest/configuration/http_filters/grpc_json_transcoder_filter#route-configs-for-transcoded-requests
  25. - match: { prefix: "/helloworld.Greeter" }
  26. route: { cluster: grpc, timeout: { seconds: 60 } }
  27. http_filters:
  28. - name: envoy.filters.http.grpc_json_transcoder
  29. typed_config:
  30. "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder
  31. proto_descriptor: "/tmp/envoy/proto.pb"
  32. services: ["helloworld.Greeter"]
  33. print_options:
  34. add_whitespace: true
  35. always_print_primitive_fields: true
  36. always_print_enums_as_ints: false
  37. preserve_proto_field_names: false
  38. - name: envoy.filters.http.router
  39. clusters:
  40. - name: grpc
  41. connect_timeout: 1.25s
  42. type: logical_dns
  43. lb_policy: round_robin
  44. dns_lookup_family: V4_ONLY
  45. http2_protocol_options: {}
  46. load_assignment:
  47. cluster_name: grpc
  48. endpoints:
  49. - lb_endpoints:
  50. - endpoint:
  51. address:
  52. socket_address:
  53. # WARNING: "docker.for.mac.localhost" has been deprecated from Docker v18.03.0.
  54. # If you're running an older version of Docker, please use "docker.for.mac.localhost" instead.
  55. # Reference: https://docs.docker.com/docker-for-mac/release-notes/#docker-community-edition-18030-ce-mac59-2018-03-26
  56. address: host.docker.internal
  57. port_value: 50051