Mesh Gateways

1.6.0+: This feature is available in Consul versions 1.6.0 and newer.

Mesh gateways enable routing of Connect traffic between different Consul datacenters. Those datacenters can reside in different clouds or runtime environments where general interconnectivity between all services in all datacenters isn’t feasible. These gateways operate by sniffing the SNI header out of the Connect session and then route the connection to the appropriate destination based on the server name requested. The data within the mTLS session is not decrypted by the Gateway.

Mesh Gateway Architecture

For a complete example of how to connect services across datacenters, review the mesh gateway tutorial.

Prerequisites

Each mesh gateway needs three things:

  1. A local Consul agent to manage its configuration.
  2. General network connectivity to all services within its local Consul datacenter.
  3. General network connectivity to all mesh gateways within remote Consul datacenters.

Mesh gateways also require that your Consul datacenters are configured correctly:

  • Consul version 1.6.0 or newer is required.
  • Consul Connect must be enabled in both datacenters.
  • Each of your datacenters must have a unique name.
  • Your datacenters must be WAN joined.
  • The primary datacenter must be set to the same value in both datacenters. This specifies which datacenter is the authority for Connect certificates and is required for services in all datacenters to establish mutual TLS with each other.
  • gRPC must be enabled.
  • If you want to enable gateways globally you must enable centralized configuration.

Currently, Envoy is the only proxy with mesh gateway capabilities in Consul.

  • Mesh gateway proxies receive their configuration through Consul, which automatically generates it based on the proxy’s registration. Currently Consul can only translate mesh gateway registration information into Envoy configuration, therefore the proxies acting as mesh gateways must be Envoy.

  • Sidecar proxies that send traffic to an upstream service through a gateway need to know the location of that gateway. They discover the gateway based on their sidecar proxy registrations. Consul can only translate the gateway registration information into Envoy configuration, so any sidecars that send upstream traffic through a gateway must be Envoy.

Sidecar proxies that don’t send upstream traffic through a gateway aren’t affected when you deploy gateways. If you are using Consul’s built-in proxy as a Connect sidecar it will continue to work for intra-datacenter traffic and will receive incoming traffic even if that traffic has passed through a gateway.

Modes of Operation

Each upstream of a Connect proxy can be configured to be routed through a mesh gateway. Depending on your network, the proxy’s connection to the gateway can happen in one of the following modes illustrated in the diagram above:

  • local - In this mode the Connect proxy makes its outbound connection to a gateway running in the same datacenter. That gateway is then responsible for ensuring the data gets forwarded along to gateways in the destination datacenter. This is the mode of operation depicted in the diagram at the beginning of the page.

  • remote - In this mode the Connect proxy makes its outbound connection to a gateway running in the destination datacenter. That gateway will then forward the data to the final destination service.

  • none - In this mode, no gateway is used and a Connect proxy makes its outbound connections directly to the destination services.

Mesh Gateway Configuration

Mesh gateways are defined similarly to other services registered with Consul, with two exceptions. The first is that the service kind must be “mesh-gateway”. Second, the mesh gateway service definition may contain a Proxy.Config entry, just like a Connect proxy service, to define opaque configuration parameters useful for the actual proxy software. For Envoy there are some supported gateway options as well as escape-hatch overrides.

Note: If ACLs are enabled, a token granting service:write for the gateway’s service name and service:read for all services in the datacenter must be added to the gateway’s service definition. These permissions authorize the token to route communications for other Connect services but does not allow decrypting any of their communications.

Connect Proxy Configuration

Configuring a Connect Proxy to use gateways is as simple as setting its mode of operation. This can be done in several different places allowing for global to more fine grained control. If the gateway mode is configured in multiple locations the order of precedence is as follows

  1. Upstream Definition
  2. Service Instance Definition
  3. Centralized service-defaults configuration entry
  4. Centralized proxy-defaults configuration entry.

Enabling Gateways Globally

The following proxy-defaults configuration will enable gateways for all Connect services in the local mode.

  1. Kind = "proxy-defaults"
  2. Name = "global"
  3. MeshGateway {
  4.    Mode = "local"
  5. }

Enabling Gateways Per-Service

The following service-defaults configuration will enable gateways for all Connect services with the name “web”.

  1. Kind = "service-defaults"
  2. Name = "web"
  3. MeshGateway {
  4.    Mode = "local"
  5. }

Enabling Gateways for a Service Instance

The following Proxy Service Registration definition will enable gateways for the service instance in the remote mode.

  1. service {
  2.    name = "web-sidecar-proxy"
  3.    kind = "connect-proxy"
  4.    port = 8181
  5.    proxy {
  6.       destination_service_name = "web"
  7.       mesh_gateway {
  8.          mode = "remote"
  9.       }
  10.       upstreams = [
  11.          {
  12.             destination_name = "api"
  13.             datacenter = "secondary"
  14.             local_bind_port = 10000
  15.          }
  16.       ]
  17.    }
  18. }

Or alternatively inline with the service definition:

  1. service {
  2.   name = "web"
  3.   port = 8181
  4.   connect {
  5.     sidecar_service {
  6.       proxy {
  7.         mesh_gateway {
  8.          mode = "remote"
  9.         }
  10.         upstreams = [
  11.           {
  12.             destination_name = "api"
  13.             datacenter = "secondary"
  14.             local_bind_port = 10000
  15.           }
  16.         ]
  17.       }
  18.     }
  19.   }
  20. }

Enabling Gateways for a Proxy Upstream

The following service definition will enable gateways in the local mode for one upstream, the remote mode for a second upstream and will disable gateways for a third upstream.

  1. service {
  2.    name = "web-sidecar-proxy"
  3.    kind = "connect-proxy"
  4.    port = 8181
  5.    proxy {
  6.       destination_service_name = "web"
  7.       upstreams = [
  8.          {
  9.             destination_name = "api"
  10.             local_bind_port = 10000
  11.             mesh_gateway {
  12.                mode = "remote"
  13.             }
  14.          },
  15.          {
  16.             destination_name = "db"
  17.             local_bind_port = 10001
  18.             mesh_gateway {
  19.                mode = "local"
  20.             }
  21.          },
  22.          {
  23.             destination_name = "logging"
  24.             local_bind_port = 10002
  25.             mesh_gateway {
  26.                mode = "none"
  27.             }
  28.          },
  29.       ]
  30.    }
  31. }