Register a Service Mesh Proxy Outside of a Service Registration

This topic describes how to declare a service mesh proxy in a service definition. The kind must be declared and information about the service they represent must be provided to function as a Consul service mesh proxy.

Configuration

Configure a service mesh proxy using the following syntax:

Basic syntax for configuring a service mesh proxy

Basic syntax for configuring a service mesh proxy

name =  <name of the service>
kind = "connect-proxy"
proxy = {
  destination_service_name = "<name of the service that the proxy represents>"
  <additional proxy parameters> = "<additional parameter values>"
}
port = <port where services can discover and connect to proxied services>

{
  "name": "<name of the service>",
  "kind": "connect-proxy",
  "proxy": {
    "destination_service_name": "<name of the service that the proxy represents>",
    "<additional proxy parameters>" : "<additional parameter values>"
  },
  "port": <port where services can discover and connect to proxied services>
}

The following table describes the parameters that must be added to the service definition to declare the service as a proxy.

You can specify several additional parameters to configure the proxy to meet your requirements. See Proxy Parameters for additional information.

Example

In the following example, a proxy named redis-proxy is registered as a service mesh proxy. It proxies to the redis service and is available at port 8181. As a result, any service mesh clients searching for a mesh-capable endpoint for redis will find this proxy.

Minimal example of a service mesh proxy

Minimal example of a service mesh proxy

kind = "connect-proxy"
name = "redis-proxy"
port = 8181
proxy = {
  destination_service_name = "redis"
}

{
  "name": "redis-proxy",
  "kind": "connect-proxy",
  "proxy": {
    "destination_service_name": "redis"
  },
  "port": 8181
}

Sidecar Proxy Configuration

Many service mesh proxies are deployed as sidecars. Sidecar proxies are co-located with the single service instance they represent and proxy all inbound traffic to.

Specify the following parameters in the proxy code block to configure a sidecar proxy in its own service registration:

• destination_service_id: String value that specifies the ID of the service being proxied. Refer to the proxy parameters reference for details.
• local_service_port: Integer value that specifies the port that the proxy should use to connect to the local service instance. Refer to the proxy parameters reference for details.
• local_service_address: String value that specifies the IP address or hostname that the proxy should use to connect to the local service. Refer to the proxy parameters reference for details.

See Sidecar Service Registration for additional information about configuring service mesh proxies as sidecars.

Complete Configuration Example

The following example includes values for all available options when registering a proxy instance.

Example that includes all configuration options when registering a proxy instance

Example that includes all configuration options when registering a proxy instance

kind = "connect-proxy"
name = "redis-proxy"
port = 8181
proxy = {
  config = {}
  destination_service_id = "redis1"
  destination_service_name = "redis"
  expose = {}
  local_service_address = "127.0.0.1"
  local_service_port = 9090
  local_service_socket_path = "/tmp/redis.sock"
  mesh_gateway = {}
  mode = "transparent"
  transparent_proxy = {}
  upstreams = []
}

{
  "name": "redis-proxy",
  "kind": "connect-proxy",
  "proxy": {
    "destination_service_name": "redis",
    "destination_service_id": "redis1",
    "local_service_address": "127.0.0.1",
    "local_service_port": 9090,
    "local_service_socket_path": "/tmp/redis.sock",
    "mode": "transparent",
    "transparent_proxy": {},
    "config": {},
    "upstreams": [],
    "mesh_gateway": {},
    "expose": {}
  },
  "port": 8181
}

Proxy Parameters

The following table describes all parameters that can be defined in the proxy block.

Upstream Configuration Reference

You can configure the service mesh proxy to create listeners for upstream services. The listeners enable the upstream service to accept requests. You can specify the following parameters to configure upstream service listeners.

Parameter	Description	Required	Default
destination_name	String value that specifies the name of the service or prepared query to route the service mesh to. The prepared query should be the name or the ID of the prepared query.	Required	None
destination_namespace	String value that specifies the namespace containing the upstream service. EnterpriseEnterprise	Optional	default
destination_peer	String value that specifies the name of the peer cluster containing the upstream service.	Optional	None
destination_partition	String value that specifies the name of the admin partition containing the upstream service. If destination_peer is set, destination_partition refers to the local admin partition in which the peering was established. EnterpriseEnterprise	Optional	default
local_bind_port	Integer value that specifies the port to bind a local listener to. The application will make outbound connections to the upstream from the local port.	Required	None
local_bind_address	String value that specifies the address to bind a local listener to. The application will make outbound connections to the upstream service from the local bind address.	Optional	127.0.0.1
local_bind_socket_path	String value that specifies the path at which to bind a Unix domain socket listener. The application will make outbound connections to the upstream from the local bind socket path. This parameter conflicts with the local_bind_port or local_bind_address parameters. Supported when using Envoy as a proxy.	Optional	None
local_bind_socket_mode	String value that specifies a Unix octal that configures file permissions for the socket.	Optional	None
destination_type	String value that specifies the type of discovery query the proxy should use for finding service mesh instances. The following values are supported: service: Queries for upstream service types. prepared_query: Queries for upstream prepared queries.	Optional	service
datacenter	String value that specifies the datacenter to issue the discovery query to.	Optional	Defaults to the local datacenter.
config	Object value that specifies opaque configuration options that will be provided to the proxy instance for the upstream. Valid JSON objects are also supported. The config parameter can specify timeouts, retries, and other proxy-specific features for the given upstream. See the built-in proxy configuration reference for configuration options when using the built-in proxy. If using Envoy as a proxy, see Envoy configuration reference	Optional	None
mesh_gateway	Object that defines the mesh gateway configuration for the proxy. Refer to the Mesh Gateway Configuration Reference for configuration details.	Optional	None

Upstream Configuration Examples

Upstreams support multiple destination types. The following examples include information about each implementation. Note that the examples in this topic use snake case, which is a convention that separates words with underscores, because the format is supported in configuration files and API registrations.

Example service destination upstream

Example service destination upstream

destination_type = "service"
destination_name = "redis"
datacenter = "dc1"
local_bind_address = "127.0.0.1"
local_bind_port = 1234
local_bind_socket_path = "/tmp/redis_5678.sock"
local_bind_socket_mode = "0700"
mesh_gateway = {
    mode = "local"
}

{
  "destination_type": "service",
  "destination_name": "redis",
  "datacenter": "dc1",
  "local_bind_address": "127.0.0.1",
  "local_bind_port": 1234,
  "local_bind_socket_path": "/tmp/redis_5678.sock",
  "local_bind_socket_mode": "0700",
  "mesh_gateway": {
    "mode": "local"
  }
},

Example prepared query upstream

Example prepared query upstream

destination_type = "prepared_query"
destination_name = "database"
local_bind_address = "127.0.0.1"
local_bind_port = 1234
config = {}

{
  "destination_type": "prepared_query",
  "destination_name": "database",
  "local_bind_address": "127.0.0.1",
  "local_bind_port": 1234,
  "config": {}
},

Example of dialing remote upstreams across admin partitions

Example of dialing remote upstreams across admin partitions

destination_partition = "finance"
destination_namespace = "default"
destination_type = "service"
destination_name = "billing"
local_bind_port = 9090

{
  "destination_partition": "finance",
  "destination_namespace": "default",
  "destination_type": "service",
  "destination_name": "billing",
  "local_bind_port": 9090
}

Example of dialing remote upstreams across peers

Example of dialing remote upstreams across peers

destination_peer = "cloud-services"
destination_partition = "finance"
destination_namespace = "default"
destination_type = "service"
destination_name = "api"
local_bind_port = 9090

{
  "destination_peer": "cloud-services",
  "destination_partition": "finance",
  "destination_namespace": "default",
  "destination_type": "service",
  "destination_name": "api",
  "local_bind_port": 9090
}

Proxy Modes

You can configure which mode a proxy operates in by specifying "direct" or "transparent" in the mode parameter. The proxy mode determines the how proxies direct traffic. This feature was added in Consul 1.10.0.

• transparent: In this mode, inbound and outbound application traffic is captured and redirected through the proxy. This mode does not enable the traffic redirection. It directs Consul to configure Envoy as if traffic is already being redirected.
• direct: In this mode, the proxy’s listeners must be dialed directly by the local application and other proxies.

You can also specify an empty string (""), which configures the proxy to operate in the default mode. The default mode is inherited from parent parameters in the following order of precedence:

1. Proxy service’s Proxy configuration
2. The service-defaults configuration for the service.
3. The global proxy-defaults.

The proxy will default to direct mode if a mode cannot be determined from the parent parameters.

Transparent Proxy Configuration Reference

The following examples show additional configuration for transparent proxies.

Note that the examples in this topic use snake case, which is a convention that separates words with underscores, because the format is supported in configuration files and API registrations.

Configure a proxy listener for outbound traffic on port 22500

{
  "outbound_listener_port": 22500,
  "dialed_directly": true
}

• outbound_listener_port (int: 15001) - The port the proxy should listen on for outbound traffic. This must be the port where outbound application traffic is captured and redirected to.

• dialed_directly (bool: false) - Determines whether this proxy instance’s IP address can be dialed directly by transparent proxies. Transparent proxies typically dial upstreams using the “virtual” tagged address, which load balances across instances. A database cluster with a leader is an example where dialing individual instances can be helpful. Cannot be used with upstreams which define a destination_peer.

  Note: Dynamic routing rules such as failovers and redirects do not apply to services dialed directly. Additionally, the connection is proxied using a TCP proxy with a connection timeout of 5 seconds.

Mesh Gateway Configuration Reference

The following examples show all possible mesh gateway configurations.

Note that the examples in this topic use snake case, which is a convention that separates words with underscores, because the format is supported in configuration files and API registrations.

Using a Local/Egress Gateway in the Local Datacenter

{
  "mode": "local"
}

Direct to a Remote/Ingress in a Remote Datacenter

{
  "mode": "remote"
}

Prevent Using a Mesh Gateway

{
  "mode": "none"
}

Default Mesh Gateway Mode

{
  "mode": ""
}

• mode (string: "") - This defines the mode of operation for how upstreams with a remote destination datacenter get resolved.
  • “local” - Mesh gateway services in the local datacenter will be used as the next-hop destination for the upstream connection.
  • “remote” - Mesh gateway services in the remote/target datacenter will be used as the next-hop destination for the upstream connection.
  • “none” - No mesh gateway services will be used and the next-hop destination for the connection will be directly to the final service(s).
  • “” - Default mode. The default mode will be "none" if no other configuration enables them. The order of precedence for setting the mode is
    1. Upstream
    2. Proxy Service’s Proxy configuration
    3. The service-defaults configuration for the service.
    4. The global proxy-defaults.

Expose Paths Configuration Reference

The following examples show possible configurations to expose HTTP paths through Envoy.

Exposing paths through Envoy enables a service to protect itself by only listening on localhost, while still allowing non-mesh-enabled applications to contact an HTTP endpoint. Some examples include: exposing a /metrics path for Prometheus or /healthz for kubelet liveness checks.

Note that the examples in this topic use snake case, which is a convention that separates words with underscores, because the format is supported in configuration files and API registrations.

Expose listeners in Envoy for HTTP and GRPC checks registered with the local Consul agent

{
  "expose": {
    "checks": true
  }
}

Expose an HTTP listener in Envoy at port 21500 that routes to an HTTP server listening at port 8080

{
  "expose": {
    "paths": [
      {
        "path": "/healthz",
        "local_path_port": 8080,
        "listener_port": 21500
      }
    ]
  }
}

Expose an HTTP2 listener in Envoy at port 21501 that routes to a gRPC server listening at port 9090

{
  "expose": {
    "paths": [
      {
        "path": "/grpc.health.v1.Health/Check",
        "protocol": "http2",
        "local_path_port": 9090,
        "listener_port": 21501
      }
    ]
  }
}

• checks (bool: false) - If enabled, all HTTP and gRPC checks registered with the agent are exposed through Envoy. Envoy will expose listeners for these checks and will only accept connections originating from localhost or Consul’s advertise address. The port for these listeners are dynamically allocated from expose_min_port to expose_max_port. This flag is useful when a Consul client cannot reach registered services over localhost. One example is when running Consul on Kubernetes, and Consul agents run in their own pods.
• paths array<Path>: [] - A list of paths to expose through Envoy.
  • path (string: "") - The HTTP path to expose. The path must be prefixed by a slash. ie: /metrics.
  • local_path_port (int: 0) - The port where the local service is listening for connections to the path.
  • listener_port (int: 0) - The port where the proxy will listen for connections. This port must be available for the listener to be set up. If the port is not free then Envoy will not expose a listener for the path, but the proxy registration will not fail.
  • protocol (string: "http") - Sets the protocol of the listener. One of http or http2. For gRPC use http2.

Unix Domain Sockets

The following examples show additional configuration for Unix domain sockets.

Added in v1.10.0.

To connect to a service via local Unix Domain Socket instead of a port, add local_bind_socket_path and optionally local_bind_socket_mode to the upstream config for a service:

upstreams = [
  {
    destination_name = "service-1"
    local_bind_socket_path = "/tmp/socket_service_1"
    local_bind_socket_mode = "0700"
  }
]

"upstreams": [
  {
      "destination_name":  "service-1",
      "local_bind_socket_path": "/tmp/socket_service_1",
      "local_bind_socket_mode": "0700"
  }
]

This will cause Envoy to create a socket with the path and mode provided, and connect that to service-1.

The mode field is optional, and if omitted will use the default mode for Envoy. This is not applicable for abstract sockets. See the Envoy documentation for details.

These options conflict with the local_bind_socket_port and local_bind_socket_address options. For a given upstream the proxy can bind either to an IP port or a Unix socket, but not both.

Similarly to expose a service listening on a Unix Domain socket to the service mesh, use either the socket_path field in the service definition or the local_service_socket_path field in the proxy definition. These fields are analogous to the port and service_port fields in their respective locations.

services {
  name = "service-2"
  socket_path = "/tmp/socket_service_2"
}

"services": {
  "name": "service-2",
  "socket_path": "/tmp/socket_service_2"
}

Or in the proxy definition:

services {
  name = "socket_service_2"
  connect {
    sidecar_service {
      proxy {
        name = "service-2"
        local_service_socket_path = "/tmp/socket_service_2"
    ...
      }
    }
  }
}

"services": {
  "name": "socket_service_2",
  "connect": {
    "sidecar_service": {
      "proxy": {
        "name": "service-2",
        "local_service_socket_path": "/tmp/socket_service_2"
        ...
      }
    }
  }
}

There is no mode field since the service is expected to create the socket it is listening on, not the Envoy proxy. Again, the socket_path and local_service_socket_path fields conflict with address/port and local_service_address/local_service_port configuration options.