Examples

Below we will use YAML representation of the config protos and a running example of a service proxying HTTP from 127.0.0.1:10000 to 127.0.0.1:1234.

Static

A minimal fully static bootstrap config is provided below:

  1. admin:
  2.   address:
  3.     socket_address: { address: 127.0.0.1, port_value: 9901 }
  5. static_resources:
  6.   listeners:
  7.   - name: listener_0
  8.     address:
  9.       socket_address: { address: 127.0.0.1, port_value: 10000 }
  10.     filter_chains:
  11.     - filters:
  12.       - name: envoy.filters.network.http_connection_manager
  13.         typed_config:
  14.           "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  15.           stat_prefix: ingress_http
  16.           codec_type: AUTO
  17.           route_config:
  18.             name: local_route
  19.             virtual_hosts:
  20.             - name: local_service
  21.               domains: ["*"]
  22.               routes:
  23.               - match: { prefix: "/" }
  24.                 route: { cluster: some_service }
  25.           http_filters:
  26.           - name: envoy.filters.http.router
  27.             typed_config:
  28.               "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  29.   clusters:
  30.   - name: some_service
  31.     connect_timeout: 0.25s
  32.     type: STATIC
  33.     lb_policy: ROUND_ROBIN
  34.     load_assignment:
  35.       cluster_name: some_service
  36.       endpoints:
  37.       - lb_endpoints:
  38.         - endpoint:
  39.             address:
  40.               socket_address:
  41.                 address: 127.0.0.1
  42.                 port_value: 1234

Mostly static with dynamic EDS

A bootstrap config that continues from the above example with dynamic endpoint discovery via an EDS gRPC management server listening on 127.0.0.1:5678 is provided below:

  1. admin:
  2.   address:
  3.     socket_address: { address: 127.0.0.1, port_value: 9901 }
  5. static_resources:
  6.   listeners:
  7.   - name: listener_0
  8.     address:
  9.       socket_address: { address: 127.0.0.1, port_value: 10000 }
  10.     filter_chains:
  11.     - filters:
  12.       - name: envoy.filters.network.http_connection_manager
  13.         typed_config:
  14.           "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  15.           stat_prefix: ingress_http
  16.           codec_type: AUTO
  17.           route_config:
  18.             name: local_route
  19.             virtual_hosts:
  20.             - name: local_service
  21.               domains: ["*"]
  22.               routes:
  23.               - match: { prefix: "/" }
  24.                 route: { cluster: some_service }
  25.           http_filters:
  26.           - name: envoy.filters.http.router
  27.             typed_config:
  28.               "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  29.   clusters:
  30.   - name: some_service
  31.     connect_timeout: 0.25s
  32.     lb_policy: ROUND_ROBIN
  33.     type: EDS
  34.     eds_cluster_config:
  35.       eds_config:
  36.         resource_api_version: V3
  37.         api_config_source:
  38.           api_type: GRPC
  39.           transport_api_version: V3
  40.           grpc_services:
  41.             - envoy_grpc:
  42.                 cluster_name: xds_cluster
  43.   - name: xds_cluster
  44.     connect_timeout: 0.25s
  45.     type: STATIC
  46.     lb_policy: ROUND_ROBIN
  47.     typed_extension_protocol_options:
  48.       envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
  49.         "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
  50.         explicit_http_config:
  51.           http2_protocol_options:
  52.             connection_keepalive:
  53.               interval: 30s
  54.               timeout: 5s
  55.     upstream_connection_options:
  56.       # configure a TCP keep-alive to detect and reconnect to the admin
  57.       # server in the event of a TCP socket half open connection
  58.       tcp_keepalive: {}
  59.     load_assignment:
  60.       cluster_name: xds_cluster
  61.       endpoints:
  62.       - lb_endpoints:
  63.         - endpoint:
  64.             address:
  65.               socket_address:
  66.                 address: 127.0.0.1
  67.                 port_value: 5678

Notice above that xds_cluster is defined to point Envoy at the management server. Even in an otherwise completely dynamic configurations, some static resources need to be defined to point Envoy at its xDS management server(s).

It’s important to set appropriate TCP Keep-Alive options in the tcp_keepalive block. This will help detect TCP half open connections to the xDS management server and re-establish a full connection.

In the above example, the EDS management server could then return a proto encoding of a DiscoveryResponse:

  1. version_info: "0"
  2. resources:
  3. - "@type": type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment
  4.   cluster_name: some_service
  5.   endpoints:
  6.   - lb_endpoints:
  7.     - endpoint:
  8.         address:
  9.           socket_address:
  10.             address: 127.0.0.1
  11.             port_value: 1234

The versioning and type URL scheme that appear above are explained in more detail in the streaming gRPC subscription protocol documentation.

Dynamic

A fully dynamic bootstrap configuration, in which all resources other than those belonging to the management server are discovered via xDS is provided below:

  1. admin:
  2.   address:
  3.     socket_address: { address: 127.0.0.1, port_value: 9901 }
  5. dynamic_resources:
  6.   lds_config:
  7.     resource_api_version: V3
  8.     api_config_source:
  9.       api_type: GRPC
  10.       transport_api_version: V3
  11.       grpc_services:
  12.         - envoy_grpc:
  13.             cluster_name: xds_cluster
  14.   cds_config:
  15.     resource_api_version: V3
  16.     api_config_source:
  17.       api_type: GRPC
  18.       transport_api_version: V3
  19.       grpc_services:
  20.         - envoy_grpc:
  21.             cluster_name: xds_cluster
  23. static_resources:
  24.   clusters:
  25.   - name: xds_cluster
  26.     connect_timeout: 0.25s
  27.     type: STATIC
  28.     lb_policy: ROUND_ROBIN
  29.     typed_extension_protocol_options:
  30.       envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
  31.         "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
  32.         explicit_http_config:
  33.           http2_protocol_options:
  34.             # Configure an HTTP/2 keep-alive to detect connection issues and reconnect
  35.             # to the admin server if the connection is no longer responsive.
  36.             connection_keepalive:
  37.               interval: 30s
  38.               timeout: 5s
  39.     load_assignment:
  40.       cluster_name: xds_cluster
  41.       endpoints:
  42.       - lb_endpoints:
  43.         - endpoint:
  44.             address:
  45.               socket_address:
  46.                 address: 127.0.0.1
  47.                 port_value: 5678

The management server could respond to LDS requests with:

  1. version_info: "0"
  2. resources:
  3. - "@type": type.googleapis.com/envoy.config.listener.v3.Listener
  4.   name: listener_0
  5.   address:
  6.     socket_address:
  7.       address: 127.0.0.1
  8.       port_value: 10000
  9.   filter_chains:
  10.   - filters:
  11.     - name: envoy.filters.network.http_connection_manager
  12.       typed_config:
  13.         "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  14.         stat_prefix: ingress_http
  15.         codec_type: AUTO
  16.         rds:
  17.           route_config_name: local_route
  18.           config_source:
  19.             resource_api_version: V3
  20.             api_config_source:
  21.               api_type: GRPC
  22.               transport_api_version: V3
  23.               grpc_services:
  24.                 - envoy_grpc:
  25.                     cluster_name: xds_cluster
  26.         http_filters:
  27.         - name: envoy.filters.http.router
  28.           typed_config:
  29.             "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

The management server could respond to RDS requests with:

  1. version_info: "0"
  2. resources:
  3. - "@type": type.googleapis.com/envoy.config.route.v3.RouteConfiguration
  4.   name: local_route
  5.   virtual_hosts:
  6.   - name: local_service
  7.     domains: ["*"]
  8.     routes:
  9.     - match: { prefix: "/" }
  10.       route: { cluster: some_service }

The management server could respond to CDS requests with:

  1. version_info: "0"
  2. resources:
  3. - "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster
  4.   name: some_service
  5.   connect_timeout: 0.25s
  6.   lb_policy: ROUND_ROBIN
  7.   type: EDS
  8.   eds_cluster_config:
  9.     eds_config:
  10.       resource_api_version: V3
  11.       api_config_source:
  12.         api_type: GRPC
  13.         transport_api_version: V3
  14.         grpc_services:
  15.           - envoy_grpc:
  16.               cluster_name: xds_cluster

The management server could respond to EDS requests with:

  1. version_info: "0"
  2. resources:
  3. - "@type": type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment
  4.   cluster_name: some_service
  5.   endpoints:
  6.   - lb_endpoints:
  7.     - endpoint:
  8.         address:
  9.           socket_address:
  10.             address: 127.0.0.1
  11.             port_value: 1234

Special YAML usage

When loading YAML configuration, the Envoy loader will interpret map keys tagged with !ignore specially, and omit them entirely from the native configuration tree. Ordinarily, the YAML stream must adhere strictly to the proto schemas defined for Envoy configuration. This allows content to be declared that is explicitly handled as a non-represented type.

This lets you split your file into two parts: one in which we have YAML content not subject to parsing according to the schema and another part that is parsed. YAML anchors in the first part may be referenced by aliases in the second part. This mechanism can simplify setups that need to re-use or dynamically generate configuration fragments.

See the following example:

  1. !ignore dynamic_sockets:
  2. - &admin_address {address: 127.0.0.1, port_value: 9901}
  3. - &listener_address {address: 127.0.0.1, port_value: 10000}
  4. - &lb_address {address: 127.0.0.1, port_value: 1234}
  6. admin:
  7.   address:
  8.     socket_address: *admin_address
  10. static_resources:
  11.   listeners:
  12.   - name: listener_0
  13.     address:
  14.       socket_address: *listener_address
  15.     filter_chains:
  16.     - filters:
  17.       - name: envoy.filters.network.http_connection_manager
  18.         typed_config:
  19.           "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  20.           stat_prefix: ingress_http
  21.           codec_type: AUTO
  22.           route_config:
  23.             name: local_route
  24.             virtual_hosts:
  25.             - name: local_service
  26.               domains: ["*"]
  27.               routes:
  28.               - match: {prefix: "/"}
  29.                 route: {cluster: some_service}
  30.           http_filters:
  31.           - name: envoy.filters.http.router
  32.             typed_config:
  33.               "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
  34.   clusters:
  35.   - name: some_service
  36.     connect_timeout: 0.25s
  37.     type: STATIC
  38.     lb_policy: ROUND_ROBIN
  39.     load_assignment:
  40.       cluster_name: some_service
  41.       endpoints:
  42.       - lb_endpoints:
  43.         - endpoint:
  44.             address:
  45.               socket_address: *lb_address

Warning

If you parse Envoy YAML configuration using external loaders, you may need to inform these loaders about the !ignore tag. Compliant YAML loaders will typically expose an interface to allow you to choose how to handle a custom tag.

For example, this will instruct PyYAML to treat an ignored node as a simple scalar when loading:

  1. yaml.SafeLoader.add_constructor('!ignore', yaml.loader.SafeConstructor.construct_scalar)

Alternatively, this is how Envoy registers the !ignore tag in config validation.