TLS

Envoy supports both TLS termination in listeners as well as TLS origination when making connections to upstream clusters. Support is sufficient for Envoy to perform standard edge proxy duties for modern web services as well as to initiate connections with external services that have advanced TLS requirements (TLS1.2, SNI, etc.). Envoy supports the following TLS features:

  • Configurable ciphers: Each TLS listener and client can specify the ciphers that it supports.

  • Client certificates: Upstream/client connections can present a client certificate in addition to server certificate verification.

  • Certificate verification and pinning: Certificate verification options include basic chain verification, subject name verification, and hash pinning.

  • Certificate revocation: Envoy can check peer certificates against a certificate revocation list (CRL) if one is provided.

  • ALPN: TLS listeners support ALPN. The HTTP connection manager uses this information (in addition to protocol inference) to determine whether a client is speaking HTTP/1.1 or HTTP/2.

  • SNI: SNI is supported for both server (listener) and client (upstream) connections.

  • Session resumption: Server connections support resuming previous sessions via TLS session tickets (see RFC 5077). Resumption can be performed across hot restarts and between parallel Envoy instances (typically useful in a front proxy configuration).

  • BoringSSL private key methods: TLS private key operations (signing and decrypting) can be performed asynchronously from an extension. This allows extending Envoy to support various key management schemes (such as TPM) and TLS acceleration. This mechanism uses BoringSSL private key method interface.

Underlying implementation

Currently Envoy is written to use BoringSSL as the TLS provider.

FIPS 140-2

BoringSSL can be built in a FIPS-compliant mode, following the build instructions from the Security Policy for BoringCrypto module, using --define boringssl=fips Bazel option. Currently, this option is only available on Linux-x86_64.

The correctness of the FIPS build can be verified by checking the presence of BoringSSL-FIPS in the --version output.

It’s important to note that while using FIPS-compliant module is necessary for FIPS compliance, it’s not sufficient by itself, and depending on the context, additional steps might be necessary. The extra requirements may include using only approved algorithms and/or using only private keys generated by a module operating in FIPS-approved mode. For more information, please refer to the Security Policy for BoringCrypto module and/or an accredited CMVP laboratory.

Please note that the FIPS-compliant build is based on an older version of BoringSSL than the non-FIPS build, and it predates the final version of TLS 1.3.

Enabling certificate verification

Certificate verification of both upstream and downstream connections is not enabled unless the validation context specifies one or more trusted authority certificates.

Example configuration

  1. static_resources:
  2.   listeners:
  3.   - name: listener_0
  4.     address: { socket_address: { address: 127.0.0.1, port_value: 10000 } }
  5.     filter_chains:
  6.     - filters:
  7.       - name: envoy.filters.network.http_connection_manager
  8.         # ...
  9.       transport_socket:
  10.         name: envoy.transport_sockets.tls
  11.         typed_config:
  12.           "@type": type.googleapis.com/envoy.api.v2.auth.DownstreamTlsContext
  13.           common_tls_context:
  14.             validation_context:
  15.               trusted_ca:
  16.                 filename: /usr/local/my-client-ca.crt
  17.   clusters:
  18.   - name: some_service
  19.     connect_timeout: 0.25s
  20.     type: STATIC
  21.     lb_policy: ROUND_ROBIN
  22.     load_assignment:
  23.       cluster_name: some_service
  24.       endpoints:
  25.       - lb_endpoints:
  26.         - endpoint:
  27.             address:
  28.               socket_address:
  29.                 address: 127.0.0.2
  30.                 port_value: 1234
  31.     transport_socket:
  32.       name: envoy.transport_sockets.tls
  33.       typed_config:
  34.         "@type": type.googleapis.com/envoy.api.v2.auth.UpstreamTlsContext
  35.         common_tls_context:
  36.           tls_certificates:
  37.             certificate_chain: { "filename": "/cert.crt" }
  38.             private_key: { "filename": "/cert.key" }
  39.           validation_context:
  40.             verify_subject_alt_name: [ foo ]
  41.             trusted_ca:
  42.               filename: /etc/ssl/certs/ca-certificates.crt

/etc/ssl/certs/ca-certificates.crt is the default path for the system CA bundle on Debian systems. trusted_ca along with verify_subject_alt_name makes Envoy verify the server identity of 127.0.0.2:1234 as “foo” in the same way as e.g. cURL does on standard Debian installations. Common paths for system CA bundles on Linux and BSD are:

  • /etc/ssl/certs/ca-certificates.crt (Debian/Ubuntu/Gentoo etc.)

  • /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem (CentOS/RHEL 7)

  • /etc/pki/tls/certs/ca-bundle.crt (Fedora/RHEL 6)

  • /etc/ssl/ca-bundle.pem (OpenSUSE)

  • /usr/local/etc/ssl/cert.pem (FreeBSD)

  • /etc/ssl/cert.pem (OpenBSD)

See the reference for UpstreamTlsContexts and DownstreamTlsContexts for other TLS options.

Attention

If only trusted_ca is specified, Envoy will verify the certificate chain of the presented certificate, but not its subject name, hash, etc. Other validation context configuration is typically required depending on the deployment.

Certificate selection

DownstreamTlsContexts support multiple TLS certificates. These may be a mix of RSA and P-256 ECDSA certificates. The following rules apply:

  • Only one certificate of a particular type (RSA or ECDSA) may be specified.

  • Non-P-256 server ECDSA certificates are rejected.

  • If the client supports P-256 ECDSA, a P-256 ECDSA certificate will be selected if present in the DownstreamTlsContext.

  • If the client only supports RSA certificates, a RSA certificate will be selected if present in the DownstreamTlsContext.

  • Otherwise, the first certificate listed is used. This will result in a failed handshake if the client only supports RSA certificates and the server only has ECDSA certificates.

  • Static and SDS certificates may not be mixed in a given DownstreamTlsContext.

Only a single TLS certificate is supported today for UpstreamTlsContexts.

Secret discovery service (SDS)

TLS certificates can be specified in the static resource or can be fetched remotely. Certificate rotation is supported for static resources by sourcing SDS configuration from the filesystem or by pushing updates from the SDS server. Please see SDS for details.

Authentication filter

Envoy provides a network filter that performs TLS client authentication via principals fetched from a REST VPN service. This filter matches the presented client certificate hash against the principal list to determine whether the connection should be allowed or not. Optional IP white listing can also be configured. This functionality can be used to build edge proxy VPN support for web infrastructure.

Client TLS authentication filter configuration reference.

Trouble shooting

When Envoy originates TLS when making connections to upstream clusters, any errors will be logged into UPSTREAM_TRANSPORT_FAILURE_REASON field or AccessLogCommon.upstream_transport_failure_reason field. Common errors are:

  • Secret is not supplied by SDS: Envoy is still waiting SDS to deliver key/cert or root CA.

  • SSLV3_ALERT_CERTIFICATE_EXPIRED: Peer certificate is expired and not allowed in config.

  • SSLV3_ALERT_CERTIFICATE_UNKNOWN: Peer certificate is not in config specified SPKI.

  • SSLV3_ALERT_HANDSHAKE_FAILURE: Handshake failed, usually due to upstream requires client certificate but not presented.

  • TLSV1_ALERT_PROTOCOL_VERSION: TLS protocol version mismatch.

  • TLSV1_ALERT_UNKNOWN_CA: Peer certificate CA is not in trusted CA.

More detailed list of error that can be raised by BoringSSL can be found here