GEP-1742: HTTPRoute Timeouts

  • Issue: #1742
  • Status: Experimental

(See status definitions here.)

TLDR

Create some sort of design so that Gateway API objects can be used to configure timeouts for different types of connection.

Goals

  • Create some method to configure some timeouts.
  • Timeout config must be applicable to most if not all Gateway API implementations.

Non-Goals

  • A standard API for every possible timeout that implementations may support.

Introduction

In talking about Gateway API objects, particularly HTTPRoute, we’ve mentioned timeout configuration many times in the past as “too hard” to find the common ground necessary to make more generic configuration. This GEP intends firstly to make this process less difficult, then to find common timeouts that we can build into Gateway API.

For this initial round, we’ll focus on Layer 7 HTTP traffic, while acknowledging that Layer 4 connections have their own interesting timeouts as well.

The following sections will review all the implementations, then document what timeouts are available for the various data planes.

Background on implementations

Most implementations that handle HTTPRoute objects use a proxy as the data plane implementation, that actually forwards flows as directed by Gateway API configuration.

The following table is a review of all the listed implementations of Gateway API at the time of writing, with the data plane they use for Layer 7, based on what information could be found online. If there are errors here, or if the implementation doesn’t support layer 7, please feel free to correct them.

ImplementationData Plane
Acnodal EPICEnvoy
Apache APISIXNginx
BIG-IP Kubernetes GatewayF5 BIG-IP
CiliumEnvoy
ContourEnvoy
Emissary IngressEnvoy
Envoy GatewayEnvoy
Flomesh Service MeshPipy
Gloo EdgeEnvoy
Google Kubernetes Engine (GKE)Similar to Envoy Timeouts
HAProxy IngressHAProxy
Hashicorp ConsulEnvoy
IstioEnvoy
KongNginx
KumaEnvoy
LitespeedLitespeed WebADC
NGINX Gateway FabricNginx
TraefikTraefik

Flow diagrams with available timeouts

The following flow diagrams are based off the basic diagram below, with all the timeouts I could find included.

In general, timeouts are recorded with the setting name or similar that the data plane uses for them, and are correct as far as I’ve parsed the documentation correctly.

Idle timeouts are marked as such.

  1. sequenceDiagram
  2.     participant C as Client
  3.     participant P as Proxy
  4.     participant U as Upstream
  5.     C->>P: Connection Started
  6.     C->>P: Starts sending Request
  7.     C->>P: Finishes Headers
  8.     C->>P: Finishes request
  9.     P->>U: Connection Started
  10.     P->>U: Starts sending Request
  11.     P->>U: Finishes request
  12.     P->>U: Finishes Headers
  13.     U->>P: Starts Response
  14.     U->>P: Finishes Headers
  15.     U->>P: Finishes Response
  16.     P->>C: Starts Response
  17.     P->>C: Finishes Headers
  18.     P->>C: Finishes Response
  19.     Note right of P: Repeat if connection sharing
  20.     U->>C: Connection ended

Envoy Timeouts

For Envoy, some timeouts are configurable at either the HTTP Connection Manager (very, very roughly equivalent to a Listener), the Route (equivalent to a HTTPRoute) level, or the Cluster (usually close to the Service) or some combination. These are noted in the below diagram with a CM, R, or Cluster prefix respectively.

  1. sequenceDiagram
  2.     participant C as Client
  3.     participant P as Envoy
  4.     participant U as Upstream
  5.     C->>P: Connection Started
  6.     activate P
  7.     Note left of P: transport_socket_connect_timeout for TLS
  8.     deactivate P
  9.     C->>P: Starts sending Request
  10.     activate C
  11.     activate P
  12.     activate P
  13.     C->>P: Finishes Headers
  14.     note left of P: CM request_headers_timeout
  15.         C->>P: Finishes request
  16.     deactivate P
  17.     activate U
  18.     note left of U: Cluster connect_timeout
  19.     deactivate U
  20.     P->>U: Connection Started
  21.         activate U
  22.     note right of U: CM idle_timeout<br />CM max_connection_duration
  23.     P->>U: Starts sending Request
  24.     P->>U: Finishes Headers
  25.     note left of P: CM request_timeout
  26.         P->>U: Finishes request
  27.     deactivate P
  28.     activate U
  29.     U->>P: Starts Response
  30.     U->>P: Finishes Headers
  31.         note right of U: R timeout<br/>R per_try_timeout<br/>R per_try_idle_timeout
  32.     U->>P: Finishes Response
  33.     deactivate U
  34.     P->>C: Starts Response
  35.     P->>C: Finishes Headers
  36.     P->>C: Finishes Response
  37.     Note left of C: CM stream_idle_timeout<br />R idle_timeout<br />CM,R max_stream_duration<br/>TCP proxy idle_timeout<br />TCP protocol idle_timeout
  38.     deactivate C
  39.     Note right of P: Repeat if connection sharing
  40.     U->>C: Connection ended
  41.     deactivate U

Nginx timeouts

Nginx allows setting of GRPC and general HTTP timeouts separately, although the purposes seem to be roughly equivalent.

  1. sequenceDiagram
  2.     participant C as Client
  3.     participant P as Nginx
  4.     participant U as Upstream
  5.     C->>P: Connection Started
  6.     activate P
  7.     C->>P: Starts sending Request
  8.     C->>P: Finishes Headers
  9.     Note right of P: client_headers_timeout
  10.     deactivate P
  11.     activate P
  12.     C->>P: Finishes request
  13.     deactivate P
  14.     Note right of P: client_body_timeout
  15.     activate U
  16.     note left of U: proxy_connect_timeout<br/>grpc_connect_timeout
  17.     deactivate U
  18.     P->>U: Connection Started
  19.     Activate U
  20.       Activate U
  21.     P->>U: Starts sending Request
  22.     P->>U: Finishes Headers
  23.         P->>U: Finishes request
  24.     Note right of U: (between write operations)<br/>proxy_send_timeout<br/>grpc_send_timeout
  25.     deactivate U
  26.         activate U
  27.     U->>P: Starts Response
  28.     U->>P: Finishes Headers
  29.         Note right of U: (between read operations)<br/>proxy_read_timeout<br/>grpc_read_timeout
  30.     U->>P: Finishes Response
  31.     deactivate U
  32.     activate P
  33.     P->>C: Starts Response
  34.     P->>C: Finishes Headers
  35.     P->>C: Finishes Response
  36.     deactivate P
  37.     Note left of P: send_timeout (only between two successive write operations)
  38.     Note left of C: Repeat if connection is shared until server's keepalive_timeout is hit
  39.     Note Right of U: upstream's keepalive_timeout (if keepalive enabled)
  40.     U->>C: Connection ended
  41.         deactivate U

HAProxy timeouts

  1. sequenceDiagram
  2.     participant C as Client
  3.     participant P as Proxy
  4.     participant U as Upstream
  6.     C->>P: Connection Started
  7.     activate U
  8.     activate C
  9.     activate P
  10.     note left of P: timeout client (idle)
  11.     C->>P: Starts sending Request
  12.     C->>P: Finishes Headers
  13.     C->>P: Finishes request
  14.     note left of C: timeout http-request
  15.     deactivate C
  16.             activate C
  17.     note left of C: timeout client-fin
  18.     deactivate C
  19.         deactivate P
  20.     activate U
  21.     note left of U: timeout queue<br/>(wait for available server)
  22.     deactivate U
  24.     P->>U: Connection Started
  25.     activate U
  26.     P->>U: Starts sending Request
  27.     activate U
  28.     P->>U: Finishes Headers
  29.     P->>U: Finishes request
  31.     note right of U: timeout connect
  32.     deactivate U
  33.     note left of U: timeout server<br/>(idle timeout)
  34.     deactivate U
  35.     activate U
  36.     note left of U: timeout server-fin
  37.     deactivate U
  38.     U->>P: Starts Response
  39.     U->>P: Finishes Headers
  40.     U->>P: Finishes Response
  41.     P->>C: Starts Response
  42.     P->>C: Finishes Headers
  43.     P->>C: Finishes Response
  44.     activate C
  45.     note left of C: timeout http-keep-alive
  46.     deactivate C
  47.     Note right of P: Repeat if connection sharing
  48.     Note right of U: timeout tunnel<br/>(for upgraded connections)
  49.     deactivate U
  50.     U->>C: Connection ended

Traefik timeouts

  1. sequenceDiagram
  2.     participant C as Client
  3.     participant P as Proxy
  4.     participant U as Upstream
  5.     C->>P: Connection Started
  6.     activate U
  7.     C->>P: Starts sending Request
  8.     activate P
  9.     C->>P: Finishes Headers
  10.     Note right of P: respondingTimeouts<br/>readTimeout
  11.     C->>P: Finishes request
  12.     deactivate P
  13.     P->>U: Connection Started
  14.     activate U
  15.     Note right of U: forwardingTimeouts<br/>dialTimeout
  16.     deactivate U
  17.     P->>U: Starts sending Request
  18.     P->>U: Finishes request
  19.     P->>U: Finishes Headers
  20.     U->>P: Starts Response
  21.     activate U
  22.     note right of U: forwardingTimeouts<br/>responseHeaderTimeout
  23.     U->>P: Finishes Headers
  24.     deactivate U
  25.     U->>P: Finishes Response
  26.     P->>C: Starts Response
  27.     activate P
  28.     P->>C: Finishes Headers
  29.     Note right of P: respondingTimeouts<br/>writeTimeout
  30.     P->>C: Finishes Response
  31.     deactivate P
  32.     Note right of P: Repeat if connection sharing
  33.     Note right of U: respondingTimeouts<br/>idleTimeout<br/>Keepalive connections only
  34.     deactivate U
  35.     U->>C: Connection ended

F5 BIG-IP Timeouts

Could not find any HTTP specific timeouts. PRs welcomed. 😊

Pipy Timeouts

Could not find any HTTP specific timeouts. PRs welcomed. 😊

Litespeed WebADC Timeouts

Could not find any HTTP specific timeouts. PRs welcomed. 😊

API

The above diagrams show that there are many different kinds of configurable timeouts supported by Gateway implementations: connect, idle, request, upstream, downstream. Although there may be opportunity for the specification of a common API for more of them in the future, this GEP will focus on the L7 timeouts in HTTPRoutes that are most valuable to clients.

From the above analysis, it appears that most implementations are capable of supporting the configuration of simple client downstream request timeouts on HTTPRoute rules. This is a relatively small addition that would benefit many users.

Some implementations support configuring a timeout for individual backend requests, separate from the overall client request timeout. This is particularly useful if a client HTTP request to a gateway can result in more than one call from the gateway to the destination backend service, for example, if automatic retries are supported. Adding support for this would also benefit many users.

Timeout values

There are 2 kinds of timeouts that can be configured in an HTTPRouteRule:

  1. timeouts.request is the timeout for the Gateway API implementation to send a response to a client HTTP request. Whether the gateway starts the timeout before or after the entire client request stream has been received, is implementation dependent. This field is optional Extended support.

  2. timeouts.backendRequest is a timeout for a single request from the gateway to a backend. This field is optional Extended support. Typically used in conjunction with retry configuration, if supported by an implementation. Note that retry configuration will be the subject of a separate GEP (GEP-1731).

  1. sequenceDiagram
  2.     participant C as Client
  3.     participant P as Proxy
  4.     participant U as Upstream
  5.     C->>P: Connection Started
  6.     note left of P: timeouts.request start time (min)
  7.     C->>P: Starts sending Request
  8.     C->>P: Finishes Headers
  9.     C->>P: Finishes request
  10.     note left of P: timeouts.request start time (max)
  11.     P->>U: Connection Started
  12.     note right of P: timeouts.backendRequest start time
  13.     P->>U: Starts sending Request
  14.     P->>U: Finishes request
  15.     P->>U: Finishes Headers
  16.     U->>P: Starts Response
  17.     U->>P: Finishes Headers
  18.     note right of P: timeouts.backendRequest end time
  19.     note left of P: timeouts.request end time
  20.     U->>P: Finishes Response
  21.     note right of P: Repeat if retry
  22.     P->>C: Starts Response
  23.     P->>C: Finishes Headers
  24.     P->>C: Finishes Response
  25.     Note right of P: Repeat if connection sharing
  26.     U->>C: Connection ended

Both timeout fields are GEP-2257 Duration values. A zero-valued timeout (“0s”) MUST be interpreted as disabling the timeout; a non-zero-valued timeout MUST be >= 1ms.

GO

  1. type HTTPRouteRule struct {
  2.     // Timeouts defines the timeouts that can be configured for an HTTP request.
  3.     //
  4.     // Support: Extended
  5.     //
  6.     // +optional
  7.     // <gateway:experimental>
  8.     Timeouts *HTTPRouteTimeouts `json:"timeouts,omitempty"`
  10.     // ...
  11. }
  13. // HTTPRouteTimeouts defines timeouts that can be configured for an HTTPRoute.
  14. // Timeout values are represented with Gateway API Duration formatting.
  15. // Specifying a zero value such as "0s" is interpreted as no timeout.
  16. //
  17. // +kubebuilder:validation:XValidation:message="backendRequest timeout cannot be longer than request timeout",rule="!(has(self.request) && has(self.backendRequest) && duration(self.request) != duration('0s') && duration(self.backendRequest) > duration(self.request))"
  18. type HTTPRouteTimeouts struct {
  19.     // Request specifies the maximum duration for a gateway to respond to an HTTP request.
  20.     // If the gateway has not been able to respond before this deadline is met, the gateway
  21.     // MUST return a timeout error.
  22.     //
  23.     // For example, setting the `rules.timeouts.request` field to the value `10s` in an
  24.     // `HTTPRoute` will cause a timeout if a client request is taking longer than 10 seconds
  25.     // to complete.
  26.     //
  27.     // This timeout is intended to cover as close to the whole request-response transaction
  28.     // as possible although an implementation MAY choose to start the timeout after the entire
  29.     // request stream has been received instead of immediately after the transaction is
  30.     // initiated by the client.
  31.     //
  32.     // When this field is unspecified, request timeout behavior is implementation-specific.
  33.     //
  34.     // Support: Extended
  35.     //
  36.     // +optional
  37.     Request *Duration `json:"request,omitempty"`
  39.     // BackendRequest specifies a timeout for an individual request from the gateway
  40.     // to a backend. This covers the time from when the request first starts being
  41.     // sent from the gateway to when the full response has been received from the backend.
  42.     //
  43.     // An entire client HTTP transaction with a gateway, covered by the Request timeout,
  44.     // may result in more than one call from the gateway to the destination backend,
  45.     // for example, if automatic retries are supported.
  46.     //
  47.     // Because the Request timeout encompasses the BackendRequest timeout, the value of
  48.     // BackendRequest must be <= the value of Request timeout.
  49.     //
  50.     // Support: Extended
  51.     //
  52.     // +optional
  53.     BackendRequest *Duration `json:"backendRequest,omitempty"`
  54. }
  56. // Duration is a string value representing a duration in time. The foramat is as specified
  57. // in GEP-2257, a strict subset of the syntax parsed by Golang time.ParseDuration.
  58. //
  59. // +kubebuilder:validation:Pattern=`^([0-9]{1,5}(h|m|s|ms)){1,4}$`
  60. type Duration string

YAML

  1. apiVersion: gateway.networking.k8s.io/v1beta1
  2. kind: HTTPRoute
  3. metadata:
  4.   name: timeout-example
  5. spec:
  6.   ...
  7.   rules:
  8.   - backendRefs:
  9.     - name: some-service
  10.       port: 8080
  11.     timeouts:
  12.       request: 10s
  13.       backendRequest: 2s

Conformance Details

Gateway implementations can indicate support for the optional behavior in this GEP using the following feature names:

  • HTTPRouteRequestTimeout: supports rules.timeouts.request in an HTTPRoute.
  • HTTPRouteBackendTimeout: supports rules.timeouts.backendRequest in an HTTPRoute.

Alternatives

Timeouts could be configured using policy attachments or in objects other than HTTPRouteRule.

Policy Attachment

Instead of configuring timeouts directly on an API object, they could be configured using policy attachments. The advantage to this approach would be that timeout policies can be not only configured for an HTTPRouteRule, but can also be added/overriden at a more fine (e.g., HTTPBackendRef) or course (e.g. HTTPRoute) level of granularity.

The downside, however, is complexity introduced for the most common use case, adding a simple timeout for an HTTP request. Setting a single field in the route rule, instead of needing to create a policy resource, for this simple case seems much better.

In the future, we could consider using policy attachments to configure less common kinds of timeouts that may be needed, but it would probably be better to instead extend the proposed API to support those timeouts as well.

The default values of the proposed timeout fields could also be overridden using policy attachments in the future. For example, a policy attachment could be used to set the default value of rules.timeouts.request for all routes under an HTTPRoute or Gateway.

Other API Objects

The new timeouts field could be added to a different API struct, instead of HTTPRouteRule.

Putting it on an HTTPBackendRef, for example, would allow users to set different timeouts for different backends. This is a feature that we believe has not been requested by existing proxy or service mesh clients and is also not implementable using available timeouts of most proxies.

Another alternative is to move the timeouts configuration up a level in the API to HTTPRoute. This would be convenient when a user wants the same timeout on all rules, but would be overly restrictive. Using policy attachments to override the default timeout value for all rules, as described in the previous section, is likely a better way to handle timeout configuration above the route rule level.

References

(Add any additional document links. Again, we should try to avoid too much content not in version control to avoid broken links)