Kubernetes Gateway API

In addition to its own traffic management API, Istio includes beta support for the Kubernetes Gateway API and intends to make it the default API for traffic management in the future. This document describes the differences between the Istio and Kubernetes APIs and provides a simple example that shows you how to configure Istio to expose a service outside the service mesh cluster using the Gateway API. Note that these APIs are an actively developed evolution of the Kubernetes Service and Ingress APIs.

Many of the Istio traffic management documents include instructions for using either the Istio or Kubernetes API (see the control ingress traffic task, for example). You can even use the Gateway API, right from the start, by following the future getting started instructions.

Setup

  1. The Gateway APIs do not come installed by default on most Kubernetes clusters. Install the Gateway API CRDs if they are not present:

    1. $ kubectl get crd gateways.gateway.networking.k8s.io &> /dev/null || \
    2.   { kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=444631bfe06f3bcca5d0eadf1857eac1d369421d" | kubectl apply -f -; }

  2. Install Istio using the minimal profile:

    1. $ istioctl install --set profile=minimal -y

Differences from Istio APIs

The Gateway APIs share a lot of similarities to the Istio APIs such as Gateway and VirtualService. The main resource shares the same name, Gateway, and the resources serve similar goals.

The new Gateway APIs aim to take the learnings from various Kubernetes ingress implementations, including Istio, to build a standardized vendor neutral API. These APIs generally serve the same purposes as Istio Gateway and VirtualService, with a few key differences:

  • In Istio APIs, a Gateway configures an existing gateway Deployment/Service that has been deployed. In the Gateway APIs, the Gateway resource both configures and deploys a gateway. See Deployment Methods for more information.
  • In the Istio VirtualService, all protocols are configured within a single resource. In the Gateway APIs, each protocol type has its own resource, such as HTTPRoute and TCPRoute.
  • While the Gateway APIs offer a lot of rich routing functionality, it does not yet cover 100% of Istio’s feature set. Work is ongoing to extend the API to cover these use cases, as well as utilizing the APIs extensibility to better expose Istio functionality.

Configuring a Gateway

See the Gateway API documentation for information about the APIs.

In this example, we will deploy a simple application and expose it externally using a Gateway.

  1. First, deploy the httpbin test application:

    Zip

    1. $ kubectl apply -f @samples/httpbin/httpbin.yaml@

  2. Deploy the Gateway API configuration including a single exposed route (i.e., /get):

    1. $ kubectl create namespace istio-ingress
    2. $ kubectl apply -f - <<EOF
    3. apiVersion: gateway.networking.k8s.io/v1beta1
    4. kind: Gateway
    5. metadata:
    6.   name: gateway
    7.   namespace: istio-ingress
    8. spec:
    9.   gatewayClassName: istio
    10.   listeners:
    11.   - name: default
    12.     hostname: "*.example.com"
    13.     port: 80
    14.     protocol: HTTP
    15.     allowedRoutes:
    16.       namespaces:
    17.         from: All
    18. ---
    19. apiVersion: gateway.networking.k8s.io/v1beta1
    20. kind: HTTPRoute
    21. metadata:
    22.   name: http
    23.   namespace: default
    24. spec:
    25.   parentRefs:
    26.   - name: gateway
    27.     namespace: istio-ingress
    28.   hostnames: ["httpbin.example.com"]
    29.   rules:
    30.   - matches:
    31.     - path:
    32.         type: PathPrefix
    33.         value: /get
    34.     backendRefs:
    35.     - name: httpbin
    36.       port: 8000
    37. EOF

  3. Set the Ingress Host environment variable:

    1. $ kubectl wait -n istio-ingress --for=condition=programmed gateways.gateway.networking.k8s.io gateway
    2. $ export INGRESS_HOST=$(kubectl get gateways.gateway.networking.k8s.io gateway -n istio-ingress -ojsonpath='{.status.addresses[0].value}')

  4. Access the httpbin service using curl:

    1. $ curl -s -I -HHost:httpbin.example.com "http://$INGRESS_HOST/get"
    2. HTTP/1.1 200 OK
    3. server: istio-envoy
    4. ...

    Note the use of the -H flag to set the Host HTTP header to “httpbin.example.com”. This is needed because the HTTPRoute is configured to handle “httpbin.example.com”, but in your test environment you have no DNS binding for that host and are simply sending your request to the ingress IP.

  5. Access any other URL that has not been explicitly exposed. You should see an HTTP 404 error:

    1. $ curl -s -I -HHost:httpbin.example.com "http://$INGRESS_HOST/headers"
    2. HTTP/1.1 404 Not Found
    3. ...

  6. Update the route rule to also expose /headers and to add a header to the request:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: gateway.networking.k8s.io/v1beta1
    3. kind: HTTPRoute
    4. metadata:
    5.   name: http
    6.   namespace: default
    7. spec:
    8.   parentRefs:
    9.   - name: gateway
    10.     namespace: istio-ingress
    11.   hostnames: ["httpbin.example.com"]
    12.   rules:
    13.   - matches:
    14.     - path:
    15.         type: PathPrefix
    16.         value: /get
    17.     - path:
    18.         type: PathPrefix
    19.         value: /headers
    20.     filters:
    21.     - type: RequestHeaderModifier
    22.       requestHeaderModifier:
    23.         add:
    24.         - name: my-added-header
    25.           value: added-value
    26.     backendRefs:
    27.     - name: httpbin
    28.       port: 8000
    29. EOF

  7. Access /headers again and notice header My-Added-Header has been added to the request:

    1. $ curl -s -HHost:httpbin.example.com "http://$INGRESS_HOST/headers"
    2. {
    3.   "headers": {
    4.     "Accept": "*/*",
    5.     "Host": "httpbin.example.com",
    6.     "My-Added-Header": "added-value",
    7. ...

Deployment methods

In the example above, you did not need to install an ingress gateway Deployment prior to configuring a Gateway. In the default configuration, a gateway Deployment and Service is automatically provisioned based on the Gateway configuration. For advanced use cases, manual deployment is still allowed.

Automated Deployment

By default, each Gateway will automatically provision a Service and Deployment of the same name. These configurations will be updated automatically if the Gateway changes (for example, if a new port is added).

These resources can be customized in a few ways:

  • Annotations and labels on the Gateway will be copied to the Service and Deployment. This allows configuring things such as Internal load balancers that read from these fields.

  • Istio offers an additional annotation to configure the generated resources:

    AnnotationPurpose
    networking.istio.io/service-typeControls the Service.spec.type field. For example, set to ClusterIP to not expose the service externally. The default is LoadBalancer.

  • The Service.spec.loadBalancerIP field can be explicit set by configuring the addresses field:

    1. apiVersion: gateway.networking.k8s.io/v1beta1
    2. kind: Gateway
    3. metadata:
    4.   name: gateway
    5. spec:
    6.   addresses:
    7.   - value: 192.0.2.0
    8.     type: IPAddress
    9. ...

Note: only one address may be specified.

Resource Attachment and Scaling

Resource attachment is currently experimental.

Resources can be attached to a Gateway to customize it. However, most Kubernetes resources do not currently support attaching directly to a Gateway, but they can be attached to the corresponding generated Deployment and Service instead. This is easily done because both of these resources are generated with name <gateway name>-<gateway class name> and with a label gateway.networking.k8s.io/gateway-name: <gateway name>.

For example, to deploy a Gateway with a HorizontalPodAutoscaler and PodDisruptionBudget:

  1. apiVersion: gateway.networking.k8s.io/v1beta1
  2. kind: Gateway
  3. metadata:
  4.   name: gateway
  5. spec:
  6.   gatewayClassName: istio
  7.   listeners:
  8.   - name: default
  9.     hostname: "*.example.com"
  10.     port: 80
  11.     protocol: HTTP
  12.     allowedRoutes:
  13.       namespaces:
  14.         from: All
  15. ---
  16. apiVersion: autoscaling/v2
  17. kind: HorizontalPodAutoscaler
  18. metadata:
  19.   name: gateway
  20. spec:
  21.   # Match the generated Deployment by reference
  22.   # Note: Do not use `kind: Gateway`.
  23.   scaleTargetRef:
  24.     apiVersion: apps/v1
  25.     kind: Deployment
  26.     name: gateway-istio
  27.   minReplicas: 2
  28.   maxReplicas: 5
  29.   metrics:
  30.   - type: Resource
  31.     resource:
  32.       name: cpu
  33.       target:
  34.         type: Utilization
  35.         averageUtilization: 50
  36. ---
  37. apiVersion: policy/v1
  38. kind: PodDisruptionBudget
  39. metadata:
  40.   name: gateway
  41. spec:
  42.   minAvailable: 1
  43.   selector:
  44.     # Match the generated Deployment by label
  45.     matchLabels:
  46.       gateway.networking.k8s.io/gateway-name: gateway

Manual Deployment

If you do not want to have an automated deployment, a Deployment and Service can be configured manually.

When this option is done, you will need to manually link the Gateway to the Service, as well as keep their port configuration in sync.

To link a Gateway to a Service, configure the addresses field to point to a single Hostname.

  1. apiVersion: gateway.networking.k8s.io/v1beta1
  2. kind: Gateway
  3. metadata:
  4.   name: gateway
  5. spec:
  6.   addresses:
  7.   - value: ingress.istio-gateways.svc.cluster.local
  8.     type: Hostname
  9. ...

Mesh Traffic

Configuring internal mesh traffic using the Gateway API is an experimental feature currently under development.

The Gateway API can also be used to configure mesh traffic. This is done by configuring the parentRef to point to a service, instead of a gateway.

For example, to add a header on all calls to an in-cluster Service named example:

  1. apiVersion: gateway.networking.k8s.io/v1beta1
  2. kind: HTTPRoute
  3. metadata:
  4.   name: mesh
  5. spec:
  6.   parentRefs:
  7.   - group: ""
  8.     kind: Service
  9.     name: example
  10.   rules:
  11.   - filters:
  12.     - type: RequestHeaderModifier
  13.       requestHeaderModifier:
  14.         add:
  15.         - name: my-added-header
  16.           value: added-value
  17.     backendRefs:
  18.     - name: example
  19.       port: 80

More details and examples can be found in other traffic management tasks.

Cleanup

  1. Remove the httpbin sample and gateway:

    Zip

    1. $ kubectl delete -f @samples/httpbin/httpbin.yaml@
    2. $ kubectl delete httproute http
    3. $ kubectl delete gateways.gateway.networking.k8s.io gateway -n istio-ingress
    4. $ kubectl delete ns istio-ingress

  2. Uninstall Istio:

    1. $ istioctl uninstall -y --purge
    2. $ kubectl delete ns istio-system

  3. Remove the Gateway API CRDs if they are no longer needed:

    1. $ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=444631bfe06f3bcca5d0eadf1857eac1d369421d" | kubectl delete -f -