Egress using Wildcard Hosts

The Accessing External Services task and the Configure an Egress Gateway example describe how to configure egress traffic for specific hostnames, like edition.cnn.com. This example shows how to enable egress traffic for a set of hosts in a common domain, for example *.wikipedia.org, instead of configuring each and every host separately.

Background

Suppose you want to enable egress traffic in Istio for the wikipedia.org sites in all languages. Each version of wikipedia.org in a particular language has its own hostname, e.g., en.wikipedia.org and de.wikipedia.org in the English and the German languages, respectively. You want to enable egress traffic by common configuration items for all the Wikipedia sites, without the need to specify every language’s site separately.

Istio includes beta support for the Kubernetes Gateway API and intends to make it the default API for traffic management in the future. The following instructions allow you to choose to use either the Gateway API or the Istio configuration API when configuring traffic management in the mesh. Follow instructions under either the Gateway API or Istio APIs tab, according to your preference.

This document configures internal mesh (east-west) traffic that requires Gateway API features that are either experimental or Istio specific. Before using the Gateway API instructions, make sure to:

  1. Install the experimental version of the Gateway API CRDs:

    1. $ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=v1.1.0" | kubectl apply -f -

  2. Configure Istio to read the alpha Gateway API resources by setting the PILOT_ENABLE_ALPHA_GATEWAY_API environment variable to true when installing Istio:

    1. $ istioctl install --set values.pilot.env.PILOT_ENABLE_ALPHA_GATEWAY_API=true --set profile=minimal -y

Before you begin

  • Install Istio with access logging enabled and with the blocking-by-default outbound traffic policy:
  1. $ istioctl install --set profile=demo --set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY

You can run this task on an Istio configuration other than the demo profile as long as you make sure to deploy the Istio egress gateway, enable Envoy’s access logging, and apply the blocking-by-default outbound traffic policy in your installation.

  1. $ istioctl install --set profile=minimal -y \
  2.     --set values.pilot.env.PILOT_ENABLE_ALPHA_GATEWAY_API=true \
  3.     --set meshConfig.accessLogFile=/dev/stdout \
  4.     --set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY

  • Deploy the sleep sample app to use as a test source for sending requests. If you have automatic sidecar injection enabled, run the following command to deploy the sample app:

    Zip

    1. $ kubectl apply -f @samples/sleep/sleep.yaml@

    Otherwise, manually inject the sidecar before deploying the sleep application with the following command:

    Zip

    1. $ kubectl apply -f <(istioctl kube-inject -f @samples/sleep/sleep.yaml@)

    You can use any pod with curl installed as a test source.

  • Set the SOURCE_POD environment variable to the name of your source pod:

    1. $ export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})

Configure direct traffic to a wildcard host

The first, and simplest, way to access a set of hosts within a common domain is by configuring a simple ServiceEntry with a wildcard host and calling the services directly from the sidecar. When calling services directly (i.e., not via an egress gateway), the configuration for a wildcard host is no different than that of any other (e.g., fully qualified) host, only much more convenient when there are many hosts within the common domain.

Note that the configuration below can be easily bypassed by a malicious application. For a secure egress traffic control, direct the traffic through an egress gateway.

Note that the DNS resolution cannot be used for wildcard hosts. This is why the NONE resolution (omitted since it is the default) is used in the service entry below.

  1. Define a ServiceEntry for *.wikipedia.org:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: ServiceEntry
    4. metadata:
    5.   name: wikipedia
    6. spec:
    7.   hosts:
    8.   - "*.wikipedia.org"
    9.   ports:
    10.   - number: 443
    11.     name: https
    12.     protocol: HTTPS
    13. EOF

  2. Send HTTPS requests to https://en.wikipedia.org and https://de.wikipedia.org:

    1. $ kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
    2. <title>Wikipedia, the free encyclopedia</title>
    3. <title>Wikipedia – Die freie Enzyklopädie</title>

Cleanup direct traffic to a wildcard host

  1. $ kubectl delete serviceentry wikipedia

Configure egress gateway traffic to a wildcard host

When all wildcard hosts are served by a single server, the configuration for egress gateway-based access to a wildcard host is very similar to that of any host, with one exception: the configured route destination will not be the same as the configured host, i.e., the wildcard. It will instead be configured with the host of the single server for the set of domains.

  1. Create an egress Gateway for *.wikipedia.org and route rules to direct the traffic through the egress gateway and from the egress gateway to the external service:
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: Gateway
  4. metadata:
  5.   name: istio-egressgateway
  6. spec:
  7.   selector:
  8.     istio: egressgateway
  9.   servers:
  10.   - port:
  11.       number: 443
  12.       name: https
  13.       protocol: HTTPS
  14.     hosts:
  15.     - "*.wikipedia.org"
  16.     tls:
  17.       mode: PASSTHROUGH
  18. ---
  19. apiVersion: networking.istio.io/v1alpha3
  20. kind: DestinationRule
  21. metadata:
  22.   name: egressgateway-for-wikipedia
  23. spec:
  24.   host: istio-egressgateway.istio-system.svc.cluster.local
  25.   subsets:
  26.     - name: wikipedia
  27. ---
  28. apiVersion: networking.istio.io/v1alpha3
  29. kind: VirtualService
  30. metadata:
  31.   name: direct-wikipedia-through-egress-gateway
  32. spec:
  33.   hosts:
  34.   - "*.wikipedia.org"
  35.   gateways:
  36.   - mesh
  37.   - istio-egressgateway
  38.   tls:
  39.   - match:
  40.     - gateways:
  41.       - mesh
  42.       port: 443
  43.       sniHosts:
  44.       - "*.wikipedia.org"
  45.     route:
  46.     - destination:
  47.         host: istio-egressgateway.istio-system.svc.cluster.local
  48.         subset: wikipedia
  49.         port:
  50.           number: 443
  51.       weight: 100
  52.   - match:
  53.     - gateways:
  54.       - istio-egressgateway
  55.       port: 443
  56.       sniHosts:
  57.       - "*.wikipedia.org"
  58.     route:
  59.     - destination:
  60.         host: www.wikipedia.org
  61.         port:
  62.           number: 443
  63.       weight: 100
  64. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: gateway.networking.k8s.io/v1
  3. kind: Gateway
  4. metadata:
  5.   name: wikipedia-egress-gateway
  6.   annotations:
  7.     networking.istio.io/service-type: ClusterIP
  8. spec:
  9.   gatewayClassName: istio
  10.   listeners:
  11.   - name: tls
  12.     hostname: "*.wikipedia.org"
  13.     port: 443
  14.     protocol: TLS
  15.     tls:
  16.       mode: Passthrough
  17.     allowedRoutes:
  18.       namespaces:
  19.         from: Same
  20. ---
  21. apiVersion: gateway.networking.k8s.io/v1alpha2
  22. kind: TLSRoute
  23. metadata:
  24.   name: direct-wikipedia-to-egress-gateway
  25. spec:
  26.   parentRefs:
  27.   - kind: ServiceEntry
  28.     group: networking.istio.io
  29.     name: wikipedia
  30.   rules:
  31.   - backendRefs:
  32.     - name: wikipedia-egress-gateway-istio
  33.       port: 443
  34. ---
  35. apiVersion: gateway.networking.k8s.io/v1alpha2
  36. kind: TLSRoute
  37. metadata:
  38.   name: forward-wikipedia-from-egress-gateway
  39. spec:
  40.   parentRefs:
  41.   - name: wikipedia-egress-gateway
  42.   hostnames:
  43.   - "*.wikipedia.org"
  44.   rules:
  45.   - backendRefs:
  46.     - kind: Hostname
  47.       group: networking.istio.io
  48.       name: www.wikipedia.org
  49.       port: 443
  50. ---
  51. apiVersion: networking.istio.io/v1alpha3
  52. kind: ServiceEntry
  53. metadata:
  54.   name: wikipedia
  55. spec:
  56.   hosts:
  57.   - "*.wikipedia.org"
  58.   ports:
  59.   - number: 443
  60.     name: https
  61.     protocol: HTTPS
  62. EOF

  1. Create a ServiceEntry for the destination server, www.wikipedia.org:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: ServiceEntry
    4. metadata:
    5.   name: www-wikipedia
    6. spec:
    7.   hosts:
    8.   - www.wikipedia.org
    9.   ports:
    10.   - number: 443
    11.     name: https
    12.     protocol: HTTPS
    13.   resolution: DNS
    14. EOF

  2. Send HTTPS requests to https://en.wikipedia.org and https://de.wikipedia.org:

    1. $ kubectl exec "$SOURCE_POD" -c sleep -- sh -c 'curl -s https://en.wikipedia.org/wiki/Main_Page | grep -o "<title>.*</title>"; curl -s https://de.wikipedia.org/wiki/Wikipedia:Hauptseite | grep -o "<title>.*</title>"'
    2. <title>Wikipedia, the free encyclopedia</title>
    3. <title>Wikipedia – Die freie Enzyklopädie</title>

  3. Check the statistics of the egress gateway’s proxy for the counter that corresponds to your requests to *.wikipedia.org:

  1. $ kubectl exec "$(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}')" -c istio-proxy -n istio-system -- pilot-agent request GET clusters | grep '^outbound|443||www.wikipedia.org.*cx_total:'
  2. outbound|443||www.wikipedia.org::208.80.154.224:443::cx_total::2
  1. $ kubectl exec "$(kubectl get pod -l gateway.networking.k8s.io/gateway-name=wikipedia-egress-gateway -o jsonpath='{.items[0].metadata.name}')" -c istio-proxy -- pilot-agent request GET clusters | grep '^outbound|443||www.wikipedia.org.*cx_total:'
  2. outbound|443||www.wikipedia.org::208.80.154.224:443::cx_total::2

Cleanup egress gateway traffic to a wildcard host

  1. $ kubectl delete serviceentry www-wikipedia
  2. $ kubectl delete gateway istio-egressgateway
  3. $ kubectl delete virtualservice direct-wikipedia-through-egress-gateway
  4. $ kubectl delete destinationrule egressgateway-for-wikipedia
  1. $ kubectl delete se wikipedia
  2. $ kubectl delete se www-wikipedia
  3. $ kubectl delete gtw wikipedia-egress-gateway
  4. $ kubectl delete tlsroute direct-wikipedia-to-egress-gateway
  5. $ kubectl delete tlsroute forward-wikipedia-from-egress-gateway

Wildcard configuration for arbitrary domains

The configuration in the previous section worked because all the *.wikipedia.org sites can be served by any one of the wikipedia.org servers. However, this is not always the case. For example, you may want to configure egress control for access to more general wildcard domains like *.com or *.org. Configuring traffic to arbitrary wildcard domains introduces a challenge for Istio gateways; an Istio gateway can only be configured to route traffic to predefined hosts, predefined IP addresses, or to the original destination IP address of the request.

In the previous section you configured the virtual service to direct traffic to the predefined host www.wikipedia.org. In the general case, however, you don’t know the host or IP address that can serve an arbitrary host received in a request, which leaves the original destination address of the request as the only value with which to route the request. Unfortunately, when using an egress gateway, the original destination address of the request is lost since the original request is redirected to the gateway, causing the destination IP address to become the IP address of the gateway.

Although not as easy and somewhat fragile as it relies on Istio implementation details, you can use Envoy filters to configure a gateway to support arbitrary domains by using the SNI value in an HTTPS, or any TLS, request to identify the original destination to which to route the request. One example of this configuration approach can be found in routing egress traffic to wildcard destinations.

Cleanup

  • Shutdown the sleep service:

    Zip

    1. $ kubectl delete -f @samples/sleep/sleep.yaml@

  • Uninstall Istio from your cluster:

    1. $ istioctl uninstall --purge -y