Egress Gateways with TLS Origination

The TLS Origination for Egress Traffic example shows how to configure Istio to perform TLS origination for traffic to an external service. The Configure an Egress Gateway example shows how to configure Istio to direct egress traffic through a dedicated egress gateway service. This example combines the previous two by describing how to configure an egress gateway to perform TLS origination for traffic to external services.

Istio includes beta support for the Kubernetes Gateway API and intends to make it the default API for traffic management in the future. The following instructions allow you to choose to use either the Gateway API or the Istio configuration API when configuring traffic management in the mesh. Follow instructions under either the Gateway API or Istio APIs tab, according to your preference.

This document configures internal mesh (east-west) traffic that requires Gateway API features that are either experimental or Istio specific. Before using the Gateway API instructions, make sure to:

  1. Install the experimental version of the Gateway API CRDs:

    1. $ kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd/experimental?ref=v1.1.0" | kubectl apply -f -

  2. Configure Istio to read the alpha Gateway API resources by setting the PILOT_ENABLE_ALPHA_GATEWAY_API environment variable to true when installing Istio:

    1. $ istioctl install --set values.pilot.env.PILOT_ENABLE_ALPHA_GATEWAY_API=true --set profile=minimal -y

Before you begin

  • Setup Istio by following the instructions in the Installation guide.

  • Start the sleep sample which will be used as a test source for external calls.

    If you have enabled automatic sidecar injection, do

    Zip

    1. $ kubectl apply -f @samples/sleep/sleep.yaml@

    otherwise, you have to manually inject the sidecar before deploying the sleep application:

    Zip

    1. $ kubectl apply -f <(istioctl kube-inject -f @samples/sleep/sleep.yaml@)

    Note that any pod that you can exec and curl from would do.

  • Create a shell variable to hold the name of the source pod for sending requests to external services. If you used the sleep sample, run:

    1. $ export SOURCE_POD=$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})

  • For macOS users, verify that you are using openssl version 1.1 or later:

    1. $ openssl version -a | grep OpenSSL
    2. OpenSSL 1.1.1g  21 Apr 2020

    If the previous command outputs a version 1.1 or later, as shown, your openssl command should work correctly with the instructions in this task. Otherwise, upgrade your openssl or try a different implementation of openssl, for example on a Linux machine.

  • Enable Envoy’s access logging if not already enabled. For example, using istioctl:

    1. $ istioctl install <flags-you-used-to-install-Istio> --set meshConfig.accessLogFile=/dev/stdout

  • If you are NOT using the Gateway API instructions, make sure to deploy the Istio egress gateway.

Perform TLS origination with an egress gateway

This section describes how to perform the same TLS origination as in the TLS Origination for Egress Traffic example, only this time using an egress gateway. Note that in this case the TLS origination will be done by the egress gateway, as opposed to by the sidecar in the previous example.

  1. Define a ServiceEntry for edition.cnn.com:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: ServiceEntry
    4. metadata:
    5.   name: cnn
    6. spec:
    7.   hosts:
    8.   - edition.cnn.com
    9.   ports:
    10.   - number: 80
    11.     name: http
    12.     protocol: HTTP
    13.   - number: 443
    14.     name: https
    15.     protocol: HTTPS
    16.   resolution: DNS
    17. EOF

  2. Verify that your ServiceEntry was applied correctly by sending a request to http://edition.cnn.com/politics.

    1. $ kubectl exec "${SOURCE_POD}" -c sleep -- curl -sSL -o /dev/null -D - http://edition.cnn.com/politics
    2. HTTP/1.1 301 Moved Permanently
    3. ...
    4. location: https://edition.cnn.com/politics
    5. ...

    Your ServiceEntry was configured correctly if you see 301 Moved Permanently in the output.

  3. Create an egress Gateway for edition.cnn.com, port 80, and a destination rule for sidecar requests that will be directed to the egress gateway.

  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: Gateway
  4. metadata:
  5.   name: istio-egressgateway
  6. spec:
  7.   selector:
  8.     istio: egressgateway
  9.   servers:
  10.   - port:
  11.       number: 80
  12.       name: https-port-for-tls-origination
  13.       protocol: HTTPS
  14.     hosts:
  15.     - edition.cnn.com
  16.     tls:
  17.       mode: ISTIO_MUTUAL
  18. ---
  19. apiVersion: networking.istio.io/v1alpha3
  20. kind: DestinationRule
  21. metadata:
  22.   name: egressgateway-for-cnn
  23. spec:
  24.   host: istio-egressgateway.istio-system.svc.cluster.local
  25.   subsets:
  26.   - name: cnn
  27.     trafficPolicy:
  28.       loadBalancer:
  29.         simple: ROUND_ROBIN
  30.       portLevelSettings:
  31.       - port:
  32.           number: 80
  33.         tls:
  34.           mode: ISTIO_MUTUAL
  35.           sni: edition.cnn.com
  36. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: gateway.networking.k8s.io/v1
  3. kind: Gateway
  4. metadata:
  5.   name: cnn-egress-gateway
  6.   annotations:
  7.     networking.istio.io/service-type: ClusterIP
  8. spec:
  9.   gatewayClassName: istio
  10.   listeners:
  11.   - name: https-listener-for-tls-origination
  12.     hostname: edition.cnn.com
  13.     port: 80
  14.     protocol: HTTPS
  15.     tls:
  16.       mode: Terminate
  17.       options:
  18.         gateway.istio.io/tls-terminate-mode: ISTIO_MUTUAL
  19.     allowedRoutes:
  20.       namespaces:
  21.         from: Same
  22. ---
  23. apiVersion: networking.istio.io/v1alpha3
  24. kind: DestinationRule
  25. metadata:
  26.   name: egressgateway-for-cnn
  27. spec:
  28.   host: cnn-egress-gateway-istio.default.svc.cluster.local
  29.   trafficPolicy:
  30.     loadBalancer:
  31.       simple: ROUND_ROBIN
  32.     portLevelSettings:
  33.     - port:
  34.         number: 80
  35.       tls:
  36.         mode: ISTIO_MUTUAL
  37.         sni: edition.cnn.com
  38. EOF
  1. Configure route rules to direct traffic through the egress gateway:
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: VirtualService
  4. metadata:
  5.   name: direct-cnn-through-egress-gateway
  6. spec:
  7.   hosts:
  8.   - edition.cnn.com
  9.   gateways:
  10.   - istio-egressgateway
  11.   - mesh
  12.   http:
  13.   - match:
  14.     - gateways:
  15.       - mesh
  16.       port: 80
  17.     route:
  18.     - destination:
  19.         host: istio-egressgateway.istio-system.svc.cluster.local
  20.         subset: cnn
  21.         port:
  22.           number: 80
  23.       weight: 100
  24.   - match:
  25.     - gateways:
  26.       - istio-egressgateway
  27.       port: 80
  28.     route:
  29.     - destination:
  30.         host: edition.cnn.com
  31.         port:
  32.           number: 443
  33.       weight: 100
  34. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: gateway.networking.k8s.io/v1
  3. kind: HTTPRoute
  4. metadata:
  5.   name: direct-cnn-to-egress-gateway
  6. spec:
  7.   parentRefs:
  8.   - kind: ServiceEntry
  9.     group: networking.istio.io
  10.     name: cnn
  11.   rules:
  12.   - backendRefs:
  13.     - name: cnn-egress-gateway-istio
  14.       port: 80
  15. ---
  16. apiVersion: gateway.networking.k8s.io/v1
  17. kind: HTTPRoute
  18. metadata:
  19.   name: forward-cnn-from-egress-gateway
  20. spec:
  21.   parentRefs:
  22.   - name: cnn-egress-gateway
  23.   hostnames:
  24.   - edition.cnn.com
  25.   rules:
  26.   - backendRefs:
  27.     - kind: Hostname
  28.       group: networking.istio.io
  29.       name: edition.cnn.com
  30.       port: 443
  31. EOF

  1. Define a DestinationRule to perform TLS origination for requests to edition.cnn.com:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: DestinationRule
    4. metadata:
    5.   name: originate-tls-for-edition-cnn-com
    6. spec:
    7.   host: edition.cnn.com
    8.   trafficPolicy:
    9.     loadBalancer:
    10.       simple: ROUND_ROBIN
    11.     portLevelSettings:
    12.     - port:
    13.         number: 443
    14.       tls:
    15.         mode: SIMPLE # initiates HTTPS for connections to edition.cnn.com
    16. EOF

  2. Send an HTTP request to http://edition.cnn.com/politics.

    1. $ kubectl exec "${SOURCE_POD}" -c sleep -- curl -sSL -o /dev/null -D - http://edition.cnn.com/politics
    2. HTTP/1.1 200 OK
    3. ...

    The output should be the same as in the TLS Origination for Egress Traffic example, with TLS origination: without the 301 Moved Permanently message.

  3. Check the log of the egress gateway’s proxy.

If Istio is deployed in the istio-system namespace, the command to print the log is:

  1. $ kubectl logs -l istio=egressgateway -c istio-proxy -n istio-system | tail

You should see a line similar to the following:

  1. [2020-06-30T16:17:56.763Z] "GET /politics HTTP/2" 200 - "-" "-" 0 1295938 529 89 "10.244.0.171" "curl/7.64.0" "cf76518d-3209-9ab7-a1d0-e6002728ef5b" "edition.cnn.com" "151.101.129.67:443" outbound|443||edition.cnn.com 10.244.0.170:54280 10.244.0.170:8080 10.244.0.171:35628 - -

Access the log corresponding to the egress gateway using the Istio-generated pod label:

  1. $ kubectl logs -l gateway.networking.k8s.io/gateway-name=cnn-egress-gateway -c istio-proxy | tail

You should see a line similar to the following:

  1. [2024-03-14T18:37:01.451Z] "GET /politics HTTP/1.1" 200 - via_upstream - "-" 0 2484998 59 37 "172.30.239.26" "curl/7.87.0-DEV" "b80c8732-8b10-4916-9a73-c3e1c848ed1e" "edition.cnn.com" "151.101.131.5:443" outbound|443||edition.cnn.com 172.30.239.33:51270 172.30.239.33:80 172.30.239.26:35192 edition.cnn.com default.forward-cnn-from-egress-gateway.0

Cleanup the TLS origination example

Remove the Istio configuration items you created:

  1. $ kubectl delete gw istio-egressgateway
  2. $ kubectl delete serviceentry cnn
  3. $ kubectl delete virtualservice direct-cnn-through-egress-gateway
  4. $ kubectl delete destinationrule originate-tls-for-edition-cnn-com
  5. $ kubectl delete destinationrule egressgateway-for-cnn
  1. $ kubectl delete serviceentry cnn
  2. $ kubectl delete gtw cnn-egress-gateway
  3. $ kubectl delete httproute direct-cnn-to-egress-gateway
  4. $ kubectl delete httproute forward-cnn-from-egress-gateway
  5. $ kubectl delete destinationrule egressgateway-for-cnn
  6. $ kubectl delete destinationrule originate-tls-for-edition-cnn-com

Perform mutual TLS origination with an egress gateway

Similar to the previous section, this section describes how to configure an egress gateway to perform TLS origination for an external service, only this time using a service that requires mutual TLS.

This example is considerably more involved because you need to first:

  1. generate client and server certificates
  2. deploy an external service that supports the mutual TLS protocol
  3. redeploy the egress gateway with the needed mutual TLS certs

Only then can you configure the external traffic to go through the egress gateway which will perform TLS origination.

Generate client and server certificates and keys

For this task you can use your favorite tool to generate certificates and keys. The commands below use openssl

  1. Create a root certificate and private key to sign the certificate for your services:

    1. $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt

  2. Create a certificate and a private key for my-nginx.mesh-external.svc.cluster.local:

    1. $ openssl req -out my-nginx.mesh-external.svc.cluster.local.csr -newkey rsa:2048 -nodes -keyout my-nginx.mesh-external.svc.cluster.local.key -subj "/CN=my-nginx.mesh-external.svc.cluster.local/O=some organization"
    2. $ openssl x509 -req -sha256 -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in my-nginx.mesh-external.svc.cluster.local.csr -out my-nginx.mesh-external.svc.cluster.local.crt

    Optionally, you can add SubjectAltNames to the certificate if you want to enable SAN validation for the destination. For example:

    1. $ cat > san.conf <<EOF
    2. [req]
    3. distinguished_name = req_distinguished_name
    4. req_extensions = v3_req
    5. x509_extensions = v3_req
    6. prompt = no
    7. [req_distinguished_name]
    8. countryName = US
    9. [v3_req]
    10. keyUsage = critical, digitalSignature, keyEncipherment
    11. extendedKeyUsage = serverAuth, clientAuth
    12. basicConstraints = critical, CA:FALSE
    13. subjectAltName = critical, @alt_names
    14. [alt_names]
    15. DNS = my-nginx.mesh-external.svc.cluster.local
    16. EOF
    17. $
    18. $ openssl req -out my-nginx.mesh-external.svc.cluster.local.csr -newkey rsa:4096 -nodes -keyout my-nginx.mesh-external.svc.cluster.local.key -subj "/CN=my-nginx.mesh-external.svc.cluster.local/O=some organization" -config san.conf
    19. $ openssl x509 -req -sha256 -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in my-nginx.mesh-external.svc.cluster.local.csr -out my-nginx.mesh-external.svc.cluster.local.crt -extfile san.conf -extensions v3_req

  3. Generate client certificate and private key:

    1. $ openssl req -out client.example.com.csr -newkey rsa:2048 -nodes -keyout client.example.com.key -subj "/CN=client.example.com/O=client organization"
    2. $ openssl x509 -req -sha256 -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 1 -in client.example.com.csr -out client.example.com.crt

Deploy a mutual TLS server

To simulate an actual external service that supports the mutual TLS protocol, deploy an NGINX server in your Kubernetes cluster, but running outside of the Istio service mesh, i.e., in a namespace without Istio sidecar proxy injection enabled.

  1. Create a namespace to represent services outside the Istio mesh, namely mesh-external. Note that the sidecar proxy will not be automatically injected into the pods in this namespace since the automatic sidecar injection was not enabled on it.

    1. $ kubectl create namespace mesh-external

  2. Create Kubernetes Secrets to hold the server’s and CA certificates.

    1. $ kubectl create -n mesh-external secret tls nginx-server-certs --key my-nginx.mesh-external.svc.cluster.local.key --cert my-nginx.mesh-external.svc.cluster.local.crt
    2. $ kubectl create -n mesh-external secret generic nginx-ca-certs --from-file=example.com.crt

  3. Create a configuration file for the NGINX server:

    1. $ cat <<\EOF > ./nginx.conf
    2. events {
    3. }
    5. http {
    6.   log_format main '$remote_addr - $remote_user [$time_local]  $status '
    7.   '"$request" $body_bytes_sent "$http_referer" '
    8.   '"$http_user_agent" "$http_x_forwarded_for"';
    9.   access_log /var/log/nginx/access.log main;
    10.   error_log  /var/log/nginx/error.log;
    12.   server {
    13.     listen 443 ssl;
    15.     root /usr/share/nginx/html;
    16.     index index.html;
    18.     server_name my-nginx.mesh-external.svc.cluster.local;
    19.     ssl_certificate /etc/nginx-server-certs/tls.crt;
    20.     ssl_certificate_key /etc/nginx-server-certs/tls.key;
    21.     ssl_client_certificate /etc/nginx-ca-certs/example.com.crt;
    22.     ssl_verify_client on;
    23.   }
    24. }
    25. EOF

  4. Create a Kubernetes ConfigMap to hold the configuration of the NGINX server:

    1. $ kubectl create configmap nginx-configmap -n mesh-external --from-file=nginx.conf=./nginx.conf

  5. Deploy the NGINX server:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: v1
    3. kind: Service
    4. metadata:
    5.   name: my-nginx
    6.   namespace: mesh-external
    7.   labels:
    8.     run: my-nginx
    9. spec:
    10.   ports:
    11.   - port: 443
    12.     protocol: TCP
    13.   selector:
    14.     run: my-nginx
    15. ---
    16. apiVersion: apps/v1
    17. kind: Deployment
    18. metadata:
    19.   name: my-nginx
    20.   namespace: mesh-external
    21. spec:
    22.   selector:
    23.     matchLabels:
    24.       run: my-nginx
    25.   replicas: 1
    26.   template:
    27.     metadata:
    28.       labels:
    29.         run: my-nginx
    30.     spec:
    31.       containers:
    32.       - name: my-nginx
    33.         image: nginx
    34.         ports:
    35.         - containerPort: 443
    36.         volumeMounts:
    37.         - name: nginx-config
    38.           mountPath: /etc/nginx
    39.           readOnly: true
    40.         - name: nginx-server-certs
    41.           mountPath: /etc/nginx-server-certs
    42.           readOnly: true
    43.         - name: nginx-ca-certs
    44.           mountPath: /etc/nginx-ca-certs
    45.           readOnly: true
    46.       volumes:
    47.       - name: nginx-config
    48.         configMap:
    49.           name: nginx-configmap
    50.       - name: nginx-server-certs
    51.         secret:
    52.           secretName: nginx-server-certs
    53.       - name: nginx-ca-certs
    54.         secret:
    55.           secretName: nginx-ca-certs
    56. EOF

Configure mutual TLS origination for egress traffic

  1. Create a Kubernetes Secret in the same namespace as the egress gateway is deployed in, to hold the client’s certificates:
  1. $ kubectl create secret -n istio-system generic client-credential --from-file=tls.key=client.example.com.key \
  2.   --from-file=tls.crt=client.example.com.crt --from-file=ca.crt=example.com.crt

To support integration with various tools, Istio supports a few different Secret formats. In this example, a single generic Secret with keys tls.key, tls.crt, and ca.crt is used.

Optionally, the credential may include a certificate revocation list (CRL) using the key ca.crl. If so, add another argument to the above example to provide the CRL: –from-file=ca.crl=/some/path/to/your-crl.pem.

  1. $ kubectl create secret -n default generic client-credential --from-file=tls.key=client.example.com.key \
  2.   --from-file=tls.crt=client.example.com.crt --from-file=ca.crt=example.com.crt

To support integration with various tools, Istio supports a few different Secret formats. In this example, a single generic Secret with keys tls.key, tls.crt, and ca.crt is used.

Optionally, the credential may include a certificate revocation list (CRL) using the key ca.crl. If so, add another argument to the above example to provide the CRL: –from-file=ca.crl=/some/path/to/your-crl.pem.

  1. Create an egress Gateway for my-nginx.mesh-external.svc.cluster.local, port 443, and a destination rule for sidecar requests that will be directed to the egress gateway:
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: Gateway
  4. metadata:
  5.   name: istio-egressgateway
  6. spec:
  7.   selector:
  8.     istio: egressgateway
  9.   servers:
  10.   - port:
  11.       number: 443
  12.       name: https
  13.       protocol: HTTPS
  14.     hosts:
  15.     - my-nginx.mesh-external.svc.cluster.local
  16.     tls:
  17.       mode: ISTIO_MUTUAL
  18. ---
  19. apiVersion: networking.istio.io/v1alpha3
  20. kind: DestinationRule
  21. metadata:
  22.   name: egressgateway-for-nginx
  23. spec:
  24.   host: istio-egressgateway.istio-system.svc.cluster.local
  25.   subsets:
  26.   - name: nginx
  27.     trafficPolicy:
  28.       loadBalancer:
  29.         simple: ROUND_ROBIN
  30.       portLevelSettings:
  31.       - port:
  32.           number: 443
  33.         tls:
  34.           mode: ISTIO_MUTUAL
  35.           sni: my-nginx.mesh-external.svc.cluster.local
  36. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: gateway.networking.k8s.io/v1
  3. kind: Gateway
  4. metadata:
  5.   name: nginx-egressgateway
  6.   annotations:
  7.     networking.istio.io/service-type: ClusterIP
  8. spec:
  9.   gatewayClassName: istio
  10.   listeners:
  11.   - name: https
  12.     hostname: my-nginx.mesh-external.svc.cluster.local
  13.     port: 443
  14.     protocol: HTTPS
  15.     tls:
  16.       mode: Terminate
  17.       options:
  18.         gateway.istio.io/tls-terminate-mode: ISTIO_MUTUAL
  19.     allowedRoutes:
  20.       namespaces:
  21.         from: Same
  22. ---
  23. apiVersion: rbac.authorization.k8s.io/v1
  24. kind: Role
  25. metadata:
  26.   name: nginx-egressgateway-istio-sds
  27. rules:
  28. - apiGroups:
  29.   - ""
  30.   resources:
  31.   - secrets
  32.   verbs:
  33.   - get
  34.   - watch
  35.   - list
  36. ---
  37. apiVersion: rbac.authorization.k8s.io/v1
  38. kind: RoleBinding
  39. metadata:
  40.   name: nginx-egressgateway-istio-sds
  41. roleRef:
  42.   apiGroup: rbac.authorization.k8s.io
  43.   kind: Role
  44.   name: nginx-egressgateway-istio-sds
  45. subjects:
  46. - kind: ServiceAccount
  47.   name: nginx-egressgateway-istio
  48. ---
  49. apiVersion: networking.istio.io/v1alpha3
  50. kind: DestinationRule
  51. metadata:
  52.   name: egressgateway-for-nginx
  53. spec:
  54.   host: nginx-egressgateway-istio.default.svc.cluster.local
  55.   trafficPolicy:
  56.     loadBalancer:
  57.       simple: ROUND_ROBIN
  58.     portLevelSettings:
  59.     - port:
  60.         number: 443
  61.       tls:
  62.         mode: ISTIO_MUTUAL
  63.         sni: my-nginx.mesh-external.svc.cluster.local
  64. EOF
  1. Configure route rules to direct traffic through the egress gateway:
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: VirtualService
  4. metadata:
  5.   name: direct-nginx-through-egress-gateway
  6. spec:
  7.   hosts:
  8.   - my-nginx.mesh-external.svc.cluster.local
  9.   gateways:
  10.   - istio-egressgateway
  11.   - mesh
  12.   http:
  13.   - match:
  14.     - gateways:
  15.       - mesh
  16.       port: 80
  17.     route:
  18.     - destination:
  19.         host: istio-egressgateway.istio-system.svc.cluster.local
  20.         subset: nginx
  21.         port:
  22.           number: 443
  23.       weight: 100
  24.   - match:
  25.     - gateways:
  26.       - istio-egressgateway
  27.       port: 443
  28.     route:
  29.     - destination:
  30.         host: my-nginx.mesh-external.svc.cluster.local
  31.         port:
  32.           number: 443
  33.       weight: 100
  34. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: VirtualService
  4. metadata:
  5.   name: direct-nginx-to-egress-gateway
  6. spec:
  7.   hosts:
  8.   - my-nginx.mesh-external.svc.cluster.local
  9.   gateways:
  10.   - mesh
  11.   http:
  12.   - match:
  13.     - port: 80
  14.     route:
  15.     - destination:
  16.         host: nginx-egressgateway-istio.default.svc.cluster.local
  17.         port:
  18.           number: 443
  19. ---
  20. apiVersion: gateway.networking.k8s.io/v1
  21. kind: HTTPRoute
  22. metadata:
  23.   name: forward-nginx-from-egress-gateway
  24. spec:
  25.   parentRefs:
  26.   - name: nginx-egressgateway
  27.   hostnames:
  28.   - my-nginx.mesh-external.svc.cluster.local
  29.   rules:
  30.   - backendRefs:
  31.     - name: my-nginx
  32.       namespace: mesh-external
  33.       port: 443
  34. ---
  35. apiVersion: gateway.networking.k8s.io/v1beta1
  36. kind: ReferenceGrant
  37. metadata:
  38.   name: my-nginx-reference-grant
  39.   namespace: mesh-external
  40. spec:
  41.   from:
  42.     - group: gateway.networking.k8s.io
  43.       kind: HTTPRoute
  44.       namespace: default
  45.   to:
  46.     - group: ""
  47.       kind: Service
  48.       name: my-nginx
  49. EOF

TODO: figure out why using an HTTPRoute, instead of the above VirtualService, doesn’t work. It completely ignores the HTTPRoute and tries to pass through to the destination service, which times out. The only difference from the above VirtualService is that the generated VirtualService includes annotation: internal.istio.io/route-semantics": "gateway".

  1. Add a DestinationRule to perform mutual TLS origination:
  1. $ kubectl apply -n istio-system -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: DestinationRule
  4. metadata:
  5.   name: originate-mtls-for-nginx
  6. spec:
  7.   host: my-nginx.mesh-external.svc.cluster.local
  8.   trafficPolicy:
  9.     loadBalancer:
  10.       simple: ROUND_ROBIN
  11.     portLevelSettings:
  12.     - port:
  13.         number: 443
  14.       tls:
  15.         mode: MUTUAL
  16.         credentialName: client-credential # this must match the secret created earlier to hold client certs
  17.         sni: my-nginx.mesh-external.svc.cluster.local
  18.         # subjectAltNames: # can be enabled if the certificate was generated with SAN as specified in previous section
  19.         # - my-nginx.mesh-external.svc.cluster.local
  20. EOF
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: DestinationRule
  4. metadata:
  5.   name: originate-mtls-for-nginx
  6. spec:
  7.   host: my-nginx.mesh-external.svc.cluster.local
  8.   trafficPolicy:
  9.     loadBalancer:
  10.       simple: ROUND_ROBIN
  11.     portLevelSettings:
  12.     - port:
  13.         number: 443
  14.       tls:
  15.         mode: MUTUAL
  16.         credentialName: client-credential # this must match the secret created earlier to hold client certs
  17.         sni: my-nginx.mesh-external.svc.cluster.local
  18.         # subjectAltNames: # can be enabled if the certificate was generated with SAN as specified in previous section
  19.         # - my-nginx.mesh-external.svc.cluster.local
  20. EOF
  1. Verify that the credential is supplied to the egress gateway and active:
  1. $ istioctl -n istio-system proxy-config secret deploy/istio-egressgateway | grep client-credential
  2. kubernetes://client-credential            Cert Chain     ACTIVE     true           1                                          2024-06-04T12:46:28Z     2023-06-05T12:46:28Z
  3. kubernetes://client-credential-cacert     Cert Chain     ACTIVE     true           16491643791048004260                       2024-06-04T12:46:28Z     2023-06-05T12:46:28Z
  1. $ istioctl proxy-config secret deploy/nginx-egressgateway-istio | grep client-credential
  2. kubernetes://client-credential            Cert Chain     ACTIVE     true           1                                          2024-06-04T12:46:28Z     2023-06-05T12:46:28Z
  3. kubernetes://client-credential-cacert     Cert Chain     ACTIVE     true           16491643791048004260                       2024-06-04T12:46:28Z     2023-06-05T12:46:28Z

  1. Send an HTTP request to http://my-nginx.mesh-external.svc.cluster.local:

    1. $ kubectl exec "$(kubectl get pod -l app=sleep -o jsonpath={.items..metadata.name})" -c sleep -- curl -sS http://my-nginx.mesh-external.svc.cluster.local
    2. <!DOCTYPE html>
    3. <html>
    4. <head>
    5. <title>Welcome to nginx!</title>
    6. ...

  2. Check the log of the egress gateway’s proxy:

If Istio is deployed in the istio-system namespace, the command to print the log is:

  1. $ kubectl logs -l istio=egressgateway -n istio-system | grep 'my-nginx.mesh-external.svc.cluster.local' | grep HTTP

You should see a line similar to the following:

  1. [2018-08-19T18:20:40.096Z] "GET / HTTP/1.1" 200 - 0 612 7 5 "172.30.146.114" "curl/7.35.0" "b942b587-fac2-9756-8ec6-303561356204" "my-nginx.mesh-external.svc.cluster.local" "172.21.72.197:443"

Access the log corresponding to the egress gateway using the Istio-generated pod label:

  1. $ kubectl logs -l gateway.networking.k8s.io/gateway-name=nginx-egressgateway | grep 'my-nginx.mesh-external.svc.cluster.local' | grep HTTP

You should see a line similar to the following:

  1. [2024-04-08T20:08:18.451Z] "GET / HTTP/1.1" 200 - via_upstream - "-" 0 615 5 5 "172.30.239.41" "curl/7.87.0-DEV" "86e54df0-6dc3-46b3-a8b8-139474c32a4d" "my-nginx.mesh-external.svc.cluster.local" "172.30.239.57:443" outbound|443||my-nginx.mesh-external.svc.cluster.local 172.30.239.53:48530 172.30.239.53:443 172.30.239.41:53694 my-nginx.mesh-external.svc.cluster.local default.forward-nginx-from-egress-gateway.0

Cleanup the mutual TLS origination example

  1. Remove the NGINX mutual TLS server resources:

    1. $ kubectl delete secret nginx-server-certs nginx-ca-certs -n mesh-external
    2. $ kubectl delete configmap nginx-configmap -n mesh-external
    3. $ kubectl delete service my-nginx -n mesh-external
    4. $ kubectl delete deployment my-nginx -n mesh-external
    5. $ kubectl delete namespace mesh-external

  2. Remove the gateway configuration resources:

  1. $ kubectl delete secret client-credential -n istio-system
  2. $ kubectl delete gw istio-egressgateway
  3. $ kubectl delete virtualservice direct-nginx-through-egress-gateway
  4. $ kubectl delete destinationrule -n istio-system originate-mtls-for-nginx
  5. $ kubectl delete destinationrule egressgateway-for-nginx
  1. $ kubectl delete secret client-credential
  2. $ kubectl delete gtw nginx-egressgateway
  3. $ kubectl delete role nginx-egressgateway-istio-sds
  4. $ kubectl delete rolebinding nginx-egressgateway-istio-sds
  5. $ kubectl delete virtualservice direct-nginx-to-egress-gateway
  6. $ kubectl delete httproute forward-nginx-from-egress-gateway
  7. $ kubectl delete destinationrule originate-mtls-for-nginx
  8. $ kubectl delete destinationrule egressgateway-for-nginx
  9. $ kubectl delete referencegrant my-nginx-reference-grant -n mesh-external

  1. Delete the certificates and private keys:

    1. $ rm example.com.crt example.com.key my-nginx.mesh-external.svc.cluster.local.crt my-nginx.mesh-external.svc.cluster.local.key my-nginx.mesh-external.svc.cluster.local.csr client.example.com.crt client.example.com.csr client.example.com.key

  2. Delete the generated configuration files used in this example:

    1. $ rm ./nginx.conf

Cleanup

Delete the sleep service and deployment:

Zip

  1. $ kubectl delete -f @samples/sleep/sleep.yaml@