Configuring Gateway Network Topology (Development)

Configuring network topologies (Development)

This feature is actively in development and is considered pre-alpha.

Istio provides the ability to manage settings like X-Forwarded-For (XFF) and X-Forwarded-Client-Cert (XFCC), which are dependent on how the gateway workloads are deployed. This is currently an in-development feature. For more information on X-Forwarded-For, see the IETF’s RFC.

You might choose to deploy Istio ingress gateways in various network topologies (e.g. behind Cloud Load Balancers, a self-managed Load Balancer or directly expose the Istio ingress gateway to the Internet). As such, these topologies require different ingress gateway configurations for transporting correct client attributes like IP addresses and certificates to the workloads running in the cluster.

Configuration of XFF and XFCC headers is managed via MeshConfig during Istio installation or by adding a pod annotation. Note that the Meshconfig configuration is a global setting for all gateway workloads, while pod annotations override the global setting on a per-workload basis.

To simplify configuring network topology during installation, create a single YAML file to pass to istioctl:

  1. $ cat <<EOF > topology.yaml
  2. apiVersion: install.istio.io/v1alpha1
  3. kind: IstioOperator
  4. spec:
  5. meshConfig:
  6. defaultConfig:
  7. gatewayTopology:
  8. EOF

If Istio ingress gateway was already running prior to application of the MeshConfig, restart any Istio ingress gateway pods.

You can configure both of these settings using the proxy.istio.io/config annotation to the Pod spec of your Istio ingress gateway.

  1. ...
  2. metadata:
  3. annotations:
  4. "proxy.istio.io/config": '{"gatewayTopology" : { "numTrustedProxies": 2 } }'

Configuring X-Forwarded-For Headers

Applications rely on reverse proxies to forward client attributes in a request, such as X-Forward-For header. However, due to the variety of network topologies Istio can be deployed in, you must set the number of trusted proxies deployed in front of the Istio gateway proxy, so that the client address can be extracted correctly.

To set the number of trusted proxies, add the following to your topology.yaml file.

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4. meshConfig:
  5. defaultConfig:
  6. gatewayTopology:
  7. numTrustedProxies: <VALUE>

For example, if you have a cloud based Load Balancer, a reverse proxy, and an Istio gateway proxy, then <VALUE> would be 2.

Note that all proxies in front of the Istio gateway proxy must parse HTTP traffic and append to the X-Forwarded-For header at each hop. If the number of entries in the X-Forwarded-For header is less than the number of trusted hops configured, Envoy falls back to using the immediate downstream address as the trusted client address. Please refer to the Envoy documentation to understand how X-Forwarded-For headers and trusted client addresses are determined.

Example using X-Forwarded-For capability with httpbin

  1. Specify numTrustedProxies as 2 either using MeshConfig or an proxy.istop/io/config annotation. If you are using MeshConfig, run the following command to create a file named topology.yaml and apply it to your cluster:

    1. $ cat <<EOF > topology.yaml
    2. apiVersion: install.istio.io/v1alpha1
    3. kind: IstioOperator
    4. spec:
    5. meshConfig:
    6. defaultConfig:
    7. gatewayTopology:
    8. numTrustedProxies: 2
    9. EOF
    10. $ istioctl install -f topology.yaml

    If you previously installed an Istio ingress gateway, restart all ingress gateway pods after step 1.

  2. Create an httpbin namespace:

    1. $ kubectl create namespace httpbin
    2. namespace/httpbin created
  3. Set the istio-injection label to enabled for sidecar injection:

    1. $ kubectl label --overwrite namespace httpbin istio-injection=enabled
    2. namespace/httpbin labeled
  4. Deploy httpbin in the httpbin namespace:

    1. $ kubectl apply -n httpbin -f samples/httpbin/httpbin.yaml
  5. Deploy a gateway associated with httpbin:

    1. $ kubectl apply -n httpbin -f samples/httpbin/httpbin-gateway.yaml
  6. Set a local GATEWAY_URL environmental variable based on your Istio ingress gateway’s IP address:

    1. $ export GATEWAY_URL=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  7. Run the following curl command to verify the X-Envoy-External-Address and X-Forwarded-For are set correctly:

    1. $ curl -H 'X-Forwarded-For: 56.5.6.7, 72.9.5.6, 98.1.2.3' $GATEWAY_URL/get?show_env=true
    2. {
    3. "args": {
    4. "show_env": "true"
    5. },
    6. "headers": {
    7. ...
    8. "X-Envoy-External-Address": "72.9.5.6",
    9. ...
    10. "X-Forwarded-For": "56.5.6.7, 72.9.5.6, 98.1.2.3, <YOUR GATEWAY IP>",
    11. ...
    12. },
    13. ...
    14. }

Note that the X-Envoy-External-Address is set to the “second” from last address in the X-Forwarded-For header as per your numTrustedProxies setting. Additionally, the gateway workload appends its IP in the X-Forwarded-For header before forwarding it to the upstream httpbin workload.

Configuring X-Forwarded-Client-Cert Headers

From Envoy’s documentation regarding XFCC:

x-forwarded-client-cert (XFCC) is a proxy header which indicates certificate information of part or all of the clients or proxies that a request has flowed through, on its way from the client to the server. A proxy may choose to sanitize/append/forward the XFCC header before proxying the request.

To configure how XFCC Headers are handled, add the following to your topology.yaml file.

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4. meshConfig:
  5. defaultConfig:
  6. gatewayTopology:
  7. forwardClientCertDetails: <ENUM_VALUE>

where ENUM_VALUE can be of the following type.

ENUM_VALUE
UNDEFINEDField is not set.
SANITIZEDo not send the XFCC header to the next hop. This is the default value for a gateway.
FORWARD_ONLYWhen the client connection is mTLS (Mutual TLS), forward the XFCC header in the request.
APPEND_FORWARDWhen the client connection is mTLS, append the client certificate information to the request’s XFCC header and forward it.
SANITIZE_SETWhen the client connection is mTLS, reset the XFCC header with the client certificate information and send it to the next hop.
ALWAYS_FORWARD_ONLYAlways forward the XFCC header in the request, regardless of whether the client connection is mTLS.

See the Envoy documentation for examples of using this capability.

See also

Direct encrypted traffic from IBM Cloud Kubernetes Service Ingress to Istio Ingress Gateway

Configure the IBM Cloud Kubernetes Service Application Load Balancer to direct traffic to the Istio Ingress gateway with mutual TLS.

Multicluster Istio configuration and service discovery using Admiral

Automating Istio configuration for Istio deployments (clusters) that work as a single mesh.

Istio as a Proxy for External Services

Configure Istio ingress gateway to act as a proxy for external services.

Multi-Mesh Deployments for Isolation and Boundary Protection

Deploy environments that require isolation into separate meshes and enable inter-mesh communication by mesh federation.

Secure Control of Egress Traffic in Istio, part 3

Comparison of alternative solutions to control egress traffic including performance considerations.

Secure Control of Egress Traffic in Istio, part 2

Use Istio Egress Traffic Control to prevent attacks involving egress traffic.