Upgrade Problems

EnvoyFilter migration

EnvoyFilter is an alpha API that is tightly coupled to the implementation details of Istio xDS configuration generation. Production use of the EnvoyFilter alpha API must be carefully curated during the upgrade of Istio’s control or data plane. In many instances, EnvoyFilter can be replaced with a first-class Istio API which carries substantially lower upgrade risks.

Use Telemetry API for metrics customization

The usage of IstioOperator to customize Prometheus metrics generation has been replaced by the Telemetry API, because IstioOperator relies on a template EnvoyFilter to change the metrics filter configuration. Note that the two methods are incompatible, and the Telemetry API does not work with EnvoyFilter or IstioOperator metric customization configuration.

As an example, the following IstioOperator configuration adds a destination_port tag:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   values:
  5.     telemetry:
  6.       v2:
  7.         prometheus:
  8.           configOverride:
  9.             inboundSidecar:
  10.               metrics:
  11.                 - name: requests_total
  12.                   dimensions:
  13.                     destination_port: string(destination.port)

The following Telemetry configuration replaces the above:

  1. apiVersion: telemetry.istio.io/v1alpha1
  2. kind: Telemetry
  3. metadata:
  4.   name: namespace-metrics
  5. spec:
  6.   metrics:
  7.   - providers:
  8.     - name: prometheus
  9.     overrides:
  10.     - match:
  11.         metric: REQUEST_COUNT
  12.       mode: SERVER
  13.       tagOverrides:
  14.         destination_port:
  15.           value: "string(destination.port)"

Use the WasmPlugin API for Wasm data plane extensibility

The usage of EnvoyFilter to inject Wasm filters has been replaced by the WasmPlugin API. WasmPlugin API allows dynamic loading of the plugins from artifact registries, URLs, or local files. The “Null” plugin runtime is no longer a recommended option for deployment of Wasm code.

Use gateway topology to set the number of the trusted hops

The usage of EnvoyFilter to configure the number of the trusted hops in the HTTP connection manager has been replaced by the gatewayTopology field in ProxyConfig. For example, the following EnvoyFilter configuration should use an annotation on the pod or the mesh default. Instead of:

  1. apiVersion: networking.istio.io/v1alpha3
  2. kind: EnvoyFilter
  3. metadata:
  4.   name: ingressgateway-redirect-config
  5. spec:
  6.   configPatches:
  7.   - applyTo: NETWORK_FILTER
  8.     match:
  9.       context: GATEWAY
  10.       listener:
  11.         filterChain:
  12.           filter:
  13.             name: envoy.filters.network.http_connection_manager
  14.     patch:
  15.       operation: MERGE
  16.       value:
  17.         typed_config:
  18.           '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
  19.           xff_num_trusted_hops: 1
  20.   workloadSelector:
  21.     labels:
  22.       istio: ingress-gateway

Use the equivalent ingress gateway pod proxy configuration annotation:

  1. metadata:
  2.   annotations:
  3.     "proxy.istio.io/config": '{"gatewayTopology" : { "numTrustedProxies": 1 }}'

Use gateway topology to enable PROXY protocol on the ingress gateways

The usage of EnvoyFilter to enable PROXY protocol on the ingress gateways has been replaced by the gatewayTopology field in ProxyConfig. For example, the following EnvoyFilter configuration should use an annotation on the pod or the mesh default. Instead of:

  1. apiVersion: networking.istio.io/v1alpha3
  2. kind: EnvoyFilter
  3. metadata:
  4.   name: proxy-protocol
  5. spec:
  6.   configPatches:
  7.   - applyTo: LISTENER_FILTER
  8.     patch:
  9.       operation: INSERT_FIRST
  10.       value:
  11.         name: proxy_protocol
  12.         typed_config:
  13.           "@type": "type.googleapis.com/envoy.extensions.filters.listener.proxy_protocol.v3.ProxyProtocol"
  14.   workloadSelector:
  15.     labels:
  16.       istio: ingress-gateway

Use the equivalent ingress gateway pod proxy configuration annotation:

  1. metadata:
  2.   annotations:
  3.     "proxy.istio.io/config": '{"gatewayTopology" : { "proxyProtocol": {} }}'

Use a proxy annotation to customize the histogram bucket sizes

The usage of EnvoyFilter and the experimental bootstrap discovery service to configure the bucket sizes for the histogram metrics has been replaced by the proxy annotation sidecar.istio.io/statsHistogramBuckets. For example, the following EnvoyFilter configuration should use an annotation on the pod. Instead of:

  1. apiVersion: networking.istio.io/v1alpha3
  2. kind: EnvoyFilter
  3. metadata:
  4.   name: envoy-stats-1
  5.   namespace: istio-system
  6. spec:
  7.   workloadSelector:
  8.     labels:
  9.       istio: ingressgateway
  10.   configPatches:
  11.   - applyTo: BOOTSTRAP
  12.     patch:
  13.       operation: MERGE
  14.       value:
  15.         stats_config:
  16.           histogram_bucket_settings:
  17.             - match:
  18.                 prefix: istiocustom
  19.               buckets: [1,5,50,500,5000,10000]

Use the equivalent pod annotation:

  1. metadata:
  2.   annotations:
  3.     "sidecar.istio.io/statsHistogramBuckets": '{"istiocustom":[1,5,50,500,5000,10000]}'