Multi-cluster Traffic Management
Within a multicluster mesh, traffic rules specific to the cluster topology may be desirable. This document describes a few ways to manage traffic in a multicluster mesh. Before reading this guide:
- Read Deployment Models
- Make sure your deployed services follow the concept of namespace sameness.
Keeping traffic in-cluster
In some cases the default cross-cluster load balancing behavior is not desirable. To keep traffic “cluster-local” (i.e. traffic sent from cluster-a
will only reach destinations in cluster-a
), mark hostnames or wildcards as clusterLocal
using MeshConfig.serviceSettings.
For example, you can enforce cluster-local traffic for an individual service, all services in a particular namespace, or globally for all services in the mesh, as follows:
per-service per-namespace global
serviceSettings:
- settings:
clusterLocal: true
hosts:
- "mysvc.myns.svc.cluster.local"
serviceSettings:
- settings:
clusterLocal: true
hosts:
- "*.myns.svc.cluster.local"
serviceSettings:
- settings:
clusterLocal: true
hosts:
- "*"
Partitioning Services
DestinationRule.subsets allows partitioning a service by selecting labels. These labels can be the labels from Kubernetes metadata, or from built-in labels. One of these built-in labels, topology.istio.io/cluster
, in the subset selector for a DestinationRule
allows creating per-cluster subsets.
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: mysvc-per-cluster-dr
spec:
host: mysvc.myns.svc.cluster.local
subsets:
- name: cluster-1
labels:
topology.istio.io/cluster: cluster-1
- name: cluster-2
labels:
topology.istio.io/cluster: cluster-2
Using these subsets you can create various routing rules based on the cluster such as mirroring or shifting.
This provides another option to create cluster-local traffic rules by restricting the destination subset in a VirtualService
:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: mysvc-cluster-local-vs
spec:
hosts:
- mysvc.myns.svc.cluster.local
http:
- name: "cluster-1-local"
match:
- sourceLabels:
topology.istio.io/cluster: "cluster-1"
route:
- destination:
host: mysvc.myns.svc.cluster.local
subset: cluster-1
- name: "cluster-2-local"
match:
- sourceLabels:
topology.istio.io/cluster: "cluster-2"
route:
- destination:
host: mysvc.myns.svc.cluster.local
subset: cluster-2
Using subset-based routing this way to control cluster-local traffic, as opposed to MeshConfig.serviceSettings, has the downside of mixing service-level policy with topology-level policy. For example, a rule that sends 10% of traffic to v2
of a service will need twice the number of subsets (e.g., cluster-1-v2
, cluster-2-v2
). This approach is best limited to situations where more granular control of cluster-based routing is needed.