Locality Weighted Load Balancing

Requirements

Sandbox environment

Setup your sandbox environment with Docker and Docker Compose, and clone the Envoy repository with Git.

curl

Used to make HTTP requests.

This example demonstrates the locality weighted load balancing feature in Envoy proxy. The demo simulates a scenario that a backend service resides in two local zones and one remote zone.

The components used in this demo are as follows:

  • A client container: runs Envoy proxy

  • Backend container in the same locality as the client, with priority set to 0, referred to as local-1.

  • Backend container in the same locality as the client, with priority set to 1, referred to as local-2.

  • Backend container in the the remote locality, with priority set to 1, referred to as remote-1.

  • Backend container in the the remote locality, with priority set to 2, referred to as remote-2.

The client Envoy proxy configures the 4 backend containers in the same Envoy cluster, so that Envoy handles load balancing to those backend servers. From here we can see, we have localities with 3 different priorities:

  • priority 0: local-1

  • priority 1: local-2 and remote-1

  • priority 2: remote-2

In Envoy, when the healthiness of a given locality drops below a threshold (71% by default), the next priority locality will start to share the request loads. The demo below will show this behavior.

Step 1: Start all of our containers

In terminal, move to the examples/locality_load_balancing directory.

To build this sandbox example and start the example services, run the following commands:

  1. # Start demo
  2. $ docker-compose up --build -d

The locality configuration is set in the client container via static Envoy configuration file. Please refer to the cluster section of the proxy configuration file.

Step 2: Scenario with one replica in the highest priority locality

In this scenario, each locality has 1 healthy replica running and all the requests should be sent to the locality with the highest priority (i.e. lowest integer set for priority - 0), which is local-1.

  1. # all requests to local-1
  2. $ docker-compose exec -T client-envoy python3 client.py http://localhost:3000/ 100
  3. Hello from backend-local-1!: 100, 100.0%
  4. Failed: 0

If locality local-1 becomes unhealthy (i.e. fails the Envoy health check), the requests should be load balanced among the subsequent priority localities, which are local-2 and remote-1. They both have priority 1. We then send 100 requests to the backend cluster, and check the responders.

  1. # bring down local-1
  2. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-1_1:8000/unhealthy
  3. [backend-local-1] Set to unhealthy
  5. # local-2 and remote-1 localities split the traffic 50:50
  6. $ docker-compose exec -T client-envoy python3 client.py http://localhost:3000/ 100
  7. Hello from backend-remote-1!: 51, 51.0%
  8. Hello from backend-local-2!: 49, 49.0%
  9. Failed: 0

Now if local-2 becomes unhealthy also, priority 1 locality is only 50% healthy. Thus priority 2 locality starts to share the request load. Requests will be sent to both remote-1 and remote-2.

  1. # bring down local-2
  2. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-2_1:8000/unhealthy
  4. # remote-1 locality receive 100% of the traffic
  5. $ docker-compose exec -T client-envoy python3 client.py http://localhost:3000/ 100
  6. Hello from backend-remote-1!: actual weight 69.0%
  7. Hello from backend-remote-2!: actual weight 31.0%
  8. Failed: 0

Step 3: Recover servers

Before moving on, we need to server local-1 and local-2 first.

  1. # recover local-1 and local-2 after the demo
  2. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-1_1:8000/healthy
  3. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-2_1:8000/healthy

Step 4: Scenario with multiple replicas in the highest priority locality

To demonstrate how locality based load balancing works in multiple replicas setup, let’s now scale up the local-1 locality to 5 replicas.

  1. $ docker-compose up --scale backend-local-1=5 -d

We are going to show the scenario that local-1 is just partially healthy. So let’s bring down 4 of the replicas in local-1.

  1. # bring down local-1 replicas
  2. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-1_2:8000/unhealthy
  3. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-1_3:8000/unhealthy
  4. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-1_4:8000/unhealthy
  5. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-1_5:8000/unhealthy

Then we check the endpoints again:

  1. # check healthiness
  2. $ docker-compose exec -T client-envoy curl -s localhost:8001/clusters | grep health_flags
  4. backend::172.28.0.4:8000::health_flags::/failed_active_hc
  5. backend::172.28.0.2:8000::health_flags::/failed_active_hc
  6. backend::172.28.0.5:8000::health_flags::/failed_active_hc
  7. backend::172.28.0.6:8000::health_flags::/failed_active_hc
  8. backend::172.28.0.7:8000::health_flags::healthy
  9. backend::172.28.0.8:8000::health_flags::healthy
  10. backend::172.28.0.3:8000::health_flags::healthy

We can confirm that 4 backend endpoints become unhealthy.

Now we send the 100 requests again.

  1. # watch traffic change
  2. $ docker-compose exec -T client-envoy python3 client.py http://localhost:3000/ 100
  4. Hello from backend-remote-1!: actual weight 37.0%
  5. Hello from backend-local-2!: actual weight 36.0%
  6. Hello from backend-local-1!: actual weight 27.0%
  7. Failed: 0

As local-1 does not have enough healthy workloads, requests are partially shared by secondary localities.

If we bring down all the servers in priority 1 locality, it will make priority 1 locality 0% healthy. The traffic should split between priority 0 and priority 2 localities.

  1. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-local-2_1:8000/unhealthy
  2. $ docker-compose exec -T client-envoy curl -s locality-load-balancing_backend-remote-1_1:8000/unhealthy
  3. $ docker-compose exec -T client-envoy python3 client.py http://localhost:3000/ 100
  5. Hello from backend-remote-2!: actual weight 77.0%
  6. Hello from backend-local-1!: actual weight 23.0%
  7. Failed: 0