Load Balancing

Load balancing is a method of distributing API request traffic across multiple upstream services. Load balancing improves overall system responsiveness and reduces failures by preventing overloading of individual resources.

In the following example, you’ll use an application deployed across two different servers, or upstream targets. Kong Gateway needs to load balance across both servers, so that if one of the servers is unavailable, it automatically detects the problem and routes all traffic to the working server.

An upstream refers to the service applications sitting behind Kong Gateway, to which client requests are forwarded. In Kong Gateway, an upstream represents a virtual hostname and can be used to health check, circuit break, and load balance incoming requests over multiple target backend services.

In this section, you’ll re-configure the service created earlier, (example_service) to point to an upstream instead of a specific host. For the purposes of our example, the upstream will point to two different targets, httpbin.org and httpbun.com. More commonly, targets will be instances of the same backend service running on different host systems.

Here is a diagram illustrating the setup:

  2. flowchart LR
  3.   A("`Route 
  4.   (/mock)`")
  5.   B("`Service
  6.   (example_service)`")
  7.   C(Load balancer)
  8.   D(httpbin.org)
  9.   E(httpbun.com)
  11.   subgraph id1 ["`**KONG GATEWAY**`"]
  12.     A --> B --> C
  13.   end
  15.   subgraph id2 ["`Targets (example_upstream)`"]
  16.     C --> D & E
  17.   end
  19.   style id1 rx:10,ry:10
  20.   style id2 stroke:none

Enable load balancing

In this section, you will create an upstream named example_upstream and add two targets to it.

Prerequisites

This chapter is part of the Get Started with Kong series. For the best experience, it is recommended that you follow the series from the beginning.

Start with the introduction, Get Kong, which includes a list of prerequisites and instructions for running a local Kong Gateway.

Step two of the guide, Services and Routes, includes instructions for installing a mock service used throughout this series.

If you haven’t completed these steps already, complete them before proceeding.

Steps to enable load balancing

  1. Create an upstream

    Use the Admin API to create an upstream named example_upstream:

    1. curl -X POST http://localhost:8001/upstreams \
    2.   --data name=example_upstream

  2. Create upstream targets

    Create two targets for example_upstream. Each request creates a new target, and sets the backend service connection endpoint:

    1. curl -X POST http://localhost:8001/upstreams/example_upstream/targets \
    2.   --data target='httpbun.com:80'
    3. curl -X POST http://localhost:8001/upstreams/example_upstream/targets \
    4.   --data target='httpbin.org:80'

  3. Update the service

    In the services and routes section of this guide, you created example_service which pointed to an explicit host, http://httpbun.com. Now you’ll modify that service to point to the upstream instead:

    1. curl -X PATCH http://localhost:8001/services/example_service \
    2.   --data host='example_upstream'

    You now have an upstream with two targets, httpbin.org and httpbun.com, and a service pointing to that upstream.

  4. Validate

    Validate that the upstream you configured is working by visiting the route http://localhost:8000/mock using a web browser or CLI.

    • Web browser: Visit http://localhost:8000/mock and refresh the page several times to see the site change from httpbin to httpbun.
    • CLI: Execute the command curl -s http://localhost:8000/mock/headers |grep -i -A1 '"host"' several times. You will see the hostname change between httpbin and httpbun.

What’s next

You’ve completed the Get Started with Kong guide, but a lot more is possible with Kong Gateway. The following are guides to advanced features of Kong Gateway:

Previous Key Authentication

Next Get started with AI Gateway