Use PorterLB in BGP Mode

This document demonstrates how to use PorterLB in BGP mode to expose a Service backed by two Pods. The BgpConf, BgpPeer, Eip, Deployment and Service described in this document are examples only and you need to customize the commands and YAML configurations based on your requirements.

Instead of using a real router, this document uses a Linux server with BIRD to simulate a router so that users without a real router can also use PorterLB in BGP mode for tests.

Prerequisites

  • You need to prepare a Kubernetes cluster where PorterLB has been installed.
  • You need to prepare a Linux server that communicates with the Kubernetes cluster properly. BIRD will be installed on the server to simulate a BGP router.
  • If you use a real router instead of BIRD, the router must support BGP and Equal-Cost Multi-Path (ECMP) routing. In addition, the router must also support receiving multiple equivalent routes from the same neighbor.

This document uses the following devices as an example:

Device NameIP AddressDescription
master1192.168.0.2Kubernetes cluster master, where PorterLB is installed.
worker-p001192.168.0.3Kubernetes cluster worker 1
worker-p002192.168.0.4Kubernetes cluster worker 2
i-f3fozos0192.168.0.5BIRD machine, where BIRD will be installed to simulate a BGP router.

Step 1: Install and Configure BIRD

If you use a real router, you can skip this step and perform configuration on the router instead.

  1. Log in to the BIRD machine and run the following commands to install BIRD:

    1. sudo add-apt-repository ppa:cz.nic-labs/bird
    2. sudo apt-get update 
    3. sudo apt-get install bird
    4. sudo systemctl enable bird

    NOTE

    • BIRD 1.5 does not support ECMP. To use all features of PorterLB, you are advised to install BIRD 1.6 or later.
    • The preceding commands apply only to Debian-based OSs such as Debian and Ubuntu. On Red Hat-based OSs such as RHEL and CentOS, use yum instead.
    • You can also install BIRD according to the official BIRD documentation.

  2. Run the following command to edit the BIRD configuration file:

    1. vi /etc/bird/bird.conf

  3. Configure the BIRD configuration file as follows:

    1. router id 192.168.0.5;
    3. protocol kernel {
    4.     scan time 60;
    5.     import none;
    6.     export all;
    7.     merge paths on;
    8. }
    10. protocol bgp neighbor1 {
    11.     local as 50001;
    12.     neighbor 192.168.0.2 port 17900 as 50000;
    13.     source address 192.168.0.5;
    14.     import all;
    15.     export all;
    16.     enable route refresh off;
    17.     add paths on;
    18. }

    NOTE

    • For test usage, you only need to customize the following fields in the preceding configuration:

      router id: Router ID of the BIRD machine, which is usually set to the IP address of the BIRD machine.

      protocol bgp neighbor1:

      • local as: ASN of the BIRD machine, which must be different from the ASN of the Kubernetes cluster.
      • neighbor: Master node IP address, BGP port number, and ASN of the Kubernetes cluster. Use port 17900 instead of the default BGP port 179 to avoid conflicts with other BGP components in the system.
      • source address: IP address of the BIRD machine.

    • If multiple nodes in the Kubernetes are used as BGP neighbors, you need to configure multiple BGP neighbors in the BIRD configuration file.

    • For details about the BIRD configuration file, see the official BIRD documentation.

  4. Run the following command to restart BIRD:

    1. sudo systemctl restart bird

  5. Run the following command to check whether the status of BIRD is active:

    1. sudo systemctl status bird

    bird-status

    NOTE

    If the status of BIRD is not active, you can run the following command to check the error logs:

    1. journalctl -f -u bird

Step 2: Create a BgpConf Object

The BgpConf object is used to configure the local (Kubernetes cluster) BGP properties on PorterLB.

  1. Run the following command to create a YAML file for the BgpConf object:

    1. vi porter-bgp-conf.yaml

  2. Add the following information to the YAML file:

    1. apiVersion: network.kubesphere.io/v1alpha2
    2. kind: BgpConf
    3. metadata:
    4.   name: default
    5. spec:
    6.   as: 50000
    7.   listenPort: 17900
    8.   routerId: 192.168.0.2

    NOTE

    For details about the fields in the BgpConf YAML configuration, see Configure Local BGP Properties Using BgpConf.

  3. Run the following command to create the BgpConf object:

    1. kubectl apply -f porter-bgp-conf.yaml

Step 3: Create a BgpPeer Object

The BgpPeer object is used to configure the peer (BIRD machine) BGP properties on PorterLB.

  1. Run the following command to create a YAML file for the BgpPeer object:

    1. vi porter-bgp-peer.yaml

  2. Add the following information to the YAML file:

    1. apiVersion: network.kubesphere.io/v1alpha2
    2. kind: BgpPeer
    3. metadata:
    4.   name: porter-bgp-peer
    5. spec:
    6.   conf:
    7.     peerAs: 50001
    8.     neighborAddress: 192.168.0.5

    NOTE

    For details about the fields in the BgpPeer YAML configuration, see Configure Peer BGP Properties Using BgpPeer.

  3. Run the following command to create the BgpPeer object:

    1. kubectl apply -f porter-bgp-peer.yaml

Step 4: Create an Eip Object

The Eip object functions as an IP address pool for PorterLB.

  1. Run the following command to create a YAML file for the Eip object:

    1. vi porter-bgp-eip.yaml

  2. Add the following information to the YAML file:

    1. apiVersion: network.kubesphere.io/v1alpha2
    2. kind: Eip
    3. metadata:
    4.   name: porter-bgp-eip
    5. spec:
    6.   address: 172.22.0.2-172.22.0.10

    NOTE

    For details about the fields in the Eip YAML configuration, see Configure IP Address Pools Using Eip.

  3. Run the following command to create the Eip object:

    1. kubectl apply -f porter-bgp-eip.yaml

Step 5: Create a Deployment

The following creates a Deployment of two Pods using the luksa/kubia image. Each Pod returns its own Pod name to external requests.

  1. Run the following command to create a YAML file for the Deployment:

    1. vi porter-bgp.yaml

  2. Add the following information to the YAML file:

    1. apiVersion: apps/v1
    2. kind: Deployment
    3. metadata:
    4.   name: porter-bgp
    5. spec:
    6.   replicas: 2
    7.   selector:
    8.     matchLabels:
    9.       app: porter-bgp
    10.   template:
    11.     metadata:
    12.       labels:
    13.         app: porter-bgp
    14.     spec:
    15.       containers:
    16.         - image: luksa/kubia
    17.           name: kubia
    18.           ports:
    19.             - containerPort: 8080

  3. Run the following command to create the Deployment:

    1. kubectl apply -f porter-bgp.yaml

Step 6: Create a Service

  1. Run the following command to create a YAML file for the Service:

    1. vi porter-bgp-svc.yaml

  2. Add the following information to the YAML file:

    1. kind: Service
    2. apiVersion: v1
    3. metadata:
    4.   name: porter-bgp-svc
    5.   annotations:
    6.     lb.kubesphere.io/v1alpha1: porter
    7.     protocol.porter.kubesphere.io/v1alpha1: bgp
    8.     eip.porter.kubesphere.io/v1alpha2: porter-bgp-eip
    9. spec:
    10.   selector:
    11.     app: porter-bgp
    12.   type: LoadBalancer
    13.   ports:
    14.     - name: http
    15.       port: 80
    16.       targetPort: 8080
    17.   externalTrafficPolicy: Cluster

    NOTE

    • You must set spec:type to LoadBalancer.
    • The lb.kubesphere.io/v1alpha1: porter annotation specifies that the Service uses PorterLB.
    • The protocol.porter.kubesphere.io/v1alpha1: bgp annotation specifies that PorterLB is used in BGP mode.
    • The eip.porter.kubesphere.io/v1alpha2: porter-bgp-eip annotation specifies the Eip object used by PorterLB. If this annotation is not configured, PorterLB automatically uses the first available Eip object that matches the protocol. You can also delete this annotation and add the spec:loadBalancerIP field (for example, spec:loadBalancerIP: 172.22.0.2) to assign a specific IP address to the Service.
    • In the BGP mode, you can set spec:loadBalancerIP of multiple Services to the same value for IP address sharing (the Services are distinguished by different Service ports). In this case, you must set spec:ports:port to different values and spec:externalTrafficPolicy to Cluster for the Services.
    • If spec:externalTrafficPolicy is set to Cluster (default value), PorterLB uses all Kubernetes cluster nodes as the next hops destined for the Service.
    • If spec:externalTrafficPolicy is set to Local, PorterLB uses only Kubernetes cluster nodes that contain Pods as the next hops destined for the Service.

  3. Run the following command to create the Service:

    1. kubectl apply -f porter-bgp-svc.yaml

Step 7: Verify PorterLB in BGP Mode

The following verifies whether PorterLB functions properly.

  1. In the Kubernetes cluster, run the following command to obtain the external IP address of the Service:

    1. kubectl get svc

    service-ip

  2. In the Kubernetes cluster, run the following command to obtain the IP addresses of the cluster nodes:

    node-ips

  3. On the BIRD machine, run the following command to check the routing table. If equivalent routes using the Kubernetes cluster nodes as next hops destined for the Service are displayed, PorterLB functions properly.

    1. ip route

    If spec:externalTrafficPolicy in the Service YAML configuration is set to Cluster, all Kubernetes cluster nodes are used as the next hops.

    bgp-routes-cluster

    If spec:externalTrafficPolicy in the Service YAML configuration is set to Local, only Kubernetes cluster nodes that contain Pods are used as the next hops.

    bgp-routes-local

  4. On the BIRD machine, run the following command to access the Service:

    1. curl 172.22.0.2

    access-service

Last modified March 31, 2021: Relocated files to adapt to localization and changed links. (6b5fcb1)