Assigning Unique External IPs for Ingress Traffic

Overview

One approach to getting external traffic into the cluster is by using ExternalIP or IngressIP addresses.

OKD supports two pools of IP addresses:

• IngressIP is used by the Loadbalancer when choosing an external IP address for the service.

• ExternalIP is used when the user selects a specific IP from the configured pool.

IngressIP and ExternalIP both allow external traffic access to the cluster, and, if routed correctly, external traffic can reach that service’s endpoints via any TCP/UDP port the service exposes. This can be simpler than having to manage the port space of a limited number of shared IP addresses when manually assigning external IPs to services. Also, these addresses can be used as virtual IPs (VIPs) when configuring high availability.

OKD supports both the automatic and manual assignment of IP addresses, and each address is guaranteed to be assigned to a maximum of one service. This ensures that each service can expose its chosen ports regardless of the ports exposed by other services.

Restrictions

To use an ExternalIP, you can:

• Select an IP address from the externalIPNetworkCIDRs range.

• Have an IP address assigned from the ingressIPNetworkCIDR pool in the master configuration file. In this case, OKD implements a non-cloud version of the load balancer service type and assigns IP addresses to the services.

  You must ensure that the IP address pool you assign terminates at one or more nodes in your cluster. You can use the existing oc adm ipfailover to ensure that the external IPs are highly available.

For manually-configured external IPs, potential port clashes are handled on a first-come, first-served basis. If you request a port, it is only available if it has not yet been assigned for that IP address. For example:

Port clash example for manually-configured external IPs

Two services have been manually configured with the same external IP address of 172.7.7.7.

MongoDB service A requests port 27017, and then MongoDB service B requests the same port; the first request gets the port.

However, port clashes are not an issue for external IPs assigned by the ingress controller, because the controller assigns each service a unique address.

Configuring the Cluster to Use Unique External IPs

In non-cloud clusters, ingressIPNetworkCIDR is set by default to 172.29.0.0/16. If your cluster environment is not already using this private range, you can use the default. However, if you want to use a different range, then you must set ingressIPNetworkCIDR in the /etc/origin/master/master-config.yaml file before you assign an ingress IP. Then, restart the master service.

Sample /etc/origin/master/master-config.yaml

networkConfig:
  ingressIPNetworkCIDR: 172.29.0.0/16

Configuring an Ingress IP for a Service

To assign an ingress IP:

1. Create a YAML file for a LoadBalancer service that requests a specific IP via the **loadBalancerIP** setting:

   Sample LoadBalancer Configuration

   apiVersion: v1
   kind: Service
   metadata:
     name: egress-1
   spec:
     ports:
     - name: db
       port: 3306
     loadBalancerIP: 172.29.0.1
     type: LoadBalancer
     selector:
       name: my-db-selector

2. Create a LoadBalancer service on your pod:

   $ oc create -f loadbalancer.yaml

3. Check the service for an external IP. For example, for a service named myservice:

   $ oc get svc myservice

   When your LoadBalancer-type service has an external IP assigned, the output displays the IP:

   NAME         CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
   myservice    172.30.74.106   172.29.0.1    3306/TCP    30s

Routing the Ingress CIDR for Development or Testing

Add a static route directing traffic for the ingress CIDR to a node in the cluster. For example:

# route add -net 172.29.0.0/16 gw 10.66.140.17 eth0

In the example above, 172.29.0.0/16 is the **ingressIPNetworkCIDR**, and 10.66.140.17 is the node IP.

Service externalIPs

In addition to the cluster’s internal IP addresses, the application developer can configure IP addresses that are external to the cluster. As the OKD administrator, you are responsible for ensuring that traffic arrives at a node with this IP.

The externalIPs must be selected by the administrator from the externalIPNetworkCIDRs range configured in the master-config.yaml file. When master-config.yaml changes, the master services must be restarted.

# master-restart api
# master-restart controllers

Sample externalIPNetworkCIDR /etc/origin/master/master-config.yaml

networkConfig:
  externalIPNetworkCIDR: 172.47.0.0/24

Service externalIPs Definition (JSON)

{
    "kind": "Service",
    "apiVersion": "v1",
    "metadata": {
        "name": "my-service"
    },
    "spec": {
        "selector": {
            "app": "MyApp"
        },
        "ports": [
            {
                "name": "http",
                "protocol": "TCP",
                "port": 80,
                "targetPort": 9376
            }
        ],
        "externalIPs" : [
            "80.11.12.10"         (1)
        ]
    }
}