Securing the Kubeflow authentication with HTTPS

How to secure the Kubeflow authentication with HTTPS using the network load balancer

This guide describes how to secure the Kubeflow authentication with HTTPS. You can enable HTTPS for Kubeflow dashboard (and other web UIs) using the network load balancer (NLB) feature of the IBM Cloud Kubernetes service—choose the classic worker nodes provider in the Setting environment variables section of the Create an IBM Cloud cluster guide.

Note: For details on NLB, go to the official Classic: About network load balancers guide.

Prerequisites

Setting up an NLB

To set up an NLB for your Kubernetes cluster, follow the official Classic: Setting up basic load balancing with an NLB 1.0 guide. Notice that the setup process for a multi-zone cluster differs from that of a single-zone cluster. For details, go to Setting up an NLB 1.0 in a multi-zone cluster.

  1. To use the existing Istio ingress gateway (instead of creating a new service), you need to update the service type of istio-ingressgateway to LoadBalancer from NodePort. Run the following command:

    1. kubectl patch svc istio-ingressgateway -n istio-system -p '{"spec":{"type":"LoadBalancer"}}'

  2. Verify that the NLB was created successfully. It might take a few minutes for the service to be created and an IP address to be made available. Run the command below and check if you can see the LoadBalancer Ingress IP address:

    1. kubectl describe service istio-ingressgateway -n istio-system | grep "LoadBalancer Ingress"

  3. Store the external IP of the istio-ingressgateway service in an environment variable:

    1. export INGRESS_GATEWAY_IP=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

Exposing the Kubeflow dashboard with DNS and TLS termination

The following instructions use the Kubeflow dashboard as an example. However, they apply to other web UI applications, since they all go through the Istio ingress gateway.

  1. Store the Kubernetes cluster name in an environment variable by running the following command:

    1. export CLUSTER_NAME=<cluster_name>

  2. Create a DNS domain and certificates for the IP of the service istio-ingressgateway in namespace istio-system:

    1. ibmcloud ks nlb-dns create classic --cluster $CLUSTER_NAME --ip $INGRESS_GATEWAY_IP --secret-namespace istio-system

  3. List the registered domain names:

    1. ibmcloud ks nlb-dns ls --cluster $CLUSTER_NAME

  4. Wait until the status of the certificate—the fourth field—of the new domain name becomes created. Then, save the value of the column SSL Cert Secret Name in environment variables by running these commands (replace {SECRET_NAME} with the secret’s name as shown in the SSL Cert Secret Name column):

    1. export INGRESS_GATEWAY_SECRET={SECRET_NAME}

    Note: If there is more than one entry in the output, choose the one that matches the IP address from LoadBalancer Ingress (step 2) of service istio-ingressgateway.

  5. Create a secret named istio-ingressgateway-certs for the istio-ingressgateway pods in namespace istio-system:

    1. kubectl get secret $INGRESS_GATEWAY_SECRET -o yaml > istio-ingressgateway-certs.yaml

  6. Update the istio-ingressgateway-certs.yaml file by changing the value of metadata.name to istio-ingressgateway-certs and the value of metadata.namespace to istio-system. Then, run the following commands:

    1. kubectl apply -f istio-ingressgateway-certs.yaml -n istio-system
    2. kubectl rollout restart deploy istio-ingressgateway -n istio-system
    3. rm istio-ingressgateway-certs.yaml

  7. Update the gateway kubeflow-gateway to expose port 443. Create a resource file kubeflow-gateway.yaml as follows by replacing <hostname> with the value of the column Hostname in step 4:

    1. apiVersion: networking.istio.io/v1alpha3
    2. kind: Gateway
    3. metadata:
    4. name: kubeflow-gateway
    5. namespace: kubeflow
    6. spec:
    7.   selector:
    8.     istio: ingressgateway
    9. servers:
    10. - hosts:
    11.   - '<hostname>'
    12.   port:
    13.     name: https
    14.     number: 443
    15.     protocol: HTTPS
    16.   tls:
    17.     mode: SIMPLE
    18.     privateKey: /etc/istio/ingressgateway-certs/tls.key
    19.     serverCertificate: /etc/istio/ingressgateway-certs/tls.crt

  8. Verify that the traffic is routed via HTTPS by using the value of above-mentioned Hostname in your browser. It should redirect traffic from an HTTP address to HTTPS address automatically.

Note: The certificates for the NLB DNS host secret expire every 90 days. The secret in the default namespace is automatically renewed by IBM Cloud Kubernetes Service 37 days before it expires. After this secret is updated, you must manually copy it to the istio-ingressgateway-certs secret by repeating commands in step 5 and 6.

Last modified 06.11.2020: Uses AppID for multi-user installation on IKS (#2327) (9a094895)