TLS on Kubernetes

You can obtain TLS certificates for the OpenFaaS API Gateway and for your functions using cert-manager from JetStack.

We will use the following components:

We will split this tutorial into two parts:

  • 1.0 TLS for the Gateway
  • 2.0 TLS and custom domains for your functions

1.0 TLS for the Gateway

This part guides you through setting up all the pre-requisite components to enable TLS for your gateway. You can then access your gateway via a URL such as https://gw.example.com and each function such as: https://gw.example.com/function/nodeinfo.

Configure Helm and Tiller

First install Helm and the Tiller following the instructions provided by Helm

Install OpenFaaS

Follow the instructions found in the OpenFaaS Helm Chart. As part of these instructions you will create a basic-auth password to secure the Gateway's API and UI.

Install nginx-ingress

This example will use a Kubernetes IngressController.

Add Nginx using the helm chart-ingress:

  1. $ helm install stable/nginx-ingress --name nginxingress --set rbac.create=true

The full configuration options for nginx can be found here.

You will see a service created in the default namespace with an EXTERNAL-IP of Pending, after a few moments it should reveal the public IP allocated by your cloud provider.

  1. $ kubectl get svc
  2. nginxingress-nginx-ingress-controller        LoadBalancer   192.168.137.172   134.209.179.1

Caveats:

  • If you do not have a cloud provider for your Kubernetes cluster, but have a public IP, then you can install Nginx in "host-mode" and use the IP of one or more of your nodes for the DNS record.
  1. $ helm install stable/nginx-ingress --name nginxingress --set rbac.create=true,controller.hostNetwork=true controller.daemonset.useHostPort=true,dnsPolicy=ClusterFirstWithHostNet,controller.kind=DaemonSet

Taken from tutorial: Setup a private Docker registry with TLS on Kubernetes

  • If you do not have a public IP for your Kubernetes cluster, then you can use a project like Inlets and bypass using cert-manager. Inlets has around half a dozen examples of configurations for Kubernetes.

HTTPS for your local endpoints with inlets and Caddy

Create a DNS record

Determine the public IP address which can be used to connect to Nginx:

  • If you are using a managed cloud provider, you will receive an IP address in EXTERNAL-IP
  • If you are using AWS EKS, Nginx will receive a DNS A record in EXTERNAL-IP
  • If you are using Host Mode for Nginx, then use the IP address of your node

For most people you can create a domain such as gw.example.com using a DNS A record, for those using AWS EKS, you will have to create a DNS CNAME entry instead.

The required steps will vary depending on your domain provider and your cluster provider. For example; on Google Cloud DNS or with Route53 using AWS.

Once created, verify that what you entered into your DNS control-panel worked with ping:

  1. ping gw.example.com

You should now see the value you entered. Sometimes DNS can take 1-5 minutes to propagate.

Install cert-manager

Following the recommended default installation for cert-manager, we install it into a new cert-manager namespace using the following commands:

  1. # Install the CustomResourceDefinition resources separately
  2. kubectl apply --validate=false -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.11/deploy/manifests/00-crds.yaml
  3.  
  4. # Create the namespace for cert-manager
  5. kubectl create namespace cert-manager
  6.  
  7. # Add the Jetstack Helm repository
  8. helm repo add jetstack https://charts.jetstack.io
  9.  
  10. # Update your local Helm chart repository cache
  11. helm repo update
  12.  
  13. # Install the cert-manager Helm chart
  14. helm install \
  15.   --name cert-manager \
  16.   --namespace cert-manager \
  17.   --version v0.11.0 \
  18.   jetstack/cert-manager

This configuration will work for most deployments, but you can also see https://cert-manager.readthedocs.io/en/latest/getting-started/install.html#steps for additional instructions and options for installing cert-manager.

Configure cert-manager

In additional to the controller installed in the previous step, we must also configure an "Issuer" before cert-manager can create certificates for our services. For convenience we will create an Issuer for both Let's Encrypt's production API and their staging API. The staging API has much higher rate limits. We will use it to issue a test certificate before switching over to a production certificate if everything works as expected.

Replace <your-email-here> with the contact email that will be shown with the TLS certificate.

  1. # letsencrypt-issuer.yaml
  2. apiVersion: cert-manager.io/v1alpha2
  3. kind: Issuer
  4. metadata:
  5.   name: letsencrypt-staging
  6.   namespace: openfaas
  7. spec:
  8.   acme:
  9.     # You must replace this email address with your own.
  10.     # Let's Encrypt will use this to contact you about expiring
  11.     # certificates, and issues related to your account.
  12.     email: <your-email-here>
  13.     server: https://acme-staging-v02.api.letsencrypt.org/directory
  14.     privateKeySecretRef:
  15.       # Secret resource used to store the account's private key.
  16.       name: example-issuer-account-key
  17.     # Add a single challenge solver, HTTP01 using nginx
  18.     solvers:
  19.     - http01:
  20.         ingress:
  21.           class: nginx
  22. ---
  23. apiVersion: cert-manager.io/v1alpha2
  24. kind: Issuer
  25. metadata:
  26.   name: letsencrypt-prod
  27.   namespace: openfaas
  28. spec:
  29.   acme:
  30.     # You must replace this email address with your own.
  31.     # Let's Encrypt will use this to contact you about expiring
  32.     # certificates, and issues related to your account.
  33.     email: <your-email-here>
  34.     server: https://acme-v02.api.letsencrypt.org/directory
  35.     privateKeySecretRef:
  36.       # Secret resource used to store the account's private key.
  37.       name: example-issuer-account-key
  38.     # Add a single challenge solver, HTTP01 using nginx
  39.     solvers:
  40.     - http01:
  41.         ingress:
  42.           class: nginx
  1. $ kubectl apply -f letsencrypt-issuer.yaml

This will allow cert-manager to automatically provision Certificates just in the openfaas namespace.

Add TLS to openfaas

The OpenFaaS Helm Chart already supports the nginx-ingress, but we want to customize it further. This is easiest with a custom values file. Below, we enable and configure the ingress object to use our certificate and expose just the gateway

  1. # tls.yaml
  2. ingress:
  3.   enabled: true
  4.   annotations:
  5.     kubernetes.io/ingress.class: "nginx"    
  6.     cert-manager.io/issuer: letsencrypt-staging
  7.   tls:
  8.     - hosts:
  9.         - gw.example.com
  10.       secretName: openfaas-crt
  11.   hosts:
  12.     - host: gw.example.com
  13.       serviceName: gateway
  14.       servicePort: 8080
  15.       path: /
  1. $ helm upgrade openfaas \
  2.     --namespace openfaas \
  3.     --reuse-values \
  4.     --values tls.yaml \
  5.     openfaas/openfaas

Check the certificate

A certificate will be created automatically through "Ingress Shim", part of cert-manager. The Ingress Shim reads annotations to decide which certificates to provision for us.

You can validate that certificate has been obtained successfully using:

  1. $ kubectl describe certificate \
  2.   -n openfaas \
  3.   openfaas-crt

Switch over to the production issuer

If it was successful, then you can change to the production Let's Encrypt issuer.

  • Replace letsencrypt-staging with letsencrypt-prod in tls.yaml
  • Run the helm command again
  1. $ helm upgrade openfaas \
  2.     --namespace openfaas \
  3.     --reuse-values \
  4.     --values tls.yaml \
  5.     openfaas/openfaas

Deploy and Invoke a function

In your projects containing OpenFaaS functions, you can now deploy using your domain as the gateway, replace gw.example.com with your domain as well as adding the username and password you created when you deployed OpenFaaS.

  1. faas-cli login --gateway https://gw.example.com --username <username> --password <password>
  2. faas-cli deploy --gateway https://gw.example.com

Verify and Debug

There are several commands we can use to verify that the required kubernetes objects have been created.

  • To check that the cert-manager issuers were created:
  1. $ kubectl -n openfaas get issuer letsencrypt-prod letsencrypt-staging
  • To check that your certificate was created and that cert-manager created the required secret with the actual TLS certificate:
  1. $ kubectl -n openfaas get certificate,secret openfaas-crt
  • If you want to tail the Nginx logs, you can use
  1. $ kubectl logs -f $(kubectl get po -l "app=nginxingress,component=controller" -o jsonpath="{.items[0].metadata.name}")

2.0 TLS and custom domains for functions

This part builds on part 1.0 and now enables custom domains for any of your functions. You will need to have installed OpenFaaS and an IngressController. For TLS, which is optional you need to have cert-manager and at least one Issuer.

For example, rather than accessing a function nodeinfo via https://gw.example.com/function/nodeinfo, you can now use a custom URL such as: https://nodeinfo.example.com.

The IngressOperator introduces a new CRD (Custom Resource Definition) called FunctionIngress. The role of FunctionIngress is to create an Ingress Kubernetes object to map a function to a domain-name, and optionally to also provision a TLS certificate using cert-manager.

Deploy the IngressOperator

  1. git clone https://github.com/openfaas-incubator/ingress-operator
  2. cd ingress-operator
  3.  
  4. kubectl apply -f ./artifacts/operator-crd.yaml
  5. kubectl apply -f ./artifacts/operator-rbac.yaml
  6. kubectl apply -f ./artifacts/operator-amd64.yaml

Check that the Operator started correctly:

  1. kubectl get deploy/ingress-operator -n openfaas -o wide

If it's working, you will see AVAILABLE showing 1. Otherwise use kubectl logs or kubectl get events for more information.

Deploy a function

Let's deploy a function from the store:

  1. faas-cli store deploy nodeinfo

Now create a DNS A record in your DNS manager pointing to your IngressController's public IP.

Check the public IP with kubectl get svc/nginxingress-nginx-ingress-controller, note down the EXTERNAL-IP.

  • nodeinfo.example.com pointing to the EXTERNAL-IP

Create a FunctionIngress Custom Resource (without TLS)

Now create a FunctionIngress custom resource:

  1. apiVersion: openfaas.com/v1alpha2
  2. kind: FunctionIngress
  3. metadata:
  4.   name: nodeinfo-tls
  5.   namespace: openfaas
  6. spec:
  7.   domain: "nodeinfo-tls.myfaas.club"
  8.   function: "nodeinfo"
  9.   ingressType: "nginx"

Verify that the Ingress record was created:

  1. kubectl get ingress -n openfaas

Ingress records are always created in the same namespace as the OpenFaaS Gateway.

Create a FunctionIngress with TLS certificate

To enable TLS, we just need to add the tls section and the following fields:

  • tls.enabled - whether to create the certificate
  • issuerRef.name - as per the Issuer name created above
  • issuerRef.kind - optional: either Issuer or ClusterIssuer

Note: The FunctionIngress currently makes use of the HTTP01 challenge.

  1. apiVersion: openfaas.com/v1alpha2
  2. kind: FunctionIngress
  3. metadata:
  4.   name: nodeinfo-tls
  5.   namespace: openfaas
  6. spec:
  7.   domain: "nodeinfo-tls.myfaas.club"
  8.   function: "nodeinfo"
  9.   ingressType: "nginx"
  10.   tls:
  11.     enabled: true
  12.     issuerRef:
  13.       name: "letsencrypt-staging"
  14.       kind: "Issuer"

Verify that the Certificate record was created:

  1. kubectl get cert -n openfaas

Use Zalando's skipper IngressController

Zalando's skipper IngressController is also supported. To switch over simply add the following to your YAML definition:

  1. spec:
  2.   ingressType: "skipper"

What about IngressController X?

Feel free to raise a feature request for your IngressController on the GitHub repo.