Setting up a High-availability RKE2 Kubernetes Cluster for Rancher

Tested on v2.5.6

This section describes how to install a Kubernetes cluster according to the best practices for the Rancher server environment.

Prerequisites

These instructions assume you have set up three nodes, a load balancer, and a DNS record, as described in this section.

Note that in order for RKE2 to work correctly with the load balancer, you need to set up two listeners: one for the supervisor on port 9345, and one for the Kubernetes API on port 6443.

Rancher needs to be installed on a supported Kubernetes version. To find out which versions of Kubernetes are supported for your Rancher version, refer to the support maintenance terms. To specify the RKE2 version, use the INSTALL_RKE2_VERSION environment variable when running the RKE2 installation script.

Installing Kubernetes

1. Install Kubernetes and Set up the RKE2 Server

RKE2 server runs with embedded etcd so you will not need to set up an external datastore to run in HA mode.

On the first node, you should set up the configuration file with your own pre-shared secret as the token. The token argument can be set on startup.

If you do not specify a pre-shared secret, RKE2 will generate one and place it at /var/lib/rancher/rke2/server/node-token.

To avoid certificate errors with the fixed registration address, you should launch the server with the tls-san parameter set. This option adds an additional hostname or IP as a Subject Alternative Name in the server’s TLS cert, and it can be specified as a list if you would like to access via both the IP and the hostname.

First, you must create the directory where the RKE2 config file is going to be placed:

  1. mkdir -p /etc/rancher/rke2/

Next, create the RKE2 config file at /etc/rancher/rke2/config.yaml using the following example:

  1. token: my-shared-secret
  2. tls-san:
  3.   - my-kubernetes-domain.com
  4.   - another-kubernetes-domain.com

After that, you need to run the install command and enable and start rke2:

  1. curl -sfL https://get.rke2.io | sh -
  2. systemctl enable rke2-server.service
  3. systemctl start rke2-server.service

  1. To join the rest of the nodes, you need to configure each additional node with the same shared token or the one generated automatically. Here is an example of the configuration file:

    1.     token: my-shared-secret
    2.     server: https://<DNS-DOMAIN>:9345
    3.     tls-san:
    4.       - my-kubernetes-domain.com
    5.       - another-kubernetes-domain.com

    After that, you need to run the installer and enable, then start, rke2:

    1.     curl -sfL https://get.rke2.io | sh -
    2.     systemctl enable rke2-server.service
    3.     systemctl start rke2-server.service

  2. Repeat the same command on your third RKE2 server node.

2. Confirm that RKE2 is Running

Once you’ve launched the rke2 server process on all server nodes, ensure that the cluster has come up properly with

  1. /var/lib/rancher/rke2/bin/kubectl \
  2.         --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes
  3. You should see your server nodes in the Ready state.

Then test the health of the cluster pods:

  1. /var/lib/rancher/rke2/bin/kubectl \
  2.         --kubeconfig /etc/rancher/rke2/rke2.yaml get pods --all-namespaces

Result: You have successfully set up a RKE2 Kubernetes cluster.

3. Save and Start Using the kubeconfig File

When you installed RKE2 on each Rancher server node, a kubeconfig file was created on the node at /etc/rancher/rke2/rke2.yaml. This file contains credentials for full access to the cluster, and you should save this file in a secure location.

To use this kubeconfig file,

  1. Install kubectl, a Kubernetes command-line tool.
  2. Copy the file at /etc/rancher/rke2/rke2.yaml and save it to the directory ~/.kube/config on your local machine.
  3. In the kubeconfig file, the server directive is defined as localhost. Configure the server as the DNS of your control-plane load balancer, on port 6443. (The RKE2 Kubernetes API Server uses port 6443, while the Rancher server will be served via the NGINX Ingress on ports 80 and 443.) Here is an example rke2.yaml:
  1. apiVersion: v1
  2. clusters:
  3. - cluster:
  4.     certificate-authority-data: [CERTIFICATE-DATA]
  5.     server: [LOAD-BALANCER-DNS]:6443 # Edit this line
  6.   name: default
  7. contexts:
  8. - context:
  9.     cluster: default
  10.     user: default
  11.   name: default
  12. current-context: default
  13. kind: Config
  14. preferences: {}
  15. users:
  16. - name: default
  17.   user:
  18.     password: [PASSWORD]
  19.     username: admin

Result: You can now use kubectl to manage your RKE2 cluster. If you have more than one kubeconfig file, you can specify which one you want to use by passing in the path to the file when using kubectl:

  1. kubectl --kubeconfig ~/.kube/config/rke2.yaml get pods --all-namespaces

For more information about the kubeconfig file, refer to the RKE2 documentation or the official Kubernetes documentation about organizing cluster access using kubeconfig files.

4. Check the Health of Your Cluster Pods

Now that you have set up the kubeconfig file, you can use kubectl to access the cluster from your local machine.

Check that all the required pods and containers are healthy are ready to continue:

  1. /var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get pods -A
  2. NAMESPACE     NAME                                                    READY   STATUS      RESTARTS   AGE
  3. kube-system   cloud-controller-manager-rke2-server-1                  1/1     Running     0          2m28s
  4. kube-system   cloud-controller-manager-rke2-server-2                  1/1     Running     0          61s
  5. kube-system   cloud-controller-manager-rke2-server-3                  1/1     Running     0          49s
  6. kube-system   etcd-rke2-server-1                                      1/1     Running     0          2m13s
  7. kube-system   etcd-rke2-server-2                                      1/1     Running     0          87s
  8. kube-system   etcd-rke2-server-3                                      1/1     Running     0          56s
  9. kube-system   helm-install-rke2-canal-hs6sx                           0/1     Completed   0          2m17s
  10. kube-system   helm-install-rke2-coredns-xmzm8                         0/1     Completed   0          2m17s
  11. kube-system   helm-install-rke2-ingress-nginx-flwnl                   0/1     Completed   0          2m17s
  12. kube-system   helm-install-rke2-metrics-server-7sggn                  0/1     Completed   0          2m17s
  13. kube-system   kube-apiserver-rke2-server-1                            1/1     Running     0          116s
  14. kube-system   kube-apiserver-rke2-server-2                            1/1     Running     0          66s
  15. kube-system   kube-apiserver-rke2-server-3                            1/1     Running     0          48s
  16. kube-system   kube-controller-manager-rke2-server-1                   1/1     Running     0          2m30s
  17. kube-system   kube-controller-manager-rke2-server-2                   1/1     Running     0          57s
  18. kube-system   kube-controller-manager-rke2-server-3                   1/1     Running     0          42s
  19. kube-system   kube-proxy-rke2-server-1                                1/1     Running     0          2m25s
  20. kube-system   kube-proxy-rke2-server-2                                1/1     Running     0          59s
  21. kube-system   kube-proxy-rke2-server-3                                1/1     Running     0          85s
  22. kube-system   kube-scheduler-rke2-server-1                            1/1     Running     0          2m30s
  23. kube-system   kube-scheduler-rke2-server-2                            1/1     Running     0          57s
  24. kube-system   kube-scheduler-rke2-server-3                            1/1     Running     0          42s
  25. kube-system   rke2-canal-b9lvm                                        2/2     Running     0          91s
  26. kube-system   rke2-canal-khwp2                                        2/2     Running     0          2m5s
  27. kube-system   rke2-canal-swfmq                                        2/2     Running     0          105s
  28. kube-system   rke2-coredns-rke2-coredns-547d5499cb-6tvwb              1/1     Running     0          92s
  29. kube-system   rke2-coredns-rke2-coredns-547d5499cb-rdttj              1/1     Running     0          2m8s
  30. kube-system   rke2-coredns-rke2-coredns-autoscaler-65c9bb465d-85sq5   1/1     Running     0          2m8s
  31. kube-system   rke2-ingress-nginx-controller-69qxc                     1/1     Running     0          52s
  32. kube-system   rke2-ingress-nginx-controller-7hprp                     1/1     Running     0          52s
  33. kube-system   rke2-ingress-nginx-controller-x658h                     1/1     Running     0          52s
  34. kube-system   rke2-metrics-server-6564db4569-vdfkn                    1/1     Running     0          66s

Result: You have confirmed that you can access the cluster with kubectl and the RKE2 cluster is running successfully. Now the Rancher management server can be installed on the cluster.