Using Multi-Node Clusters

Overview

  • This tutorial will show you how to start a multi-node clusters on minikube and deploy a service to it.

Prerequisites

  • minikube 1.10.1 or higher
  • kubectl

Caveat

Default host-path volume provisioner doesn’t support multi-node clusters (#12360). To be able to provision or claim volumes in multi-node clusters, you could use CSI Hostpath Driver addon.

Tutorial

  • Start a cluster with 2 nodes in the driver of your choice:
  1. minikube start --nodes 2 -p multinode-demo
  1. 😄  [multinode-demo] minikube v1.18.1 on Opensuse-Tumbleweed
  2. ✨  Automatically selected the docker driver
  3. 👍  Starting control plane node multinode-demo in cluster multinode-demo
  4. 🔥  Creating docker container (CPUs=2, Memory=8000MB) ...
  5. 🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.3 ...
  6.     ▪ Generating certificates and keys ...
  7.     ▪ Booting up control plane ...
  8.     ▪ Configuring RBAC rules ...
  9. 🔗  Configuring CNI (Container Networking Interface) ...
  10. 🔎  Verifying Kubernetes components...
  11.     ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
  12. 🌟  Enabled addons: storage-provisioner, default-storageclass
  14. 👍  Starting node multinode-demo-m02 in cluster multinode-demo
  15. 🔥  Creating docker container (CPUs=2, Memory=8000MB) ...
  16. 🌐  Found network options:
  17.     ▪ NO_PROXY=192.168.49.2
  18. 🐳  Preparing Kubernetes v1.20.2 on Docker 20.10.3 ...
  19.     ▪ env NO_PROXY=192.168.49.2
  20. 🔎  Verifying Kubernetes components...
  21. 🏄  Done! kubectl is now configured to use "multinode-demo" cluster and "default" namespace by default
  • Get the list of your nodes:
  1. kubectl get nodes
  1. NAME                 STATUS   ROLES                  AGE   VERSION
  2. multinode-demo       Ready    control-plane,master   99s   v1.20.2
  3. multinode-demo-m02   Ready    <none>                 73s   v1.20.2
  • You can also check the status of your nodes:
  1. minikube status -p multinode-demo
  1. multinode-demo
  2. type: Control Plane
  3. host: Running
  4. kubelet: Running
  5. apiserver: Running
  6. kubeconfig: Configured
  8. multinode-demo-m02
  9. type: Worker
  10. host: Running
  11. kubelet: Running
  • Deploy our hello world deployment:
  1. kubectl apply -f hello-deployment.yaml
  1. deployment.apps/hello created
  1. kubectl rollout status deployment/hello
  1. deployment "hello" successfully rolled out
  • Deploy our hello world service, which just spits back the IP address the request was served from:
  1. kubectl apply -f hello-svc.yaml
  1. service/hello created
  • Check out the IP addresses of our pods, to note for future reference
  1. kubectl get pods -o wide
  1. NAME                     READY   STATUS    RESTARTS   AGE   IP           NODE                 NOMINATED NODE   READINESS GATES
  2. hello-695c67cf9c-bzrzk   1/1     Running   0          22s   10.244.1.2   multinode-demo-m02   <none>           <none>
  3. hello-695c67cf9c-frcvw   1/1     Running   0          22s   10.244.0.3   multinode-demo       <none>           <none>
  • Look at our service, to know what URL to hit
  1. minikube service list -p multinode-demo
  1. |-------------|------------|--------------|---------------------------|
  2. |  NAMESPACE  |    NAME    | TARGET PORT  |            URL            |
  3. |-------------|------------|--------------|---------------------------|
  4. | default     | hello      |           80 | http://192.168.49.2:31000 |
  5. | default     | kubernetes | No node port |                           |
  6. | kube-system | kube-dns   | No node port |                           |
  7. |-------------|------------|--------------|---------------------------|
  • Let’s hit the URL a few times and see what comes back
  1. curl  http://192.168.49.2:31000
  1. Hello from hello-695c67cf9c-frcvw (10.244.0.3)
  3. curl  http://192.168.49.2:31000
  4. Hello from hello-695c67cf9c-bzrzk (10.244.1.2)
  6. curl  http://192.168.49.2:31000
  7. Hello from hello-695c67cf9c-bzrzk (10.244.1.2)
  9. curl  http://192.168.49.2:31000
  10. Hello from hello-695c67cf9c-frcvw (10.244.0.3)

  • Multiple nodes!

  • Referenced YAML files

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: hello
  5. spec:
  6.   replicas: 2
  7.   strategy:
  8.     type: RollingUpdate
  9.     rollingUpdate:
  10.       maxUnavailable: 100%
  11.   selector:
  12.     matchLabels:
  13.       app: hello
  14.   template:
  15.     metadata:
  16.       labels:
  17.         app: hello
  18.     spec:
  19.       affinity:
  20.         # ⬇⬇⬇ This ensures pods will land on separate hosts
  21.         podAntiAffinity:
  22.           requiredDuringSchedulingIgnoredDuringExecution:
  23.             - labelSelector:
  24.                 matchExpressions: [{ key: app, operator: In, values: [hello] }]
  25.               topologyKey: "kubernetes.io/hostname"
  26.       containers:
  27.         - name: hello-from
  28.           image: pbitty/hello-from:latest
  29.           ports:
  30.             - name: http
  31.               containerPort: 80
  32.       terminationGracePeriodSeconds: 1
  1. apiVersion: v1
  2. kind: Service
  3. metadata:
  4.   name: hello
  5. spec:
  6.   type: NodePort
  7.   selector:
  8.     app: hello
  9.   ports:
  10.     - protocol: TCP
  11.       nodePort: 31000
  12.       port: 80
  13.       targetPort: http

Last modified February 13, 2023: add multi-node cluster doc for the csi hostpath driver (874de40ca)