Vagrant

Assuming you have Vagrant 2.0+ installed with virtualbox, libvirt/qemu or vmware, but is untested) you should be able to launch a 3 node Kubernetes cluster by simply running vagrant up. This will spin up 3 VMs and install kubernetes on them. Once they are completed you can connect to any of them by running vagrant ssh k8s-[1..3].

To give an estimate of the expected duration of a provisioning run: On a dual core i5-6300u laptop with an SSD, provisioning takes around 13 to 15 minutes, once the container images and other files are cached. Note that libvirt/qemu is recommended over virtualbox as it is quite a bit faster, especially during boot-up time.

For proper performance a minimum of 12GB RAM is recommended. It is possible to run a 3 node cluster on a laptop with 8GB of RAM using the default Vagrantfile, provided you have 8GB zram swap configured and not much more than a browser and a mail client running. If you decide to run on such a machine, then also make sure that any tmpfs devices, that are mounted, are mostly empty and disable any swapfiles mounted on HDD/SSD or you will be in for some serious swap-madness. Things can get a bit sluggish during provisioning, but when that’s done, the system will actually be able to perform quite well.

Customize Vagrant

You can override the default settings in the Vagrantfile either by directly modifying the Vagrantfile or through an override file. In the same directory as the Vagrantfile, create a folder called vagrant and create config.rb file in it. An example of how to configure this file is given below.

Use alternative OS for Vagrant

By default, Vagrant uses Ubuntu 18.04 box to provision a local cluster. You may use an alternative supported operating system for your local cluster.

Customize $os variable in Vagrantfile or as override, e.g.,:

  1. echo '$os = "flatcar-stable"' >> vagrant/config.rb

The supported operating systems for vagrant are defined in the SUPPORTED_OS constant in the Vagrantfile.

File and image caching

Kubespray can take quite a while to start on a laptop. To improve provisioning speed, the variable ‘download_run_once’ is set. This will make kubespray download all files and containers just once and then redistributes them to the other nodes and as a bonus, also cache all downloads locally and re-use them on the next provisioning run. For more information on download settings see download documentation.

Example use of Vagrant

The following is an example of setting up and running kubespray using vagrant. For repeated runs, you could save the script to a file in the root of the kubespray and run it by executing ‘source .

  1. # use virtualenv to install all python requirements
  2. VENVDIR=venv
  3. virtualenv --python=/usr/bin/python3.7 $VENVDIR
  4. source $VENVDIR/bin/activate
  5. pip install -r requirements.txt
  7. # prepare an inventory to test with
  8. INV=inventory/my_lab
  9. rm -rf ${INV}.bak &> /dev/null
  10. mv ${INV} ${INV}.bak &> /dev/null
  11. cp -a inventory/sample ${INV}
  12. rm -f ${INV}/hosts.ini
  14. # customize the vagrant environment
  15. mkdir vagrant
  16. cat << EOF > vagrant/config.rb
  17. \$instance_name_prefix = "kub"
  18. \$vm_cpus = 1
  19. \$num_instances = 3
  20. \$os = "centos-bento"
  21. \$subnet = "10.0.20"
  22. \$network_plugin = "flannel"
  23. \$inventory = "$INV"
  24. \$shared_folders = { 'temp/docker_rpms' => "/var/cache/yum/x86_64/7/docker-ce/packages" }
  25. EOF
  27. # make the rpm cache
  28. mkdir -p temp/docker_rpms
  30. vagrant up
  32. # make a copy of the downloaded docker rpm, to speed up the next provisioning run
  33. scp kub-1:/var/cache/yum/x86_64/7/docker-ce/packages/* temp/docker_rpms/
  35. # copy kubectl access configuration in place
  36. mkdir $HOME/.kube/ &> /dev/null
  37. ln -s $INV/artifacts/admin.conf $HOME/.kube/config
  38. # make the kubectl binary available
  39. sudo ln -s $INV/artifacts/kubectl /usr/local/bin/kubectl
  40. #or
  41. export PATH=$PATH:$INV/artifacts

If a vagrant run failed and you’ve made some changes to fix the issue causing the fail, here is how you would re-run ansible:

  1. ansible-playbook -vvv -i .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory cluster.yml

If all went well, you check if it’s all working as expected:

  1. kubectl get nodes

The output should look like this:

  1. $ kubectl get nodes
  2. NAME    STATUS   ROLES    AGE   VERSION
  3. kub-1   Ready    master   32m   v1.14.1
  4. kub-2   Ready    master   31m   v1.14.1
  5. kub-3   Ready    <none>   31m   v1.14.1

Another nice test is the following:

  1. kubectl get po --all-namespaces -o wide

Which should yield something like the following:

  1. NAMESPACE     NAME                                    READY   STATUS    RESTARTS   AGE   IP            NODE     NOMINATED NODE   READINESS GATES
  2. kube-system   coredns-97c4b444f-9wm86                 1/1     Running   0          31m   10.233.66.2   kub-3    <none>           <none>
  3. kube-system   coredns-97c4b444f-g7hqx                 0/1     Pending   0          30m   <none>        <none>   <none>           <none>
  4. kube-system   dns-autoscaler-5fc5fdbf6-5c48k          1/1     Running   0          31m   10.233.66.3   kub-3    <none>           <none>
  5. kube-system   kube-apiserver-kub-1                    1/1     Running   0          32m   10.0.20.101   kub-1    <none>           <none>
  6. kube-system   kube-apiserver-kub-2                    1/1     Running   0          32m   10.0.20.102   kub-2    <none>           <none>
  7. kube-system   kube-controller-manager-kub-1           1/1     Running   0          32m   10.0.20.101   kub-1    <none>           <none>
  8. kube-system   kube-controller-manager-kub-2           1/1     Running   0          32m   10.0.20.102   kub-2    <none>           <none>
  9. kube-system   kube-flannel-8tgcn                      2/2     Running   0          31m   10.0.20.103   kub-3    <none>           <none>
  10. kube-system   kube-flannel-b2hgt                      2/2     Running   0          31m   10.0.20.101   kub-1    <none>           <none>
  11. kube-system   kube-flannel-zx4bc                      2/2     Running   0          31m   10.0.20.102   kub-2    <none>           <none>
  12. kube-system   kube-proxy-4bjdn                        1/1     Running   0          31m   10.0.20.102   kub-2    <none>           <none>
  13. kube-system   kube-proxy-l5tt5                        1/1     Running   0          31m   10.0.20.103   kub-3    <none>           <none>
  14. kube-system   kube-proxy-x59q8                        1/1     Running   0          31m   10.0.20.101   kub-1    <none>           <none>
  15. kube-system   kube-scheduler-kub-1                    1/1     Running   0          32m   10.0.20.101   kub-1    <none>           <none>
  16. kube-system   kube-scheduler-kub-2                    1/1     Running   0          32m   10.0.20.102   kub-2    <none>           <none>
  17. kube-system   kubernetes-dashboard-6c7466966c-jqz42   1/1     Running   0          31m   10.233.66.4   kub-3    <none>           <none>
  18. kube-system   nginx-proxy-kub-3                       1/1     Running   0          32m   10.0.20.103   kub-3    <none>           <none>
  19. kube-system   nodelocaldns-2x7vh                      1/1     Running   0          31m   10.0.20.102   kub-2    <none>           <none>
  20. kube-system   nodelocaldns-fpvnz                      1/1     Running   0          31m   10.0.20.103   kub-3    <none>           <none>
  21. kube-system   nodelocaldns-h2f42                      1/1     Running   0          31m   10.0.20.101   kub-1    <none>           <none>

Create clusteradmin rbac and get the login token for the dashboard:

  1. kubectl create -f contrib/misc/clusteradmin-rbac.yml
  2. kubectl -n kube-system describe secret kubernetes-dashboard-token | grep 'token:' | grep -o '[^ ]\+$'

Copy it to the clipboard and now log in to the dashboard.