Docker Swarm (mode) cluster

This section explains how to create a multi-host docker cluster with swarm mode using docker-machine and how to deploy Traefik on it.

The cluster consists of:

  • 3 servers
  • 1 manager
  • 2 workers
  • 1 overlay network (multi-host networking)

Prerequisites

Cluster provisioning

First, let's create all the required nodes. It's a shorter version of the swarm tutorial.

  1. docker-machine create -d virtualbox manager
  2. docker-machine create -d virtualbox worker1
  3. docker-machine create -d virtualbox worker2

Then, let's setup the cluster, in order:

  • initialize the cluster
  • get the token for other host to join
  • on both workers, join the cluster with the token 
  1. docker-machine ssh manager "docker swarm init \
  2.     --listen-addr $(docker-machine ip manager) \
  3.     --advertise-addr $(docker-machine ip manager)"
  5. export worker_token=$(docker-machine ssh manager "docker swarm \
  6. join-token worker -q")
  8. docker-machine ssh worker1 "docker swarm join \
  9.     --token=${worker_token} \
  10.     --listen-addr $(docker-machine ip worker1) \
  11.     --advertise-addr $(docker-machine ip worker1) \
  12.     $(docker-machine ip manager)"
  14. docker-machine ssh worker2 "docker swarm join \
  15.     --token=${worker_token} \
  16.     --listen-addr $(docker-machine ip worker2) \
  17.     --advertise-addr $(docker-machine ip worker2) \
  18.     $(docker-machine ip manager)"

Let's validate the cluster is up and running.

  1. docker-machine ssh manager docker node ls
  1. ID                           HOSTNAME  STATUS  AVAILABILITY  MANAGER STATUS
  2. 013v16l1sbuwjqcn7ucbu4jwt    worker1   Ready   Active
  3. 8buzkquycd17jqjber0mo2gn8    worker2   Ready   Active
  4. fnpj8ozfc85zvahx2r540xfcf *  manager   Ready   Active        Leader

Finally, let's create a network for Traefik to use.

  1. docker-machine ssh manager "docker network create --driver=overlay traefik-net"

Deploy Traefik

Let's deploy Traefik as a docker service in our cluster. The only requirement for Traefik to work with swarm mode is that it needs to run on a manager node - we are going to use a constraint for that.

  1. docker-machine ssh manager "docker service create \
  2.     --name traefik \
  3.     --constraint=node.role==manager \
  4.     --publish 80:80 --publish 8080:8080 \
  5.     --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock \
  6.     --network traefik-net \
  7.     traefik:<stable version from https://hub.docker.com/_/traefik> \
  8.     --docker \
  9.     --docker.swarmMode \
  10.     --docker.domain=traefik \
  11.     --docker.watch \
  12.     --api"

Let's explain this command:

Option Description
--publish 80:80 --publish 8080:8080 we publish port 80 and 8080 on the cluster.
--constraint=node.role==manager we ask docker to schedule Traefik on a manager node.
--mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock we bind mount the docker socket where Traefik is scheduled to be able to speak to the daemon.
--network traefik-net we attach the Traefik service (and thus the underlying container) to the traefik-net network.
--docker enable docker provider, and --docker.swarmMode to enable the swarm mode on Traefik.
--api activate the webUI on port 8080

Deploy your apps

We can now deploy our app on the cluster, here whoami, a simple web server in Go. We start 2 services, on the traefik-net network.

  1. docker-machine ssh manager "docker service create \
  2.     --name whoami0 \
  3.     --label traefik.port=80 \
  4.     --network traefik-net \
  5.     containous/whoami"
  7. docker-machine ssh manager "docker service create \
  8.     --name whoami1 \
  9.     --label traefik.port=80 \
  10.     --network traefik-net \
  11.     --label traefik.backend.loadbalancer.sticky=true \
  12.     containous/whoami"

Note

We set whoami1 to use sticky sessions (--label traefik.backend.loadbalancer.stickiness=true). We'll demonstrate that later.

Note

If using docker stack deploy, there is a specific way that the labels must be defined in the docker-compose file.

Check that everything is scheduled and started:

  1. docker-machine ssh manager "docker service ls"
  1. ID            NAME     MODE        REPLICAS  IMAGE                     PORTS
  2. moq3dq4xqv6t  traefik  replicated  1/1       traefik:latest            *:80->80/tcp,*:8080->8080/tcp
  3. ysil6oto1wim  whoami0  replicated  1/1       containous/whoami:latest
  4. z9re2mnl34k4  whoami1  replicated  1/1       containous/whoami:latest

Access to your apps through Traefik

  1. curl -H Host:whoami0.traefik http://$(docker-machine ip manager)
  1. Hostname: 5b0b3d148359
  2. IP: 127.0.0.1
  3. IP: 10.0.0.8
  4. IP: 10.0.0.4
  5. IP: 172.18.0.5
  6. GET / HTTP/1.1
  7. Host: whoami0.traefik
  8. User-Agent: curl/7.55.1
  9. Accept: */*
  10. Accept-Encoding: gzip
  11. X-Forwarded-For: 10.255.0.2
  12. X-Forwarded-Host: whoami0.traefik
  13. X-Forwarded-Proto: http
  14. X-Forwarded-Server: 77fc29c69fe4
  1. curl -H Host:whoami1.traefik http://$(docker-machine ip manager)
  1. Hostname: 3633163970f6
  2. IP: 127.0.0.1
  3. IP: 10.0.0.14
  4. IP: 10.0.0.6
  5. IP: 172.18.0.5
  6. GET / HTTP/1.1
  7. Host: whoami1.traefik
  8. User-Agent: curl/7.55.1
  9. Accept: */*
  10. Accept-Encoding: gzip
  11. X-Forwarded-For: 10.255.0.2
  12. X-Forwarded-Host: whoami1.traefik
  13. X-Forwarded-Proto: http
  14. X-Forwarded-Server: 77fc29c69fe4

Note

As Traefik is published, you can access it from any machine and not only the manager.

  1. curl -H Host:whoami0.traefik http://$(docker-machine ip worker1)
  1. Hostname: 5b0b3d148359
  2. IP: 127.0.0.1
  3. IP: 10.0.0.8
  4. IP: 10.0.0.4
  5. IP: 172.18.0.5
  6. GET / HTTP/1.1
  7. Host: whoami0.traefik
  8. User-Agent: curl/7.55.1
  9. Accept: */*
  10. Accept-Encoding: gzip
  11. X-Forwarded-For: 10.255.0.3
  12. X-Forwarded-Host: whoami0.traefik
  13. X-Forwarded-Proto: http
  14. X-Forwarded-Server: 77fc29c69fe4
  1. curl -H Host:whoami1.traefik http://$(docker-machine ip worker2)
  1. Hostname: 3633163970f6
  2. IP: 127.0.0.1
  3. IP: 10.0.0.14
  4. IP: 10.0.0.6
  5. IP: 172.18.0.5
  6. GET / HTTP/1.1
  7. Host: whoami1.traefik
  8. User-Agent: curl/7.55.1
  9. Accept: */*
  10. Accept-Encoding: gzip
  11. X-Forwarded-For: 10.255.0.4
  12. X-Forwarded-Host: whoami1.traefik
  13. X-Forwarded-Proto: http
  14. X-Forwarded-Server: 77fc29c69fe4

Scale both services

  1. docker-machine ssh manager "docker service scale whoami0=5"
  2. docker-machine ssh manager "docker service scale whoami1=5"

Check that we now have 5 replicas of each whoami service:

  1. docker-machine ssh manager "docker service ls"
  1. ID            NAME     MODE        REPLICAS  IMAGE                     PORTS
  2. moq3dq4xqv6t  traefik  replicated  1/1       traefik:latest            *:80->80/tcp,*:8080->8080/tcp
  3. ysil6oto1wim  whoami0  replicated  5/5       containous/whoami:latest
  4. z9re2mnl34k4  whoami1  replicated  5/5       containous/whoami:latest

Access to your whoami0 through Traefik multiple times.

Repeat the following command multiple times and note that the Hostname changes each time as Traefik load balances each request against the 5 tasks:

  1. curl -H Host:whoami0.traefik http://$(docker-machine ip manager)
  1. Hostname: f3138d15b567
  2. IP: 127.0.0.1
  3. IP: 10.0.0.5
  4. IP: 10.0.0.4
  5. IP: 172.18.0.3
  6. GET / HTTP/1.1
  7. Host: whoami0.traefik
  8. User-Agent: curl/7.55.1
  9. Accept: */*
  10. Accept-Encoding: gzip
  11. X-Forwarded-For: 10.255.0.2
  12. X-Forwarded-Host: whoami0.traefik
  13. X-Forwarded-Proto: http
  14. X-Forwarded-Server: 77fc29c69fe4

Do the same against whoami1:

  1. curl -c cookies.txt -H Host:whoami1.traefik http://$(docker-machine ip manager)
  1. Hostname: 348e2f7bf432
  2. IP: 127.0.0.1
  3. IP: 10.0.0.15
  4. IP: 10.0.0.6
  5. IP: 172.18.0.6
  6. GET / HTTP/1.1
  7. Host: whoami1.traefik
  8. User-Agent: curl/7.55.1
  9. Accept: */*
  10. Accept-Encoding: gzip
  11. X-Forwarded-For: 10.255.0.2
  12. X-Forwarded-Host: whoami1.traefik
  13. X-Forwarded-Proto: http
  14. X-Forwarded-Server: 77fc29c69fe4

Because the sticky sessions require cookies to work, we used the -c cookies.txt option to store the cookie into a file. The cookie contains the IP of the container to which the session sticks:

  1. cat ./cookies.txt
  1. # Netscape HTTP Cookie File
  2. # https://curl.haxx.se/docs/http-cookies.html
  3. # This file was generated by libcurl! Edit at your own risk.
  5. whoami1.traefik FALSE  /  FALSE  0  _TRAEFIK_BACKEND  http://10.0.0.15:80

If you load the cookies file (-b cookies.txt) for the next request, you will see that stickiness is maintained:

  1. curl -b cookies.txt -H Host:whoami1.traefik http://$(docker-machine ip manager)
  1. Hostname: 348e2f7bf432
  2. IP: 127.0.0.1
  3. IP: 10.0.0.15
  4. IP: 10.0.0.6
  5. IP: 172.18.0.6
  6. GET / HTTP/1.1
  7. Host: whoami1.traefik
  8. User-Agent: curl/7.55.1
  9. Accept: */*
  10. Accept-Encoding: gzip
  11. Cookie: _TRAEFIK_BACKEND=http://10.0.0.15:80
  12. X-Forwarded-For: 10.255.0.2
  13. X-Forwarded-Host: whoami1.traefik
  14. X-Forwarded-Proto: http
  15. X-Forwarded-Server: 77fc29c69fe4

GIF Magica