kube-proxy via ipvs

Kubernetes v1.8已经支持ipvs负载均衡模式（alpha版），只需要配置kube-proxy --proxy-mode=ipvs即可启用。

kube-proxy via ipvs - 图1

ipvs示例

NAT mode

  1. # prepare local kubernetes cluster
  2. $ sudo ./hack/local-up-cluster.sh
  3. $ sudo kill -9 $KUBE_PROXY_PID
  5. # run two nginx pods
  6. $ kubectl run --image nginx --replicas=2 nginx
  8. # expose deployment
  9. $ kubectl expose deployment nginx --port=80 --target-port=80
  11. $ kubectl get services
  12. NAME         CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
  13. kubernetes   10.0.0.1     <none>        443/TCP   3m
  14. nginx        10.0.0.185   <none>        80/TCP    4s
  16. $ kubectl get pods -o wide
  17. NAME                    READY     STATUS    RESTARTS   AGE       IP           NODE
  18. nginx-348975970-7x18g   1/1       Running   0          49s       172.17.0.3   127.0.0.1
  19. nginx-348975970-rtqrz   1/1       Running   0          49s       172.17.0.4   127.0.0.1
  21. # Add dummy link
  22. $ sudo ip link add type dummy
  23. $ sudo ip addr add 10.0.0.185 dev dummy0
  25. # Add ipvs rules; real server should use nat mode, since host is essentially
  26. # the gateway.
  27. $ sudo ipvsadm -A -t 10.0.0.185:80
  28. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.3:80 -m
  29. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.4:80 -m
  30. $ sudo ipvsadm -Ln
  31. IP Virtual Server version 1.2.1 (size=4096)
  32. Prot LocalAddress:Port Scheduler Flags
  33.   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
  34. TCP  10.0.0.185:80 wlc
  35.   -> 172.17.0.3:80                Masq    1      0          1
  36.   -> 172.17.0.4:80                Masq    1      0          1
  38. # Works in container
  39. $ docker run -ti busybox wget -qO- 10.0.0.185:80
  40. <!DOCTYPE html>
  41. // truncated
  43. # Works in host
  44. $ curl 10.0.0.185:80
  45. <!DOCTYPE html>
  46. // truncated

DR mode

  1. # continue above setup;
  2. $ PID=$(docker inspect -f '{{.State.Pid}}' k8s_nginx_nginx-348975970-rtqrz_default_b1661284-2eeb-11e7-924d-8825937fa049_0)
  3. $ sudo mkdir -p /var/run/netns
  4. $ sudo ln -s /proc/$PID/ns/net /var/run/netns/$PID
  5. $ sudo ip link add type dummy
  6. $ sudo ip link set dummy1 netns $PID
  7. $ sudo ip netns exec $PID ip addr add 10.0.0.185 dev dummy1
  8. $ sudo ip netns exec $PID ip link set dummy1 up
  9. # same for the other pod
  10. $ sudo ipvsadm -D -t 10.0.0.185:80
  11. $ sudo ipvsadm -A -t 10.0.0.185:80
  12. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.3:80 -g
  13. $ sudo ipvsadm -a -t 10.0.0.185:80 -r 172.17.0.4:80 -g    
  14. $ docker run -ti busybox wget -qO- 10.0.0.185:80
  15. <!DOCTYPE html>
  16. // truncated
  18. // ignored seting arp_ignore/arp_announce