Kubernetes policy, advanced tutorial

The Kubernetes NetworkPolicy API allows users to express ingress and egress policies (starting with Kubernetes 1.8.0) to Kubernetes pods based on labels and ports.

This guide walks through using Kubernetes NetworkPolicy to define more complex network policies.

Requirements

  • A working Kubernetes cluster and access to it using kubectl
  • Your Kubernetes nodes have connectivity to the public internet
  • You are familiar with Kubernetes NetworkPolicy

Tutorial flow

  1. Create the Namespace and Nginx Service
  2. Deny all ingress traffic
  3. Allow ingress traffic to Nginx
  4. Deny all egress traffic
  5. Allow egress traffic to kube-dns
  6. Cleanup Namespace

1. Create the namespace and nginx service

We’ll use a new namespace for this guide. Run the following commands to create it and a plain nginx service listening on port 80.

  1. kubectl create ns advanced-policy-demo
  2. kubectl create deployment --namespace=advanced-policy-demo nginx --image=nginx
  3. kubectl expose --namespace=advanced-policy-demo deployment nginx --port=80

Verify access - allowed all ingress and egress

Open up a second shell session which has kubectl connectivity to the Kubernetes cluster and create a busybox pod to test policy access. This pod will be used throughout this tutorial to test policy access.

  1. kubectl run --namespace=advanced-policy-demo access --rm -ti --image busybox /bin/sh

This should open up a shell session inside the access pod, as shown below.

  1. Waiting for pod advanced-policy-demo/access-472357175-y0m47 to be running, status is Pending, pod ready: false
  2. If you don't see a command prompt, try pressing enter.
  3. / #

Now from within the busybox “access” pod execute the following command to test access to the nginx service.

  1. wget -q --timeout=5 nginx -O -

It should return the HTML of the nginx welcome page.

Still within the busybox “access” pod, issue the following command to test access to google.com.

  1. wget -q --timeout=5 google.com -O -

It should return the HTML of the google.com home page.

2. Deny all ingress traffic

Enable ingress isolation on the namespace by deploying a default deny all ingress traffic policy.

  1. kubectl create -f - <<EOF
  2. apiVersion: networking.k8s.io/v1
  3. kind: NetworkPolicy
  4. metadata:
  5. name: default-deny-ingress
  6. namespace: advanced-policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels: {}
  10. policyTypes:
  11. - Ingress
  12. EOF

Verify access - denied all ingress and allowed all egress

Because all pods in the namespace are now selected, any ingress traffic which is not explicitly allowed by a policy will be denied.

We can see that this is the case by switching over to our “access” pod in the namespace and attempting to access the nginx service.

  1. wget -q --timeout=5 nginx -O -

It should return:

  1. wget: download timed out

Next, try to access google.com.

  1. wget -q --timeout=5 google.com -O -

It should return:

  1. <!doctype html><html itemscope="" item....

We can see that the ingress access to the nginx service is denied while egress access to outbound internet is still allowed.

3. Allow ingress traffic to Nginx

Run the following to create a NetworkPolicy which allows traffic to nginx pods from any pods in the advanced-policy-demo namespace.

  1. kubectl create -f - <<EOF
  2. apiVersion: networking.k8s.io/v1
  3. kind: NetworkPolicy
  4. metadata:
  5. name: access-nginx
  6. namespace: advanced-policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels:
  10. app: nginx
  11. ingress:
  12. - from:
  13. - podSelector:
  14. matchLabels: {}
  15. EOF

Verify access - allowed nginx ingress

Now ingress traffic to nginx will be allowed. We can see that this is the case by switching over to our “access” pod in the namespace and attempting to access the nginx service.

  1. wget -q --timeout=5 nginx -O -

It should return:

  1. <!DOCTYPE html>
  2. <html>
  3. <head>
  4. <title>Welcome to nginx!</title>...

After creating the policy, we can now access the nginx Service.

4. Deny all egress traffic

Enable egress isolation on the namespace by deploying a default deny all egress traffic policy.

  1. kubectl create -f - <<EOF
  2. apiVersion: networking.k8s.io/v1
  3. kind: NetworkPolicy
  4. metadata:
  5. name: default-deny-egress
  6. namespace: advanced-policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels: {}
  10. policyTypes:
  11. - Egress
  12. EOF

Verify access - denied all egress

Now any ingress or egress traffic which is not explicitly allowed by a policy will be denied.

We can see that this is the case by switching over to our “access” pod in the namespace and attempting to nslookup nginx or wget google.com.

  1. nslookup nginx

It should return something like the following.

  1. Server: 10.96.0.10
  2. Address 1: 10.96.0.10
  3. nslookup: can't resolve 'nginx'

Next, try to access google.com.

  1. wget -q --timeout=5 google.com -O -

It should return:

  1. wget: bad address 'google.com'

Kubernetes policy, advanced tutorial - 图1note

The nslookup command can take a minute or more to timeout.

5. Allow DNS egress traffic

Run the following to create a label of name: kube-system on the kube-system namespace and a NetworkPolicy which allows DNS egress traffic from any pods in the advanced-policy-demo namespace to the kube-system namespace.

  1. kubectl label namespace kube-system name=kube-system
  2. kubectl create -f - <<EOF
  3. apiVersion: networking.k8s.io/v1
  4. kind: NetworkPolicy
  5. metadata:
  6. name: allow-dns-access
  7. namespace: advanced-policy-demo
  8. spec:
  9. podSelector:
  10. matchLabels: {}
  11. policyTypes:
  12. - Egress
  13. egress:
  14. - to:
  15. - namespaceSelector:
  16. matchLabels:
  17. name: kube-system
  18. ports:
  19. - protocol: UDP
  20. port: 53
  21. EOF

Verify access - allowed DNS access

Now egress traffic to DNS will be allowed.

We can see that this is the case by switching over to our “access” pod in the namespace and attempting to lookup nginx and google.com.

  1. nslookup nginx

It should return something like the following.

  1. Server: 10.0.0.10
  2. Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Next, try to look up google.com.

  1. nslookup google.com

It should return something like the following.

  1. Name: google.com
  2. Address 1: 2607:f8b0:4005:807::200e sfo07s16-in-x0e.1e100.net
  3. Address 2: 216.58.195.78 sfo07s16-in-f14.1e100.net

Even though DNS egress traffic is now working, all other egress traffic from all pods in the advanced-policy-demo namespace is still blocked. Therefore the HTTP egress traffic from the wget calls will still fail.

6. Allow egress traffic to nginx

Run the following to create a NetworkPolicy which allows egress traffic from any pods in the advanced-policy-demo namespace to pods with labels matching app: nginx in the same namespace.

  1. kubectl create -f - <<EOF
  2. apiVersion: networking.k8s.io/v1
  3. kind: NetworkPolicy
  4. metadata:
  5. name: allow-egress-to-advance-policy-ns
  6. namespace: advanced-policy-demo
  7. spec:
  8. podSelector:
  9. matchLabels: {}
  10. policyTypes:
  11. - Egress
  12. egress:
  13. - to:
  14. - podSelector:
  15. matchLabels:
  16. app: nginx
  17. EOF

Verify access - allowed egress access to nginx

We can see that this is the case by switching over to our “access” pod in the namespace and attempting to access nginx.

  1. wget -q --timeout=5 nginx -O -

It should return the HTML of the nginx welcome page.

  1. <!DOCTYPE html>
  2. <html>
  3. <head>
  4. <title>Welcome to nginx!</title>...

Next, try to retrieve the home page of google.com.

  1. wget -q --timeout=5 google.com -O -

It should return:

  1. wget: download timed out

Access to google.com times out because it can resolve DNS but has no egress access to anything other than pods with labels matching app: nginx in the advanced-policy-demo namespace.

7. Clean up namespace

You can clean up after this tutorial by deleting the advanced policy demo namespace.

  1. kubectl delete ns advanced-policy-demo