Configuring multi-network policy

As a cluster administrator, you can configure multi-network for additional networks. You can specify multi-network policy for SR-IOV, macvlan, and OVN-Kubernetes additional networks. Macvlan additional networks are fully supported. Other types of additional networks, such as ipvlan, are not supported.

Support for configuring multi-network policies for SR-IOV additional networks is a Technology Preview feature and is only supported with kernel network interface cards (NICs). SR-IOV is not supported for Data Plane Development Kit (DPDK) applications.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

Differences between multi-network policy and network policy

Although the MultiNetworkPolicy API implements the NetworkPolicy API, there are several important differences:

  • You must use the MultiNetworkPolicy API:

    1. apiVersion: k8s.cni.cncf.io/v1beta1
    2. kind: MultiNetworkPolicy

  • You must use the multi-networkpolicy resource name when using the CLI to interact with multi-network policies. For example, you can view a multi-network policy object with the oc get multi-networkpolicy <name> command where <name> is the name of a multi-network policy.

  • You must specify an annotation with the name of the network attachment definition that defines the macvlan or SR-IOV additional network:

    1. apiVersion: k8s.cni.cncf.io/v1beta1
    2. kind: MultiNetworkPolicy
    3. metadata:
    4.   annotations:
    5.     k8s.v1.cni.cncf.io/policy-for: <network_name>

    where:

    <network_name>

    Specifies the name of a network attachment definition.

Enabling multi-network policy for the cluster

As a cluster administrator, you can enable multi-network policy support on your cluster.

Prerequisites

  • Install the OpenShift CLI (oc).

  • Log in to the cluster with a user with cluster-admin privileges.

Procedure

  1. Create the multinetwork-enable-patch.yaml file with the following YAML:

    1. apiVersion: operator.openshift.io/v1
    2. kind: Network
    3. metadata:
    4.   name: cluster
    5. spec:
    6.   useMultiNetworkPolicy: true

  2. Configure the cluster to enable multi-network policy:

    1. $ oc patch network.operator.openshift.io cluster --type=merge --patch-file=multinetwork-enable-patch.yaml

    Example output

    1. network.operator.openshift.io/cluster patched

Supporting multi-network policies in IPv6 networks

The ICMPv6 Neighbor Discovery Protocol (NDP) is a set of messages and processes that enable devices to discover and maintain information about neighboring nodes. NDP plays a crucial role in IPv6 networks, facilitating the interaction between devices on the same link.

The Cluster Network Operator (CNO) deploys the iptables implementation of multi-network policy when the useMultiNetworkPolicy parameter is set to true.

To support multi-network policies in IPv6 networks the Cluster Network Operator deploys the following set of rules in every pod affected by a multi-network policy:

Multi-network policy custom rules

  1. kind: ConfigMap
  2. apiVersion: v1
  3. metadata:
  4.   name: multi-networkpolicy-custom-rules
  5.   namespace: openshift-multus
  6. data:
  8.   custom-v6-rules.txt: |
  9.     # accept NDP
  10.     -p icmpv6 --icmpv6-type neighbor-solicitation -j ACCEPT (1)
  11.     -p icmpv6 --icmpv6-type neighbor-advertisement -j ACCEPT (2)
  12.     # accept RA/RS
  13.     -p icmpv6 --icmpv6-type router-solicitation -j ACCEPT (3)
  14.     -p icmpv6 --icmpv6-type router-advertisement -j ACCEPT (4)
1This rule allows incoming ICMPv6 neighbor solicitation messages, which are part of the neighbor discovery protocol (NDP). These messages help determine the link-layer addresses of neighboring nodes.
2This rule allows incoming ICMPv6 neighbor advertisement messages, which are part of NDP and provide information about the link-layer address of the sender.
3This rule permits incoming ICMPv6 router solicitation messages. Hosts use these messages to request router configuration information.
4This rule allows incoming ICMPv6 router advertisement messages, which give configuration information to hosts.

You cannot edit these predefined rules.

These rules collectively enable essential ICMPv6 traffic for correct network functioning, including address resolution and router communication in an IPv6 environment. With these rules in place and a multi-network policy denying traffic, applications are not expected to experience connectivity issues.

Working with multi-network policy

As a cluster administrator, you can create, edit, view, and delete multi-network policies.

Prerequisites

  • You have enabled multi-network policy support for your cluster.

Creating a multi-network policy using the CLI

To define granular rules describing ingress or egress network traffic allowed for namespaces in your cluster, you can create a multi-network policy.

Prerequisites

  • Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin or the OpenShift SDN network plugin with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace that the multi-network policy applies to.

Procedure

  1. Create a policy rule:

    1. Create a <policy_name>.yaml file:

      1. $ touch <policy_name>.yaml

      where:

      <policy_name>

      Specifies the multi-network policy file name.

    2. Define a multi-network policy in the file that you just created, such as in the following examples:

      Deny ingress from all pods in all namespaces

      This is a fundamental policy, blocking all cross-pod networking other than cross-pod traffic allowed by the configuration of other Network Policies.

      1. apiVersion: k8s.cni.cncf.io/v1beta1
      2. kind: MultiNetworkPolicy
      3. metadata:
      4.   name: deny-by-default
      5.   annotations:
      6.     k8s.v1.cni.cncf.io/policy-for: <network_name>
      7. spec:
      8.   podSelector:
      9.   ingress: []

      where:

      <network_name>

      Specifies the name of a network attachment definition.

      Allow ingress from all pods in the same namespace

      1. apiVersion: k8s.cni.cncf.io/v1beta1
      2. kind: MultiNetworkPolicy
      3. metadata:
      4.   name: allow-same-namespace
      5.   annotations:
      6.     k8s.v1.cni.cncf.io/policy-for: <network_name>
      7. spec:
      8.   podSelector:
      9.   ingress:
      10.   - from:
      11.     - podSelector: {}

      where:

      <network_name>

      Specifies the name of a network attachment definition.

      Allow ingress traffic to one pod from a particular namespace

      This policy allows traffic to pods labelled pod-a from pods running in namespace-y.

      1. apiVersion: k8s.cni.cncf.io/v1beta1
      2. kind: MultiNetworkPolicy
      3. metadata:
      4.   name: allow-traffic-pod
      5.   annotations:
      6.     k8s.v1.cni.cncf.io/policy-for: <network_name>
      7. spec:
      8.   podSelector:
      9.    matchLabels:
      10.       pod: pod-a
      11.   policyTypes:
      12.   - Ingress
      13.   ingress:
      14.   - from:
      15.     - namespaceSelector:
      16.         matchLabels:
      17.            kubernetes.io/metadata.name: namespace-y

      where:

      <network_name>

      Specifies the name of a network attachment definition.

      Restrict traffic to a service

      This policy when applied ensures every pod with both labels app=bookstore and role=api can only be accessed by pods with label app=bookstore. In this example the application could be a REST API server, marked with labels app=bookstore and role=api.

      This example addresses the following use cases:

      • Restricting the traffic to a service to only the other microservices that need to use it.

      • Restricting the connections to a database to only permit the application using it.

        1. apiVersion: k8s.cni.cncf.io/v1beta1
        2. kind: MultiNetworkPolicy
        3. metadata:
        4.   name: api-allow
        5.   annotations:
        6.     k8s.v1.cni.cncf.io/policy-for: <network_name>
        7. spec:
        8.   podSelector:
        9.     matchLabels:
        10.       app: bookstore
        11.       role: api
        12.   ingress:
        13.   - from:
        14.       - podSelector:
        15.           matchLabels:
        16.             app: bookstore

        where:

        <network_name>

        Specifies the name of a network attachment definition.

  1. To create the multi-network policy object, enter the following command:

    1. $ oc apply -f <policy_name>.yaml -n <namespace>

    where:

    <policy_name>

    Specifies the multi-network policy file name.

    <namespace>

    Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

    Example output

    1. multinetworkpolicy.k8s.cni.cncf.io/deny-by-default created

If you log in to the web console with cluster-admin privileges, you have a choice of creating a network policy in any namespace in the cluster directly in YAML or from a form in the web console.

Editing a multi-network policy

You can edit a multi-network policy in a namespace.

Prerequisites

  • Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin or the OpenShift SDN network plugin with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace where the multi-network policy exists.

Procedure

  1. Optional: To list the multi-network policy objects in a namespace, enter the following command:

    1. $ oc get multi-networkpolicy

    where:

    <namespace>

    Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

  2. Edit the multi-network policy object.

    • If you saved the multi-network policy definition in a file, edit the file and make any necessary changes, and then enter the following command.

      1. $ oc apply -n <namespace> -f <policy_file>.yaml

      where:

      <namespace>

      Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

      <policy_file>

      Specifies the name of the file containing the network policy.

    • If you need to update the multi-network policy object directly, enter the following command:

      1. $ oc edit multi-networkpolicy <policy_name> -n <namespace>

      where:

      <policy_name>

      Specifies the name of the network policy.

      <namespace>

      Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

  3. Confirm that the multi-network policy object is updated.

    1. $ oc describe multi-networkpolicy <policy_name> -n <namespace>

    where:

    <policy_name>

    Specifies the name of the multi-network policy.

    <namespace>

    Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

If you log in to the web console with cluster-admin privileges, you have a choice of editing a network policy in any namespace in the cluster directly in YAML or from the policy in the web console through the Actions menu.

Viewing multi-network policies using the CLI

You can examine the multi-network policies in a namespace.

Prerequisites

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace where the multi-network policy exists.

Procedure

  • List multi-network policies in a namespace:

    • To view multi-network policy objects defined in a namespace, enter the following command:

      1. $ oc get multi-networkpolicy

    • Optional: To examine a specific multi-network policy, enter the following command:

      1. $ oc describe multi-networkpolicy <policy_name> -n <namespace>

      where:

      <policy_name>

      Specifies the name of the multi-network policy to inspect.

      <namespace>

      Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

If you log in to the web console with cluster-admin privileges, you have a choice of viewing a network policy in any namespace in the cluster directly in YAML or from a form in the web console.

Deleting a multi-network policy using the CLI

You can delete a multi-network policy in a namespace.

Prerequisites

  • Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin or the OpenShift SDN network plugin with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace where the multi-network policy exists.

Procedure

  • To delete a multi-network policy object, enter the following command:

    1. $ oc delete multi-networkpolicy <policy_name> -n <namespace>

    where:

    <policy_name>

    Specifies the name of the multi-network policy.

    <namespace>

    Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

    Example output

    1. multinetworkpolicy.k8s.cni.cncf.io/default-deny deleted

If you log in to the web console with cluster-admin privileges, you have a choice of deleting a network policy in any namespace in the cluster directly in YAML or from the policy in the web console through the Actions menu.

Creating a default deny all multi-network policy

This is a fundamental policy, blocking all cross-pod networking other than network traffic allowed by the configuration of other deployed network policies. This procedure enforces a default deny-by-default policy.

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Prerequisites

  • Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin or the OpenShift SDN network plugin with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace that the multi-network policy applies to.

Procedure

  1. Create the following YAML that defines a deny-by-default policy to deny ingress from all pods in all namespaces. Save the YAML in the deny-by-default.yaml file:

    1. apiVersion: k8s.cni.cncf.io/v1beta1
    2. kind: MultiNetworkPolicy
    3. metadata:
    4.   name: deny-by-default
    5.   namespace: default (1)
    6.   annotations:
    7.     k8s.v1.cni.cncf.io/policy-for: <network_name> (2)
    8. spec:
    9.   podSelector: {} (3)
    10.   ingress: [] (4)
    1namespace: default deploys this policy to the default namespace.
    2network_name: specifies the name of a network attachment definition.
    3podSelector: is empty, this means it matches all the pods. Therefore, the policy applies to all pods in the default namespace.
    4There are no ingress rules specified. This causes incoming traffic to be dropped to all pods.

  2. Apply the policy by entering the following command:

    1. $ oc apply -f deny-by-default.yaml

    Example output

    1. multinetworkpolicy.k8s.cni.cncf.io/deny-by-default created

Creating a multi-network policy to allow traffic from external clients

With the deny-by-default policy in place you can proceed to configure a policy that allows traffic from external clients to a pod with the label app=web.

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Follow this procedure to configure a policy that allows external service from the public Internet directly or by using a Load Balancer to access the pod. Traffic is only allowed to a pod with the label app=web.

Prerequisites

  • Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin or the OpenShift SDN network plugin with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace that the multi-network policy applies to.

Procedure

  1. Create a policy that allows traffic from the public Internet directly or by using a load balancer to access the pod. Save the YAML in the web-allow-external.yaml file:

    1. apiVersion: k8s.cni.cncf.io/v1beta1
    2. kind: MultiNetworkPolicy
    3. metadata:
    4.   name: web-allow-external
    5.   namespace: default
    6.   annotations:
    7.     k8s.v1.cni.cncf.io/policy-for: <network_name>
    8. spec:
    9.   policyTypes:
    10.   - Ingress
    11.   podSelector:
    12.     matchLabels:
    13.       app: web
    14.   ingress:
    15.     - {}

  2. Apply the policy by entering the following command:

    1. $ oc apply -f web-allow-external.yaml

    Example output

    1. multinetworkpolicy.k8s.cni.cncf.io/web-allow-external created

This policy allows traffic from all resources, including external traffic as illustrated in the following diagram:

Allow traffic from external clients

Creating a multi-network policy allowing traffic to an application from all namespaces

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Follow this procedure to configure a policy that allows traffic from all pods in all namespaces to a particular application.

Prerequisites

  • Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin or the OpenShift SDN network plugin with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace that the multi-network policy applies to.

Procedure

  1. Create a policy that allows traffic from all pods in all namespaces to a particular application. Save the YAML in the web-allow-all-namespaces.yaml file:

    1. apiVersion: k8s.cni.cncf.io/v1beta1
    2. kind: MultiNetworkPolicy
    3. metadata:
    4.   name: web-allow-all-namespaces
    5.   namespace: default
    6.   annotations:
    7.     k8s.v1.cni.cncf.io/policy-for: <network_name>
    8. spec:
    9.   podSelector:
    10.     matchLabels:
    11.       app: web (1)
    12.   policyTypes:
    13.   - Ingress
    14.   ingress:
    15.   - from:
    16.     - namespaceSelector: {} (2)
    1Applies the policy only to app:web pods in default namespace.
    2Selects all pods in all namespaces.

    By default, if you omit specifying a namespaceSelector it does not select any namespaces, which means the policy allows traffic only from the namespace the network policy is deployed to.

  2. Apply the policy by entering the following command:

    1. $ oc apply -f web-allow-all-namespaces.yaml

    Example output

    1. multinetworkpolicy.k8s.cni.cncf.io/web-allow-all-namespaces created

Verification

  1. Start a web service in the default namespace by entering the following command:

    1. $ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80

  2. Run the following command to deploy an alpine image in the secondary namespace and to start a shell:

    1. $ oc run test-$RANDOM --namespace=secondary --rm -i -t --image=alpine -- sh

  3. Run the following command in the shell and observe that the request is allowed:

    1. # wget -qO- --timeout=2 http://web.default

    Expected output

    1. <!DOCTYPE html>
    2. <html>
    3. <head>
    4. <title>Welcome to nginx!</title>
    5. <style>
    6. html { color-scheme: light dark; }
    7. body { width: 35em; margin: 0 auto;
    8. font-family: Tahoma, Verdana, Arial, sans-serif; }
    9. </style>
    10. </head>
    11. <body>
    12. <h1>Welcome to nginx!</h1>
    13. <p>If you see this page, the nginx web server is successfully installed and
    14. working. Further configuration is required.</p>
    16. <p>For online documentation and support please refer to
    17. <a href="http://nginx.org/">nginx.org</a>.<br/>
    18. Commercial support is available at
    19. <a href="http://nginx.com/">nginx.com</a>.</p>
    21. <p><em>Thank you for using nginx.</em></p>
    22. </body>
    23. </html>

Creating a multi-network policy allowing traffic to an application from a namespace

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Follow this procedure to configure a policy that allows traffic to a pod with the label app=web from a particular namespace. You might want to do this to:

  • Restrict traffic to a production database only to namespaces where production workloads are deployed.

  • Enable monitoring tools deployed to a particular namespace to scrape metrics from the current namespace.

Prerequisites

  • Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin or the OpenShift SDN network plugin with mode: NetworkPolicy set. This mode is the default for OpenShift SDN.

  • You installed the OpenShift CLI (oc).

  • You are logged in to the cluster with a user with cluster-admin privileges.

  • You are working in the namespace that the multi-network policy applies to.

Procedure

  1. Create a policy that allows traffic from all pods in a particular namespaces with a label purpose=production. Save the YAML in the web-allow-prod.yaml file:

    1. apiVersion: k8s.cni.cncf.io/v1beta1
    2. kind: MultiNetworkPolicy
    3. metadata:
    4.   name: web-allow-prod
    5.   namespace: default
    6.   annotations:
    7.     k8s.v1.cni.cncf.io/policy-for: <network_name>
    8. spec:
    9.   podSelector:
    10.     matchLabels:
    11.       app: web (1)
    12.   policyTypes:
    13.   - Ingress
    14.   ingress:
    15.   - from:
    16.     - namespaceSelector:
    17.         matchLabels:
    18.           purpose: production (2)
    1Applies the policy only to app:web pods in the default namespace.
    2Restricts traffic to only pods in namespaces that have the label purpose=production.

  2. Apply the policy by entering the following command:

    1. $ oc apply -f web-allow-prod.yaml

    Example output

    1. multinetworkpolicy.k8s.cni.cncf.io/web-allow-prod created

Verification

  1. Start a web service in the default namespace by entering the following command:

    1. $ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80

  2. Run the following command to create the prod namespace:

    1. $ oc create namespace prod

  3. Run the following command to label the prod namespace:

    1. $ oc label namespace/prod purpose=production

  4. Run the following command to create the dev namespace:

    1. $ oc create namespace dev

  5. Run the following command to label the dev namespace:

    1. $ oc label namespace/dev purpose=testing

  6. Run the following command to deploy an alpine image in the dev namespace and to start a shell:

    1. $ oc run test-$RANDOM --namespace=dev --rm -i -t --image=alpine -- sh

  7. Run the following command in the shell and observe that the request is blocked:

    1. # wget -qO- --timeout=2 http://web.default

    Expected output

    1. wget: download timed out

  8. Run the following command to deploy an alpine image in the prod namespace and start a shell:

    1. $ oc run test-$RANDOM --namespace=prod --rm -i -t --image=alpine -- sh

  9. Run the following command in the shell and observe that the request is allowed:

    1. # wget -qO- --timeout=2 http://web.default

    Expected output

    1. <!DOCTYPE html>
    2. <html>
    3. <head>
    4. <title>Welcome to nginx!</title>
    5. <style>
    6. html { color-scheme: light dark; }
    7. body { width: 35em; margin: 0 auto;
    8. font-family: Tahoma, Verdana, Arial, sans-serif; }
    9. </style>
    10. </head>
    11. <body>
    12. <h1>Welcome to nginx!</h1>
    13. <p>If you see this page, the nginx web server is successfully installed and
    14. working. Further configuration is required.</p>
    16. <p>For online documentation and support please refer to
    17. <a href="http://nginx.org/">nginx.org</a>.<br/>
    18. Commercial support is available at
    19. <a href="http://nginx.com/">nginx.com</a>.</p>
    21. <p><em>Thank you for using nginx.</em></p>
    22. </body>
    23. </html>

Additional resources