Configuring multi-network policy

As an administrator, you can use the MultiNetworkPolicy API to create multiple network policies that manage traffic for pods that are attached to secondary networks. For example, you can create policies that allow or deny traffic based on specific ports, IPs and ranges, or labels.

Multi-network policies can be used to manage traffic on secondary networks in the cluster. These policies cannot manage the default cluster network or primary network of user-defined networks.

As a cluster administrator, you can configure a multi-network policy for any of the following network types:

Single-Root I/O Virtualization (SR-IOV)
MAC Virtual Local Area Network (MacVLAN)
IP Virtual Local Area Network (IPVLAN)
Bond Container Network Interface (CNI) over SR-IOV
OVN-Kubernetes secondary networks

Note

Support for configuring multi-network policies for SR-IOV secondary networks is only supported with kernel network interface controllers (NICs). SR-IOV is not supported for Data Plane Development Kit (DPDK) applications.

Differences between multi-network policy and network policy

Although the MultiNetworkPolicy API implements the NetworkPolicy API, ensure that you understand the following key differences between the two policies:

You must use the MultiNetworkPolicy API, as demonstrated in the following example configuration:
```
apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
# ...
```
You must use the multi-networkpolicy resource name when using the CLI to interact with multi-network policies. For example, you can view a multi-network policy object with the oc get multi-networkpolicy <name> command where <name> is the name of a multi-network policy.
You can use the k8s.v1.cni.cncf.io/policy-for annotation on a MultiNetworkPolicy object to point to a NetworkAttachmentDefinition (NAD) custom resource (CR). The NAD CR defines the network to which the policy applies. The following example multi-network policy includes the k8s.v1.cni.cncf.io/policy-for annotation:
```
apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  annotations:
    k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name>
# ...
```
where:

<namespace_name>

Specifies the namespace name.

<network_name>

Specifies the name of a network attachment definition.

Enabling multi-network policy for the cluster

As a cluster administrator, you can enable multi-network policy support on your cluster.

Prerequisites

Install the OpenShift CLI (oc).
Log in to the cluster with a user with cluster-admin privileges.

Procedure

Create the multinetwork-enable-patch.yaml file with the following YAML:

apiVersion: operator.openshift.io/v1
kind: Network
metadata:
  name: cluster
spec:
  useMultiNetworkPolicy: true
# ...

Configure the cluster to enable multi-network policy. Successful output lists the name of the policy object and the patched status.
```
$ oc patch network.operator.openshift.io cluster --type=merge --patch-file=multinetwork-enable-patch.yaml
```

Supporting multi-network policies in IPv6 networks

The ICMPv6 Neighbor Discovery Protocol (NDP) is a set of messages and processes that enable devices to discover and maintain information about neighboring nodes. NDP plays a crucial role in IPv6 networks, facilitating the interaction between devices on the same link.

The Cluster Network Operator (CNO) deploys the iptables implementation of multi-network policy when the useMultiNetworkPolicy parameter is set to true.

To support multi-network policies in IPv6 networks the Cluster Network Operator deploys the following set of custom rules in every pod affected by a multi-network policy:

kind: ConfigMap
apiVersion: v1
metadata:
  name: multi-networkpolicy-custom-rules
  namespace: openshift-multus
data:

  custom-v6-rules.txt: |
    # accept NDP
    -p icmpv6 --icmpv6-type neighbor-solicitation -j ACCEPT 
    -p icmpv6 --icmpv6-type neighbor-advertisement -j ACCEPT 
    # accept RA/RS
    -p icmpv6 --icmpv6-type router-solicitation -j ACCEPT 
    -p icmpv6 --icmpv6-type router-advertisement -j ACCEPT

where:

icmpv6-type neighbor-solicitation: This rule allows incoming ICMPv6 neighbor solicitation messages, which are part of the neighbor discovery protocol (NDP). These messages help determine the link-layer addresses of neighboring nodes.
icmpv6-type neighbor-advertisement: This rule allows incoming ICMPv6 neighbor advertisement messages, which are part of NDP and provide information about the link-layer address of the sender.
icmpv6-type router-solicitation: This rule permits incoming ICMPv6 router solicitation messages. Hosts use these messages to request router configuration information.
icmpv6-type router-advertisement: This rule allows incoming ICMPv6 router advertisement messages, which give configuration information to hosts.

Note

You cannot edit the predefined rules.

The rules collectively enable essential ICMPv6 traffic for correct network functioning, including address resolution and router communication in an IPv6 environment. With these rules in place and a multi-network policy denying traffic, applications are not expected to experience connectivity issues.

Working with multi-network policy

To manage network traffic isolation and security for pods on secondary networks, you can create, edit, view, and delete multi-network policies. Before you work with multi-network policies, you must enable multi-network policy support for your cluster.

Creating a multi-network policy using the CLI

To define granular rules describing ingress or egress network traffic allowed for namespaces in your cluster, you can create a multi-network policy.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create a policy rule.
1. Create a <policy_name>.yaml file:
  $ touch <policy_name>.yaml
  where:
  
  <policy_name>
  
  Specifies the multi-network policy file name.
2. Define a multi-network policy in the created file. The following example denies ingress traffic from all pods in all namespaces. This is a fundamental policy, blocking all cross-pod networking other than cross-pod traffic allowed by the configuration of other Network Policies.
  apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy endif:: multi[] metadata: name: deny-by-default annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: {} policyTypes: - Ingress ingress: []
  where:
  <network_name>
  
  Specifies the name of a network attachment definition.
  
  The following example configuration allows ingress traffic from all pods in the same namespace:
  
  apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: allow-same-namespace annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: ingress: - from: - podSelector: {} # ...
  
  where:
  
  <network_name>
  
  Specifies the name of a network attachment definition.
  
  The following example allows ingress traffic to one pod from a particular namespace. This policy allows traffic to pods that have the pod-a label from pods running in namespace-y.
  
  apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: allow-traffic-pod annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: matchLabels: pod: pod-a policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: namespace-y # ...
  
  where:
  
  <network_name>
  
  Specifies the name of a network attachment definition.
  
  The following example configuration restricts traffic to a service. This policy when applied ensures every pod with both labels app=bookstore and role=api can only be accessed by pods with label app=bookstore. In this example the application could be a REST API server, marked with labels app=bookstore and role=api.
  
  This example configuration addresses the following use cases:
  
  Restricting the traffic to a service to only the other microservices that need to use it.
  
  Restricting the connections to a database to only permit the application using it.
  
  apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: api-allow annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: matchLabels: app: bookstore role: api ingress: - from: - podSelector: matchLabels: app: bookstore # ...
  
  where:
  
  <network_name>
  
  Specifies the name of a network attachment definition.
To create the multi-network policy object, enter the following command. Successful output lists the name of the policy object and the created status.
```
$ oc apply -f <policy_name>.yaml -n <namespace>
```
where:

<policy_name>

Specifies the multi-network policy file name.

<namespace>

Optional parameter. If you defined the object in a different namespace than the current namespace, the parameter specifices the namespace.

Successful output lists the name of the policy object and the created status.

Note

If you log in to the web console with cluster-admin privileges, you have a choice of creating a network policy in any namespace in the cluster directly in YAML or from a form in the web console.

Editing a multi-network policy

To modify existing policy configurations, you can edit a multi-network policy in a namespace. Edit policies by modifying the policy file and applying it with oc apply, or by using the oc edit command directly.

Note

If you log in with cluster-admin privileges, you can edit network policies in any namespace in the cluster. In the web console, you can edit policies directly in YAML or by using the Actions menu.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace where the multi-network policy exists.

Procedure

Optional: To list the multi-network policy objects in a namespace, enter the following command:
```
$ oc get multi-network policy -n <namespace>
```
where:

<namespace>

Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Edit the multi-network policy object.
1. If you saved the multi-network policy definition in a file, edit the file and make any necessary changes, and then enter the following command.
  $ oc apply -n <namespace> -f <policy_file>.yaml
  where:
  
  <namespace>
  
  Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
  
  <policy_file>
  
  Specifies the name of the file containing the network policy.
2. If you need to update the multi-network policy object directly, enter the following command:
  $ oc edit multi-network policy <policy_name> -n <namespace>
  where:
  
  <policy_name>
  
  Specifies the name of the network policy.
  
  <namespace>
  
  Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Confirm that the multi-network policy object is updated.
```
$ oc describe multi-networkpolicy <policy_name> -n <namespace>
```
where:

<policy_name>

Specifies the name of the multi-network policy.

<namespace>

Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

Viewing multi-network policies using the CLI

You can examine the multi-network policies in a namespace.

Note

If you log in with cluster-admin privileges, you can edit network policies in any namespace in the cluster. In the web console, you can edit policies directly in YAML or by using the Actions menu.

Prerequisites

You installed the OpenShift CLI (oc).
You are logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace where the multi-network policy exists.

Procedure

List multi-network policies in a namespace.
1. To view multi-network policy objects defined in a namespace enter the following command:
  $ oc get multi-networkpolicy
2. Optional: To examine a specific multi-network policy enter the following command:
  $ oc describe multi-networkpolicy <policy_name> -n <namespace>
  where:
  
  <policy_name>
  
  Specifies the name of the multi-network policy to inspect.
  
  <namespace>
  
  Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.

Deleting a multi-network policy using the CLI

You can delete a multi-network policy in a namespace.

Note

If you log in with cluster-admin privileges, you can delete network policies in any namespace in the cluster. In the web console, you can delete policies directly in YAML or by using the Actions menu.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace where the multi-network policy exists.

Procedure

To delete a multi-network policy object, enter the following command. Successful output lists the name of the policy object and the deleted status.
```
$ oc delete multi-networkpolicy <policy_name> -n <namespace>
```
where:

<policy_name>

Specifies the name of the multi-network policy.

<namespace>

Optional parameter. If you defined the object in a different namespace than the current namespace, the parameter specifices the namespace.

Creating a default deny all multi-network policy

The default deny all multi-network policy blocks all cross-pod networking other than network traffic allowed by the configuration of other deployed network policies and traffic between host-networked pods. This procedure enforces a strong deny policy by applying a deny-by-default policy in the my-project namespace.

Warning

Without configuring a NetworkPolicy custom resource (CR) that allows traffic communication, the following policy might cause communication problems across your cluster.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create the following YAML that defines a deny-by-default policy to deny ingress from all pods in all namespaces. Save the YAML in the deny-by-default.yaml file:
```
apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: deny-by-default
  namespace: my-project
  annotations:
    k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name>
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress: []
```
where:

namespace

Specifies the namespace in which to deploy the policy. For example, the my-project namespace.

annotations

Specifies the name of namespace project followed by the network attachment definition name.

podSelector

If this field is empty, the configuration matches all the pods. Therefore, the policy applies to all pods in the my-project namespace.

policyTypes

Specifies a list of rule types that the NetworkPolicy relates to.

- Ingress

Specifies Ingress only policyTypes.

ingress

Specifies ingress rules. If not specified, all incoming traffic is dropped to all pods.
Apply the policy by entering the following command. Successful output lists the name of the policy object and the created status.
```
$ oc apply -f deny-by-default.yaml
```

Creating a multi-network policy to allow traffic from external clients

With the deny-by-default policy in place you can proceed to configure a policy that allows traffic from external clients to a pod with the label app=web.

Note

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Follow this procedure to configure a policy that allows external service from the public Internet directly or by using a Load Balancer to access the pod. Traffic is only allowed to a pod with the label app=web.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create a policy that allows traffic from the public Internet directly or by using a load balancer to access the pod. Save the YAML in the web-allow-external.yaml file:

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: web-allow-external
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name>
spec:
  policyTypes:
  - Ingress
  podSelector:
    matchLabels:
      app: web
  ingress:
    - {}

Apply the policy by entering the following command. Successful output lists the name of the policy object and the created status.
```
$ oc apply -f web-allow-external.yaml
```
This policy allows traffic from all resources, including external traffic as illustrated in the following diagram:

Creating a multi-network policy allowing traffic to an application from all namespaces

You can configure a policy that allows traffic from all pods in all namespaces to a particular application.

Note

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Procedure

Create a policy that allows traffic from all pods in all namespaces to a particular application. Save the YAML in the web-allow-all-namespaces.yaml file:

+ where:

+

app

Applies the policy only to app:web pods in default namespace.

namespaceSelector

Selects all pods in all namespaces.

By default, if you do not specify a namespaceSelector parameter in the policy object, no namespaces get selected. This means the policy allows traffic only from the namespace where the network policy deployes.

Apply the policy by entering the following command. Successful output lists the name of the policy object and the created status.
```
$ oc apply -f web-allow-all-namespaces.yaml
```

Verification

Start a web service in the default namespace by entering the following command:

$ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80

Run the following command to deploy an alpine image in the secondary namespace and to start a shell:
```
$ oc run test-$RANDOM --namespace=secondary --rm -i -t --image=alpine -- sh
```

Run the following command in the shell and observe that the service allows the request:

# wget -qO- --timeout=2 http://web.default

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Creating a multi-network policy allowing traffic to an application from a namespace

You can configure a policy that allows traffic to a pod with the label app=web from a particular namespace. This configuration is useful in the following use cases:

Restrict traffic to a production database only to namespaces that have production workloads deployed.
Enable monitoring tools deployed to a particular namespace to scrape metrics from the current namespace.

Note

If you log in with a user with the cluster-admin role, then you can create a network policy in any namespace in the cluster.

Prerequisites

Your cluster uses a network plugin that supports NetworkPolicy objects, such as the OVN-Kubernetes network plugin, with mode: NetworkPolicy set.
You installed the OpenShift CLI (oc).
You logged in to the cluster with a user with cluster-admin privileges.
You are working in the namespace that the multi-network policy applies to.

Warning

Do not apply the network.openshift.io/policy-group: ingress label to custom namespace or projects. This label is Operator-managed and reserved for OpenShift Container Platform networking functions. It should not be altered on system-created namespaces.

Using this label can result in intermittent network connectivity drops, unintended application of system NetworkPolicies resource, or configuration drift as the operator attempts to reconcile the state. For custom traffic grouping, always use unique, user-defined labels as shown in the following procedure.

Procedure

Create a policy that allows traffic from all pods in a particular namespaces with a label purpose=production. Save the YAML in the web-allow-prod.yaml file:

apiVersion: k8s.cni.cncf.io/v1beta1
kind: MultiNetworkPolicy
metadata:
  name: web-allow-prod
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name>
spec:
  podSelector:
    matchLabels:
      app: web
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          purpose: production

where:

app: Applies the policy only to app:web pods in the default namespace.
purpose: Restricts traffic to only pods in namespaces that have the label purpose=production.

Apply the policy by entering the following command. Successful output lists the name of the policy object and the created status.
```
$ oc apply -f web-allow-prod.yaml
```

Verification

Start a web service in the default namespace by entering the following command:

$ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80

Run the following command to create the prod namespace:
```
$ oc create namespace prod
```
Run the following command to label the prod namespace:
```
$ oc label namespace/prod purpose=production
```
Run the following command to create the dev namespace:
```
$ oc create namespace dev
```
Run the following command to label the dev namespace:
```
$ oc label namespace/dev purpose=testing
```
Run the following command to deploy an alpine image in the dev namespace and to start a shell:
```
$ oc run test-$RANDOM --namespace=dev --rm -i -t --image=alpine -- sh
```
Run the following command in the shell and observe the reason for the blocked request. For example, expected output states wget: download timed out.
```
# wget -qO- --timeout=2 http://web.default
```
Run the following command to deploy an alpine image in the prod namespace and start a shell:
```
$ oc run test-$RANDOM --namespace=prod --rm -i -t --image=alpine -- sh
```

Run the following command in the shell and observe that the request is allowed:

# wget -qO- --timeout=2 http://web.default

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>