About advertising for the IP address pools

To provide fault-tolerant external IP addresses and load balancing for cluster services, configure MetalLB to advertise addresses by using Layer 2 protocols, the Border Gateway Protocol (BGP), or both. Selecting the appropriate protocol ensures reliable traffic routing and high availability for your application workloads.

MetalLB supports advertising by using Layer 2 and BGP for the same set of IP addresses.

MetalLB provides the flexibility to assign address pools to specific BGP peers, effectively limiting advertising to a subset of nodes on the network. This allows for more complex configurations, such as facilitating the isolation of nodes or the segmentation of the network.

About the BGPAdvertisement custom resource

To configure how the cluster announces IP addresses to external peers, define the properties of the BGPAdvertisement custom resource (CR). Specifying these parameters ensures that MetalLB correctly manages routing advertisements for your application services within the network.

The following table describes the parameters for the BGPAdvertisements CR:

Table 1. BGPAdvertisements configuration
Parameter	Type	Description
`metadata.name`	`string`	Specifies the name for the BGP advertisement.
`metadata.namespace`	`string`	Specifies the namespace for the BGP advertisement. Specify the same namespace that the MetalLB Operator uses.
`spec.aggregationLength`	`integer`	Optional: Specifies the number of bits to include in a 32-bit CIDR mask. To aggregate the routes that the speaker advertises to BGP peers, the mask is applied to the routes for several service IP addresses and the speaker advertises the aggregated route. For example, with an aggregation length of `24`, the speaker can aggregate several `10.0.1.x/32` service IP addresses and advertise a single `10.0.1.0/24` route.
`spec.aggregationLengthV6`	`integer`	Optional: Specifies the number of bits to include in a 128-bit CIDR mask. For example, with an aggregation length of `124`, the speaker can aggregate several `fc00:f853:0ccd:e799::x/128` service IP addresses and advertise a single `fc00:f853:0ccd:e799::0/124` route.
`spec.communities`	`string`	Optional: Specifies one or more BGP communities. Each community is specified as two 16-bit values separated by the colon character. Well-known communities must be specified as 16-bit values: `NO_EXPORT`: `65535:65281` `NO_ADVERTISE`: `65535:65282` `NO_EXPORT_SUBCONFED`: `65535:65283` Note You can also use community objects that are created along with the strings.
`spec.localPref`	`integer`	Optional: Specifies the local preference for this advertisement. This BGP attribute applies to BGP sessions within the Autonomous System.
`spec.ipAddressPools`	`string`	Optional: The list of `IPAddressPools` to advertise with this advertisement, selected by name.
`spec.ipAddressPoolSelectors`	`string`	Optional: A selector for the `IPAddressPools` that gets advertised with this advertisement. This is for associating the `IPAddressPool` to the advertisement based on the label assigned to the `IPAddressPool` instead of the name itself. If no `IPAddressPool` is selected by this or by the list, the advertisement is applied to all the `IPAddressPools`.
`spec.nodeSelectors`	`string`	Optional: By setting the `NodeSelectors` parameter, you can limit the nodes to announce as next hops for the load balancer IP. When empty, all the nodes are announced as next hops.
`spec.peers`	`string`	Optional: Use a list to specify the `metadata.name` values for each `BGPPeer` resource that receives advertisements for the MetalLB service IP address. The MetalLB service IP address is assigned from the IP address pool. By default, the MetalLB service IP address is advertised to all configured `BGPPeer` resources. Set this parameter to limit the advertisement to specific `BGPpeer` resources.

Configure MetalLB with a BGP advertisement and a basic use case

To advertise specific IPv4 and IPv6 routes to peer BGP routers for assigned load-balancer IP addresses, configure MetalLB by using default preference and community settings. Establishing these routes ensures reliable external traffic delivery and consistent network propagation for your application services.

Ensure that you can configure MetalLB so that the peer BGP routers receive one 203.0.113.200/32 route and one fc00:f853:ccd:e799::1/128 route for each load-balancer IP address that MetalLB assigns to a service. If you do not specify the localPref and communities parameters, MetalLB advertises the routes with localPref set to `0 and no BGP communities.

Advertising a basic address pool configuration with BGP

To ensure application services are reachable from external network peers, configure MetalLB to advertise an IPAddressPool by using the BGP advertisement. Establishing this advertisement allows the external network to correctly route traffic to the load balancer IP addresses of your cluster services.

The procedure demonstrates how to configure MetalLB to advertise the IPAddressPool by using BGP.

Prerequisites

Install the OpenShift CLI (oc).
Log in as a user with cluster-admin privileges.

Procedure

Create an IP address pool.

Create a file, such as ipaddresspool.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  namespace: metallb-system
  name: doc-example-bgp-basic
spec:
  addresses:
    - 203.0.113.200/30
    - fc00:f853:ccd:e799::/124
# ...

Apply the configuration for the IP address pool:
```
$ oc apply -f ipaddresspool.yaml
```

Create a BGP advertisement.

Create a file, such as bgpadvertisement.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: bgpadvertisement-basic
  namespace: metallb-system
spec:
  ipAddressPools:
  - doc-example-bgp-basic
# ...

Apply the configuration:
```
$ oc apply -f bgpadvertisement.yaml
```

Configuring MetalLB with a BGP advertisement and an advanced use case

To assign application services specific IP addresses from designated IPv4 and IPv6 ranges, configure MetalLB for advanced address allocation. Establishing these ranges and BGP advertisements ensures that your load-balancer services remain reachable through predictable network paths.

Configure MetalLB so that MetalLB assigns IP addresses to load-balancer services in the ranges between 203.0.113.200 and 203.0.113.203 and between fc00:f853:ccd:e799::0 and fc00:f853:ccd:e799::f.

To explain the two BGP advertisements, consider an instance when MetalLB assigns the IP address of 203.0.113.200 to a service. With that IP address as an example, the speaker advertises the following two routes to BGP peers:

203.0.113.200/32, with localPref set to 100 and the community set to the numeric value of the NO_ADVERTISE community. This specification indicates to the peer routers that they can use this route but they should not propagate information about this route to BGP peers.
203.0.113.200/30, aggregates the load-balancer IP addresses assigned by MetalLB into a single route. MetalLB advertises the aggregated route to BGP peers with the community attribute set to 8000:800. BGP peers propagate the 203.0.113.200/30 route to other BGP peers. When traffic is routed to a node with a speaker, the 203.0.113.200/32 route is used to forward the traffic into the cluster and to a pod that is associated with the service.

As you add more services and MetalLB assigns more load-balancer IP addresses from the pool, peer routers receive one local route, 203.0.113.20x/32, for each service, and the 203.0.113.200/30 aggregate route. Each service that you add generates the /30 route, but MetalLB deduplicates the routes to one BGP advertisement before communicating with peer routers.

Advertising an advanced address pool configuration with BGP

To ensure application services are reachable from external network peers through specific routing paths, configure MetalLB to advertise an advanced address pool by using the BGP. Establishing this advertisement allows your cluster to precisely communicate routing information to the external infrastructure.

The procedure demonstrates how to configure MetalLB to advertise an advanced address pool by using the BGP.

Prerequisites

Install the OpenShift CLI (oc).
Log in as a user with cluster-admin privileges.

Procedure

Create an IP address pool.

Create a file, such as ipaddresspool.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  namespace: metallb-system
  name: doc-example-bgp-adv
  labels:
    zone: east
spec:
  addresses:
    - 203.0.113.200/30
    - fc00:f853:ccd:e799::/124
  autoAssign: false
# ...

Apply the configuration for the IP address pool:
```
$ oc apply -f ipaddresspool.yaml
```

Create a BGP advertisement.

Create a file, such as bgpadvertisement1.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: bgpadvertisement-adv-1
  namespace: metallb-system
spec:
  ipAddressPools:
    - doc-example-bgp-adv
  communities:
    - 65535:65282
  aggregationLength: 32
  localPref: 100
# ...

Apply the configuration:
```
$ oc apply -f bgpadvertisement1.yaml
```

Create a file, such as bgpadvertisement2.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: bgpadvertisement-adv-2
  namespace: metallb-system
spec:
  ipAddressPools:
    - doc-example-bgp-adv
  communities:
    - 8000:800
  aggregationLength: 30
  aggregationLengthV6: 124
# ...

Apply the configuration:
```
$ oc apply -f bgpadvertisement2.yaml
```

Advertising an IP address pool from a subset of nodes

To restrict IP address advertisements to a specific set of nodes, such as a public-facing subnet, configure the nodeSelector parameter in the BGPAdvertisement custom resource (CR). When you configure these selectors, OpenShift Container Platform routes external traffic only through designated network interfaces for improved security and isolation.

Prerequisites

Install the OpenShift CLI (oc).
Log in as a user with cluster-admin privileges.

Procedure

Create an IP address pool by using a CR:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  namespace: metallb-system
  name: pool1
spec:
  addresses:
    - 4.4.4.100-4.4.4.200
    - 2001:100:4::200-2001:100:4::400
# ...

Control which cluster nodes advertise the IP address from pool1 by setting the .spec.nodeSelector value in the BGPAdvertisement CR. The following example advertises the IP address from pool1 only from NodeA and NodeB.

apiVersion: metallb.io/v1beta1
kind: BGPAdvertisement
metadata:
  name: example
spec:
  ipAddressPools:
  - pool1
  nodeSelector:
  - matchLabels:
      kubernetes.io/hostname: NodeA
  - matchLabels:
      kubernetes.io/hostname: NodeB
# ...

About the L2Advertisement custom resource

To configure how application services are announced over a Layer 2 network, define the properties in the L2Advertisement custom resource (CR). Establishing these parameters ensures that MetalLB correctly manages routing for your load-balancer IP addresses within the local network infrastructure

The following table details parameters for the l2Advertisements CR:

Table 2. L2 advertisements configuration
Parameter	Type	Description
`metadata.name`	`string`	Specifies the name for the L2 advertisement.
`metadata.namespace`	`string`	Specifies the namespace for the L2 advertisement. Specify the same namespace that the MetalLB Operator uses.
`spec.ipAddressPools`	`string`	Optional: The list of `IPAddressPools` to advertise with this advertisement, selected by name.
`spec.ipAddressPoolSelectors`	`string`	Optional: A selector for the `IPAddressPools` to advertise with this advertisement. This is for associating the `IPAddressPool` to the advertisement based on the label assigned to the `IPAddressPool` instead of the name itself. If no `IPAddressPool` is selected by this or by the list, the advertisement is applied to all the `IPAddressPools`.
`spec.nodeSelectors`	`string`	Optional: `NodeSelectors` limits the nodes to announce as next hops for the load balancer IP. If empty, MetalLB announces all nodes as next hops. Important Limiting the nodes to announce as next hops is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
`spec.interfaces`	`string`	Optional: The list of `interfaces` to announce the load balancer IP address.

Configuring MetalLB with an L2 advertisement

To provide fault-tolerant external IP addresses for cluster services, configure MetalLB to advertise an IPAddressPool by using the Layer 2 protocol. Establishing this advertisement ensures that application workloads remain reachable within the local network through standard address discovery protocols.

Prerequisites

Install the OpenShift CLI (oc).
Log in as a user with cluster-admin privileges.

Procedure

Create an IP address pool.

Create a file, such as ipaddresspool.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  namespace: metallb-system
  name: doc-example-l2
spec:
  addresses:
    - 4.4.4.0/24
  autoAssign: false
# ...

Apply the configuration for the IP address pool:
```
$ oc apply -f ipaddresspool.yaml
```

Create an L2 advertisement.

Create a file, such as l2advertisement.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2advertisement
  namespace: metallb-system
spec:
  ipAddressPools:
   - doc-example-l2
  # ...

Apply the configuration:
```
$ oc apply -f l2advertisement.yaml
```

Configuring MetalLB with an L2 advertisement and labels

To dynamically associate IP address pools with advertisements by using labels instead of specific names, configure the ipAddressPoolSelectors parameter in the MetalLB custom resource (CR). By using selectors, you can manage network routing more efficiently by automatically grouping address pools as your cluster scales.

The example in the procedure shows how to configure MetalLB so that the IPAddressPool is advertised with the L2 protocol by configuring the ipAddressPoolSelectors field.

Prerequisites

Install the OpenShift CLI (oc).
Log in as a user with cluster-admin privileges.

Procedure

Create an IP address pool.

Create a file, such as ipaddresspool.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  namespace: metallb-system
  name: doc-example-l2-label
  labels:
    zone: east
spec:
  addresses:
    - 172.31.249.87/32
# ...

Apply the configuration for the IP address pool:
```
$ oc apply -f ipaddresspool.yaml
```

Create an L2 advertisement that advertises the IP address by using ipAddressPoolSelectors.

Create a file, such as l2advertisement.yaml, with content like the following example:

apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2advertisement-label
  namespace: metallb-system
spec:
  ipAddressPoolSelectors:
    - matchExpressions:
        - key: zone
          operator: In
          values:
            - east
# ...

Apply the configuration:
```
$ oc apply -f l2advertisement.yaml
```

Configuring MetalLB with an L2 advertisement for selected interfaces

To restrict which network interfaces advertise assigned service IP addresses, configure the interfaces parameter in the MetalLB L2Advertisement custom resource (CR). Defining specific interfaces ensures that cluster services are reachable only through designated network paths for improved traffic management and isolation.

The example in the procedure shows how to configure MetalLB so that the IP address pool is advertised only from the network interfaces listed in the interfaces parameter of all nodes.

Prerequisites

You have installed the OpenShift CLI (oc).
You are logged in as a user with cluster-admin privileges.

Procedure

Create an IP address pool.
1. Create a file, such as ipaddresspool.yaml, and enter the configuration details as shown in the following example:
  apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: namespace: metallb-system name: doc-example-l2 spec: addresses: - 4.4.4.0/24 autoAssign: false # ...
2. Apply the configuration for the IP address pool as shown in the following example:
  $ oc apply -f ipaddresspool.yaml
Create an L2 advertisement with the interfaces selector to advertise the IP address.
1. Create a YAML file, such as l2advertisement.yaml, and enter the configuration details as shown the following example:
  apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: l2advertisement namespace: metallb-system spec: ipAddressPools: - doc-example-l2 interfaces: - interfaceA - interfaceB # ...
2. Apply the configuration for the advertisement as shown in the following example:
  $ oc apply -f l2advertisement.yaml
  Important
  
  The interface selector does not affect how MetalLB chooses the node to announce a given IP by using L2. The chosen node does not announce the service if the node does not have the selected interface.

Configuring MetalLB with secondary networks

To enable traffic forwarding for MetalLB on a secondary network interface, add a machine configuration that allows IP packet forwarding between interfaces. Implementing this configuration ensures that application services remain reachable when they use nondefault network paths.

Note

From OpenShift Container Platform 4.14 the default network behavior does not allow forwarding of IP packets between network interfaces.

OpenShift Container Platform clusters upgraded from 4.13 are not affected because a global parameter is set during an upgrade to enable global IP forwarding.

To enable IP forwarding for the secondary interface, you have two options:

Enable IP forwarding for a specific interface.
Enable IP forwarding for all interfaces.

Note

Enabling IP forwarding for a specific interface provides more granular control, while enabling it for all interfaces applies a global setting.

Procedure

Patch the Cluster Network Operator, setting the parameter routingViaHost to true by running the following command:

$ oc patch network.operator cluster -p '{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"gatewayConfig": {"routingViaHost": true} }}}}' --type=merge

Enable forwarding for a specific secondary interface, such as bridge-net, by creating and applying a MachineConfig CR:

Base64-encode the string that is used to configure network kernel parameters by running the following command on your local machine:
```
$ echo -e "net.ipv4.conf.bridge-net.forwarding = 1" | base64 -w0
```
Example output
```
bmV0LmlwdjQuY29uZi5icmlkZ2UtbmV0LmZvcndhcmRpbmcgPSAxCg==
```
Create the MachineConfig CR to enable IP forwarding for the specified secondary interface named bridge-net.

Save the following YAML in the enable-ip-forward.yaml file:

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: <node_role>
  name: 81-enable-global-forwarding
spec:
  config:
    ignition:
      version: 3.2.0
    storage:
      files:
      - contents:
          source: data:text/plain;charset=utf-8;base64,bmV0LmlwdjQuY29uZi5icmlkZ2UtbmV0LmZvcndhcmRpbmcgPSAxCg==
          verification: {}
        filesystem: root
        mode: 420
        path: /etc/sysctl.d/enable-global-forwarding.conf
  osImageURL: ""
# ...

where:

<node_role>: Node role where you want to enable IP forwarding, for example, worker.
contents.source: Populate with the generated Base64 string.

Apply the configuration by running the following command:
```
$ oc apply -f enable-ip-forward.yaml
```

Optional: Enable IP forwarding globally by running the following command:

$ oc patch network.operator cluster -p '{"spec":{"defaultNetwork":{"ovnKubernetesConfig":{"gatewayConfig":{"ipForwarding": "Global"}}}}}' --type=merge

Verification

After you apply the machine config, verify the changes by completing the following steps:
1. Enter into a debug session on the target node by running the following command. This command instantiates a debug pod called <node-name>-debug.
  $ oc debug node/<node-name>
2. Set /host as the root directory within the debug shell by running the following command. The debug pod mounts root file system of the host in /host within the pod. By changing the root directory to /host, you can run binaries contained in the executable paths of the host.
  $ chroot /host
3. Verify that IP forwarding is enabled by running the following command:
  $ cat /etc/sysctl.d/enable-global-forwarding.conf
  Example output
  net.ipv4.conf.bridge-net.forwarding = 1
  The output indicates that IPv4 forwarding is enabled on the bridge-net interface.

Additional resources

Configuring a community alias