Understanding Replication Modes (BUM) in NSX-T

Lets go ahead and discuss the replication modes in NSX-T. In my last post, we have discussed how to create the logical switches and connect the vSphere and KVM virtual machines to the logical Switches. One of the pre-requisites is to select the replication mode while configuring the Logical Switch. NSX-T supports two different replication modes and we will see how each of these replication modes works in this blog post.

Why do we need Replication Modes?

Each host transport node is a tunnel endpoint and has an IP address. These IP addresses can be in the same IP subnet or in different IP subnets, depending on your configuration of IP pools
or DHCP for your transport nodes. When two VMs on different hosts communicate directly, unicast-encapsulated traffic is exchanged between the two tunnel endpoint IP addresses associated with the two hypervisors without any need for flooding.

However, as with any Layer 2 network, sometimes traffic that is originated by a VM needs to be flooded, meaning that it needs to be sent to all of the other VMs belonging to the same logical switch. This is the case with Layer 2 broadcast, unknown unicast, and multicast traffic (BUM traffic). BUM traffic originated by a VM on a given hypervisor needs to be replicated to remote hypervisors that host other VMs that are connected to the same logical switch.

To enable this flooding, NSX-T supports two different replication modes.

Types of Replication Modes:

Hierarchical two-tier (sometimes called MTEP)
Head (sometimes called source)

Replication Modes

Hierarchical two-tier (MTEP) Replication Mode

Lets take an example to understand Hierarchical two-tier replication mode. Say you have Hosts in multiple IP Subnets, TN123 are in One IP Subnet, TN456 and TN789 are in different IP Subnets. All these transport nodes have VMs connected to the VNI 56207 excep the host TN6. A Tunnel endpoint table shows the host-VNI connection. TN1 examines the tunnel endpoint table for VNI 56207 and determines the tunnel endpoint IP addresses for other hosts on VNI 56207

The VNI 56207 is configured across multiple IP segments. For the hosts part of the same IP subnets, TN1 creates a copy of every BUM frame and sends the copy directly to each host. For the Other tunnel end points part of the different IP subnet, TN1 nominates one of the host as replicator. The replicator nodes TN5 and TN8 receives a copy of each BUM frame for VNI 56207. This copy will flagged as Replicate locally in the encapsulation header. It becomes the responsibility of the replicator to create a copy of the BUM frame for each host it knows about that is on VNI 56207 and in the same IP Segment as the replicator host.

Hierarchical Two Tier (MTEP) Replication Mode

Head (Source) Replication Mode:

WIth the head replication also known as headend replication, there are no replicators. TN1 simply creates a copy of each BUM frame for each tunnel endpoint it knows about on VNI 56207 and sends it. The TN6 will not receive the BUM frame as there is no VNI on TN6.

If all the host tunnel endpoints are on the same subnet, the choice of replication mode does not make any different because the behaviour will not differ. If the host tunnel endpoints are on different subnets hierarchial two-tier replication helps distribute the load among multiple hosts. Hierarchial two-tier is the default replication mode.

So this is about the replication modes. I hope this is informative for you and I thank you for reading. In my next blog post, we will discuss the switching profiles.

Please refer to the VMware Documentation for more information on NSX-T Data Center product at https://docs.vmware.com/en/VMware-NSX-T/index.html