VXLAN for Mortals (Part-1)

Disclaimer: The goal for this article is to give you good clarity on what VXLAN is, and how it works. While it covers some configuration aspects, it doesn't cover too much.

What will you learn in Part-1?

• Appreciate the need for VXLAN
• Understand how VXLAN is transported
• Understand what a VTEP is, and how VXLANs interact with it
• Understand the role multicast plays in VXLAN (when you are not using BGP EVPN)

Why do we need VXLAN?

In Data Centers, Layer-2 connectivity is very important for virtualisation stack and sometimes for applications too. And when your data-center hosts applications/VMs for multiple departments and multiple tenants, you will need separate broadcast domains for each segment.

And VLAN is our good friend to separate broadcast domains. VLANs give use 4096 broadcast domains within a physical LAN. Technically speaking, many of these 4096 are not really usable. So, lets say 4000 broadcast domains.

Are 4000 enough these days? Nope. We need much more than that.

What else? For handling Layer-2 topologies, you need Spanning-Tree Protocol. And we all know that running STP blocks several healthy ports in a bid to avoid loops. Bummer!

So, two key challenges

1. Not enough broadcast domains provided by VLANs alone
2. Handling a Layer-2 topology needs STP, and that is a bit inefficient in utilising network capacity.

Can we put another tag on top of ethernet frame that provides even more segmentation options? ??

Enter VXLAN. VXLAN offers 24-bits for segmentation, thats 16 million segments.

VXLAN Encapsulation

This is good. How to transport this new VXLAN frame now? We use UDP/IP to do this.

The moment you decide to use UDP/IP for VXLAN transport, you are unlocking the power of IP Transport - ECMP - Equal Cost Multi-Path.

ECMP with VXLAN (by Default) lets you use the available network capacity much more efficiently, compared to plain VLAN

Comparing VLAN vs VXLAN

With VLANs, your topology looks somewhat like below (for a simple scenario)

Lets make use of VXLAN here. We know we can encapsulate VLAN 10 frame inside VXLAN. So, first thing we need to understand is, for every VLAN ?? there is a VXLAN.

(On the same switch) A VXLAN can have multiple VLANs mapped to it. But a VLAN can only map to one VXLAN

And so, the new topology when you use VXLAN looks like

By doing this, you can now map VLAN-10 of different tenants to different VXLANs.

Tenant-A : VLAN 10 : VXLAN 10000
Tenant-B : VLAN 10 : VXLAN 20000        

In real-world, you will have VRFs separating the tenants too. But at layer-2 though, you still have VXLAN ?? VLAN mapping.

VXLAN also allows you to easily mix VLANs, like below

Both Host-A and Host-B are mapped to VXLAN 10000 here. Even though they have different VLANs, they can communicate with each other nicely!

VTEPs - Transporting VXLANs

We saw earlier that VXLAN is transported within UDP/IP to their respective destinations. In the above simple two switch setup, how does Switch-A send VXLAN packets to Switch-B?

Each switch that participates in VXLAN has something called an NVE (Network Virtual Interface). Its a virtual interface (like loopback).

All the VXLANs on that switch use that NVE interface IP address as their source, when they leave that switch. Similarly, to reach a VXLAN on another switch you aim your VXLAN packets at the NVE interface of the target switch.

In typical environments, you will have multiple VXLAN on a given switch. So the same NVE is usually linked to all the VXLANs

This NVE is called VTEP (Virtual Tunnel End-Point)

Multicast in VXLAN

We all know that "Flood and Learn" is the way to learn which Mac owns the IP address within a VLAN.

When you are dealing with potentially tens of thousands of segments all over the data-center with VXLAN, you don't want "Flood and Learn" to be the way your VXLAN learns which Mac owns with IP address.

(On Control Plane) The goal is still the same, you must learn which Mac owns which IP address - even within VXLAN

VXLAN uses multicast to handle BUM traffic (Broadcast, Unknown Unicast, Multicast). So, all the switches that run VXLAN must implement multicast on all the physical and loopback interfaces that participate in VXLAN.

Unknown-Unicast refers to packets that are aimed at specific mac-address that is not known to the local switch.

Here is what happens

1. Host-A sends a packet to specific MAC and IP address on VLAN 10
2. Switch-A gets the packet. It doesn't know the MAC address. It sees that VLAN 10 is mapped to VXLAN 10000.
3. It encapsulates the packet in VXLAN and sends it to all other switches where VXLAN 10000 is configured. (See that it doesn't send packets to other switches where VXLAN 10000 is not found)

Assuming Switch-B has the destination mac locally, it forwards to the right destination while also updating Switch-A about the mac-address.

VXLAN Control Plane

What we saw till now is VXLAN using Multicast in the control-plane. There is a better option - BGP as the control plane.

VXLAN BGP EVPN is the most popular data transport for data-centers today.

Coming up Next

In the upcoming articles, I will explain

• How VXLAN facilitates Layer-3 connectivity
• Anycast-Gateway in VXLAN. What is it? Why do you want it?
• Configuration Snippets for VXLAN (For L2 and L3)