Cisco ACI Traffic Forwarding Simplified | Part II

This article is part Two of a series dedicated to exploring how ACI manages various types of traffic, including Layer 2, Layer 3, and ARP traffic.

Previous Articles :

Part 1 : https://www.dhirubhai.net/pulse/cisco-aci-forwarding-simplified-part-i-mohammed-khalefa-xvkwf/

Endpoint Types

Overview

An essential aspect of ACI is understanding the types of endpoints it handles. Endpoints in ACI can be categorized as either local or remote. Let’s break down the differences between these two types and how they are managed within the system.

Local Endpoints

Local Endpoint

A local endpoint is any endpoint connected to a specific ACI leaf switch. For instance:

If Endpoint A is connected to Leaf 1 and Endpoint B is connected to Leaf 2:

o?? From Leaf 1’s perspective, Endpoint A is a local endpoint because it’s connected to Leaf 1.

o?? Similarly, from Leaf 2’s perspective, Endpoint B is a local endpoint because it’s connected to Leaf 2.

Local Endpoint Information in the Endpoint Table

Local endpoint information is stored in the Endpoint Table of the leaf switch. For example:

** If Endpoint A is connected to Leaf 1, the Endpoint Table on Leaf 1 will contain an entry such as:

o?? Endpoint A: Includes its MAC address, IP address (if available), and the interface (e.g., Ethernet 1/1) it is connected to.

Remote Endpoints

A remote endpoint is an endpoint that is not connected to the leaf switch but is reachable via another leaf switch. For example:

• From Leaf 1’s perspective, Endpoint B is a remote endpoint because it’s connected to Leaf 2, not to Leaf 1.

Remote Endpoint Learning

Remote endpoint information is also stored in the Endpoint Table, but with a key distinction:

A leaf switch will only learn about a remote endpoint if there is active communication between the remote endpoint and a local endpoint. This process is called conversational learning in ACI.

For example:

o?? If Endpoint B communicates with Endpoint A (local to Leaf 1), Leaf 1 will learn about Endpoint B as a remote endpoint.

VXLAN Tunneling for Remote Endpoints

When Leaf 1 learns about Endpoint B as a remote endpoint, it also records the location of Endpoint B. This location includes the VTEP address of Leaf 2. Communication btween endpoints over the fabric occurs over a VXLAN tunnel, which simulates a Layer 2 extension similar to a trunk link in traditional networks.

Endpoint Table Entries: Example

Let’s consider the Endpoint Table on Leaf 1:

1.??? Endpoint A (Local):

o?? Learned as a local endpoint.

o?? Includes the MAC address, IP address (if available), and the interface (e.g., Ethernet 1/1).

2.??? Endpoint B (Remote):

o?? Learned as a remote endpoint due to communication with Endpoint A.

o?? Includes:

§? The MAC address or IP address of Endpoint B.

§? The VTEP address of Leaf 2, showing how to reach Endpoint B via the VXLAN tunnel.

Key Characteristics of Remote Endpoints

·??????? Remote endpoints are stored in the Endpoint Table as a cache.

·??????? This cache is cleared faster than entries for local endpoints, meaning remote endpoint information has a shorter lifespan in the table.

COOP Database

Overview

The COOP (Council of Oracles Protocol) database is a fundamental component of Cisco ACI, designed to manage and centralize endpoint information. Understanding how the COOP database works is crucial for grasping how ACI handles endpoint learning and forwarding.

How the COOP Database Works

Local Endpoint Reporting

When a leaf switch learns about a specific local endpoint, it reports this information to the spine switches, which maintain the COOP database. Here’s an example:

• Leaf 1 learns about Endpoint A (MAC A, IP A), Leaf 1 reports Endpoint A to the spine.
• Leaf 2 learns about Endpoint B (MAC B, IP B), Leaf 2 reports Endpoint B to the spine.
• Leaf 3 learns about Endpoint C (MAC C, IP C), Leaf 3 reports Endpoint C to the spine.

Each leaf only reports its local endpoints (endpoints connected to it) to the spines.

Remote endpoints are not reported to the spines. For example:

• If Leaf 1 learns about Endpoint C as a remote endpoint (due to communication between Endpoint A, local to Leaf 1, and Endpoint C, remote), it does not report Endpoint C to the spine.

Role of the COOP Database

The COOP database serves as a centralized repository located on the spine switches. It stores information about all local endpoints across the entire ACI fabric. The spines synchronize this information among themselves to ensure consistency.

Citizen and Oracle Roles

Leaf Switches (Citizens):

o?? Report their local endpoint information to the spines.

Spine Switches (Oracles):

o?? Maintain the full COOP database.

o?? Synchronize the database with other spines to ensure redundancy and consistency.

Example:

• Leaf 1 reports Endpoint A to Spine 1.
• Spine 1 updates its COOP database and synchronizes this information with the other spines.

Forwarding with Spine Proxy

The Spine Proxy mechanism is an essential feature of ACI forwarding, leveraging the COOP database for efficient communication.

Key Principles

1.??? Reduced Leaf Overhead:

o?? Leafs do not need to store information about all endpoints in the fabric.

o?? A leaf learns about an endpoint only when there’s active communication with it (conversational learning).

2.??? Unknown Destination Handling:

o?? If a leaf receives a packet destined for an endpoint it doesn’t know, it can forward the packet to the spine.

o?? The spine, using its complete COOP database, determines the correct destination leaf and forwards the packet accordingly.

Benefits of Spine Proxy

• Efficiency: Leafs focus only on endpoints they actively communicate with, minimizing resource consumption.
• Centralized Lookups: The spines handle endpoint lookups and forwarding decisions for unknown destinations.
• Scalability: The COOP database ensures efficient and scalable endpoint management across the ACI fabric.

See You in the next one,

Mohammed Khalefa

31.1.2025

Next Article :

PART3 : https://www.dhirubhai.net/pulse/cisco-aci-traffic-forwarding-simplified-part-iii-mohammed-khalefa-urzjc/