What Is Inside aMulti-10GE Port Switch?
Fancy Wang
Helping Global Enterprises Optimize Network Performance | Ethernet Card & Switch Solutions
Fancy Wang 0429 2020
The following part of the information comes from the IXIA platform test report
10-Gigabit Ethernet LAN
The biggest market for 10-Gigabit Ethernetis likely to be in the corporate backbone.The cost of 1-Gigabit and 10-Gigabit Ethernet has dropped significantly, which will trigger the demand for 10GE services and products.
In the enterprise network, 10-Gigabit Ethernet is used for:
? Switch-to-switch links for very high speed connections within the samewiring closet or data center or between different buildings.
? Aggregation of multiple-fiber1000BASE-X or copper 1000BASE-T segments into 10GE uplinks. Nearly allnew desktops and laptops are now shipped with 10/100/1000BASE-T ports.
? System Area Networking (SAN) applications providing server interconnects for clusters of servers.10-Gigabit Ethernet plays a key role in interconnecting the new generation of high-performance servers with multigigahertz 2/4/8-way processors required for moving large amount of data between server farms.
10-Gigabit Ethernet Metro
At the Metro Area Network (MAN) level,service providers face tremendouspressure to expand capacity for broadband local access, high-speed WANs, storage,and campus /enterprise grids. Standardsbased 10-Gigabit Ethernet enables service providers to extend their 10GE networks into the metro edge using their dark fiber,especially when 10GE is combined with metro area optical fiber networks based on.
Wave Division Multiplexing (WDM).10GE MANs can be deployed in either a star or ring topology. Unlike Resilient
Packet Ring (RPR) networks, 10GE networks use the standard Ethernet MAC protocol. The latest 10GE metro switches can provide network reliability similar to that of networks based on SONET/SDH rings.
10-Gigabit Ethernet WAN 10GE WAN applications are similar to MAN applications, but leverage existing SONET networks rather than new infrastructure.
SONET is the dominant transport protocol in the WAN backbone today, and most MAN public services are offered over SONET networks.
The 10-Gigabit Ethernet interface (WAN PHY) is compatible with SONET-based Time Division Multiplexing (TDM) and with the payload rate of OC-192c/SDH VC-4-64c.
The 10-Gigabit WAN interface facilitates the transport of native Ethernet packets across the WAN transport network, with no need for protocol conversion. This not only improves the performance of the network,but makes it simpler and less costly to manage.
Ethernet’s flexibility and universal availability will fuel the demand for 10-Gigabit Ethernet deployment in multiple network segments, from corporate networks, to data centers, and throughout service provider networks. This ongoing demand will require system vendors to deliver high-performing 10-Gigabit systems that meet service providers’ stringentrequirements — requirements that ensure high availability to corporate networks and performance that meets service level agreements (SLAs).
What Is Inside a Multi-10GE Port Switch?
Testing the new generation of 10GE switches requires a clear understanding of the various building blocks that make up a switch, and their interaction, to determine the various stress points within a switch and identify the weakest link in the chain.
Figure 1 shows the major components of a 10GE switch, including the line card, the switch fabric card, and the controller card. The following subsections provide an overview of the 10GE switch/router components crucial to performance testing.
Packet buffer
The packet buffer is a temporary repository for arriving packets while they wait to be processed.
Packet processing function
The packet processor is an optimized ASIC or programmable device (Network Processing Unit, or NPU) for processing and forwarding packets in the data plane or fast path. It performs specific key tasks such as parsing the header, pattern matching or classification, table look-ups,packet modification, and packet forwarding, ideally at wire speed.
Classification tables
The classification table is a special memory used by the packet processor. It may contain the following databases:
? The routing table determines where to route incoming packets.
? Access Control Lists (ACLs) contain information to grant or deny permission to specific users or groups
? The flow classification table contains information about a particular user or group of users, protocols, and applications. Flow identification information is used to determine QoS treatment, packet policing, and perflow queuing and billing.
? The label table contains information about VLANs, stacked VLANs, MPLS labels, etc.
Context memory
Context memory contains instructions about whether to deny or forward packets,where to forward them, all the internal system headers needed to get a packet from an ingress to an egress port, and the external packet header (new MAC, IP,MPLS stacks, etc.). It also holds information relevant to metering the packet and other miscellaneous information about how to process a packet.
Traffic management function
The traffic management function is used to regulate the flow of traffic. It forwards traffic according to a user-defined set of rules pertaining to priority levels, latency and bandwidth guarantees, and congestion levels. It also provides the buffering required to work with any queuing mechanisms it uses to manage traffic flow across the switch fabric. It may also include the high-speed SERDES (serializer/deserializer) function used to connect to the switch fabric.
Switch fabric
The switch fabric provides data plane interconnection among all line cards in the system. The switch fabric typically employs a crossbar to move packets between its ingress and egress ports. An NxN crossbar switch fabric allows N line cards in a system to be interconnected in a chassis.
This allows each slot to simultaneously send and receive traffic over the switch fabric.
What Happens to a Packet When It Enters a 10GE Port?
This section provides a high-level walkthrough of the “life of a packet” as it goes from a port on an ingress line card through the switch fabric card and exits on an egress line card. Figure 2 shows the logical operation of various line cards in a multi-10GE port switch, and identifies the stress points (numbered red circles).
Storing the packet
When a packet arrives on a 10GE port, it is stored in ingress buffer memory while waiting to be processed by the packet processor. When the packet processor is ready, the packet header is copied into the packet processor memory for processing.
Processing the packet
The packet processor (ASIC or NPU) examines the packet to determine whether a packet should be filtered or forwarded. It makes this determination by parsing the packet header and then performing packet and flow classification.
The classification process maps information extracted from the packet header to information stored in local tables maintained by the control plane processor.
The information in the classification tables typically includes forwarding tables,routing tables, and profiles or rules for a given packet or flow, such as policies for ACLs, Quality of Service (QoS), and Class ofService (CoS).
The classification typically points to many fragments of information about a packet. This information is usually saved in context memory and may contain instructions about whether to deny or forward the packet, where to forward it, all additional header information to get a packet from ingress to egress port, the new header used when it leaves the egress port (new MAC, IP, MPLS stacks, etc.), information relevant to policing the packet (see "Metering and statistics recording" below), and other information about how to process the packet.
Metering and statistics recording
Packet metering checks the conformance of a flow to a subscribed traffic policy or contract. The metered information is then used to decide whether to discard or forward the packet. Bandwidth allocations may be established in contracts that specify the per-flow, steady state, and short term peak (burst) rates.
Typically, a Token Bucket Algorithm is used at the ingress port to check the compliance of each flow to its contract. (A detailed discussion of the Token Bucket Algorithm is outside the scope of this document). The metered packets that violate the assigned contract can be marked for a range of treatments: unconditional drop, drop if no internal resources available, or pass on the packet dropping decision to the downstream device. The marking of packets can also be used by the egress line card to decide whether to delay packets within a packet stream in order to conform to some defined traffic profile, referred to as traffic shaping. Traffic shaping is used to smooth out packet flows and regulate the rate and volume of traffic admitted to the network.
Packet modification
Based on the outcome of the classification process, the next step is to modify the packet header and determine the packet’s egress port. Context memory contains instructions about whether to deny or forward packets, where to forward them, and all the internal system headers needed to get a packet from an ingress to an egress port. On the egress line card, a new external packet header (new MAC, IP, MPLS stacks, etc.) will be inserted by the egress packet processing function.
Traffic management
The packet processor typically appends an internal forwarding header to each packet that it forwards to the traffic management device. This header contains information that tells the ingress/egress traffic managers where to forward the packet (which port and line card) and the characteristics of the flow. The traffic manager then passes through or discards flagged packets, depending on its policies for the ingress direction.
Depending on the architecture, forwarded packets are organized into FIFO (First-In, First-Out) queues, input queues, or Virtual Output Queues (VOQs). Queued traffic is then scheduled for transmission to the switch fabric based on a scheduling algorithm — for example, Weighted Round Robin (WRR), Weighted Fair Queuing (WFQ), etc.
Depending on whether or not the switch fabric supports fixed or variable length packets, packet segmentation into fixed cells may occur before packets are transmitted onto the backplane. The ingress traffic manager typically interacts with the switch fabric and/or egress traffic manager to meet switch fabric scheduling requirements and avoid congestion.
On the egress side, the traffic manager first performs a reassembly operation if segmentation was performed on the ingress side. Received packets are then placed into an output queue based on information from the header that was appended at ingress. If per-flow queuing is supported, the egress traffic manager maintains an additional output queue for each flow. Each output queue provides traffic shaping to maintain conformance to QoS policy.
Switch fabric
The switch fabric functions as a highperformance data plane interconnect between its ingress and egress ports. Most new crossbar switch architectures have embedded high speed SERDES (serializer/deserializer) interconnecting line cards
through centralized switch devices. A packet entering a switch is transferred to the switching engine. The switching engine.then examines the packet header and decides where to forward the packet.
When multiple ingress ports contend for an egress port, congestion becomes the primary issue at the egress switch port and, depending on the architecture, may cause Head-of-Line blocking (HOL). HOL blocking occurs with input queuing, when a cell (a segmented packet to be transmitted across the backplane) has to be held up, thereby blocking the progress of any cells behind it, even if they could otherwise be switched.
Some switch architectures also provide multicast packet replication. To get around packet replication issues, most modern switch fabrics now allow for an incoming packet to be broadcast to any number of egress ports. Typically, on the ingress and egress line cards, separate queues and scheduling disciplines are maintained for unicast and multicast traffic