What is Co-Package Optics?

A typical IPL match can last four to five hours, and streaming it on your mobile device requires significant data, especially at higher resolutions. Watching a single IPL match in FHD (1080p) will consume around 3.5 GB of data, while 4K streaming jumps to over 22 GB. According to Google, Google Meet can use at least 2.6 GB of data per hour at HD (720p) video quality. This means using Google Meet at SD quality (480p) can consume 0.9 GB of data per hour. On average, one hour of Spotify streaming at Normal quality consumes around 40 MB of data, whereas streaming at High quality will use approximately 150 MB per hour, which is quite significant.

As demand for network and compute fabric bandwidth accelerates, innovation in system and chip architectures is crucial to counteract the slowing of Moore's law. Meanwhile, copper interconnects are nearing their bandwidth-distance limits. Silicon photonics is essential for sustaining rapid data growth and supporting high-bandwidth applications like Ethernet switching, AI/ML, and high-performance computing (HPC).

?Co-Packaged Optics (CPO) represents an advanced integration of optics and silicon on a single packaged substrate engineered to address the evolving challenges of bandwidth and power consumption. By uniting expertise in fiber optics, digital signal processing (DSP), switch ASICs, and state-of-the-art packaging and testing, CPO offers transformative value for data centers and cloud infrastructure.

Copper has been a staple in networks due to its high conductivity, affordability, malleability, and heat resistance. These properties made copper cables ideal for data networking, including long-distance runs between data centers and across metropolitan areas. However, as network speeds increased, the power and bandwidth needed to drive data signals over long copper runs also grew, prompting the search for a more efficient material. The 1990s saw a shift from copper to optical cabling for long runs because optical fiber offered lower signal loss, higher bandwidth, and reduced energy requirements.

Optical fiber not only offered these benefits but also facilitated smoother upgrades to network infrastructure with the introduction of new technologies. Fiber optic cables incorporate modular optical units that include an optical engine (OE) for converting optical signals to electrical signals and vice versa. These modular units provide a simple and adaptable method to link fiber optic cables to network equipment. They are inserted into connectors positioned on the edge of a PCB and the front panel of network equipment, utilizing an electrical interface between the module and the switch/router ASIC within the network device.

However, as data network speeds surpass 400 Gbps, reliance solely on optical fibers becomes inadequate. The power required to transmit electrical signals, even over short distances—from the switch ASIC near the PCB center to the modular units at the front panel—poses significant challenges. This is where co-packaged optics (CPO) become indispensable. With increasing network speeds, the limitations of passive copper connections become more evident, necessitating a transition to optical links for a larger proportion of connections.

Broadcom is partnering with Tencent to accelerate the adoption of high-bandwidth co-packaged optics (CPO) switches. The company announced that it will deploy its first switch with integrated optics, named Humboldt, in Tencent’s data centers next year. These switches are based on a new architecture introduced in early 2021, which integrates co-packaged optical interconnects within a traditional switch. This CPO technology relocates the optics and digital signal processors (DSPs) from the pluggable modules at the front of the switch, utilizes silicon photonics to transform them into chiplets, and places them in the same package as the ASIC. According to the company, Humboldt delivers 30% power savings and reduces cost per bit by 40% compared to systems using traditional switches.

Power consumption poses a significant challenge for companies like Tencent, which are building large data centers to support their clouds and various services. By 2025, data centers could account for 15% of global electricity demand. In a standard pluggable optical transceiver, the digital signal processor (DSP) performs signal processing to correct impairments caused as the signal traverses the circuit board and multiple connector discontinuities in a typical switch system. However, this process consumes substantial power.

Broadcom asserts that co-packaged optics will reduce the cost per bit compared to traditional switches by replacing energy-intensive pluggable modules with more efficient optical engines. They highlight that customers spend approximately ten times more on optics than on switch chips, which themselves can cost thousands of dollars each. With co-packaged optics, the need for optical modules extending from the switch faceplate is eliminated, resulting in significant cost savings as these pluggable modules can cost several hundred dollars each, depending on bandwidth requirements.

Broadcom notes that the world’s largest cloud-scale data centers, which are predominantly powered by Broadcom’s Tomahawk chips, contain more than 10,000 switches. These switches house over a million optical interconnects that facilitate data transfer with much lower power usage and cost compared to electrical interconnects at high speeds.

As the demand for increased speeds and reliability within data centers grows while simultaneously reducing bits-per-joule costs, co-packaged optics switches are seen as a transformative innovation. The Broadcom Tomahawk 5, a 51.2Tbps switch chip, incorporates eight 64-channel silicon photonic engines. This new chip powers 800Gbps of traffic at 5.5W by reducing the need for electrical signal driving to pluggable optics at the front of the switch. For context, the previous Tomahawk 4 Humboldt platform required approximately 6.4W for an 800Gbps link.

Benefits:

Initial implementations of Co-Packaged Optics (CPO) by Broadcom and Cisco demonstrate significant reductions in power consumption, achieving savings of 30-50% with an interconnection power of less than 1 pJ/bit.

• Elimination of lossy copper traces: Unlike traditional pluggable optics, CPO design removes the necessity for signals to traverse energy-draining copper links across the board from the ASIC chip to the front panel. Instead, CPO design integrates fiber directly into the switch, facilitating short, low-loss communication between the chip and the optical engine.
• Reduced reliance on digital signal processors (DSPs): Current architectures above 25G/lane require DSP-based retimers in pluggable optics to actively correct signal degradation, distortions, and timing issues, significantly increasing overall system power consumption by 25-30%. By eliminating off-chip lossy copper traces between the ASIC and optics, CPOs allow designers to eliminate one level of DSPs, leading to power savings and cost reduction.
• Integration of lasers: There are two prevailing strategies for laser placement. The conventional method uses an external laser, necessitating light transmission through fiber and coupling into the CPO, typically resulting in an optical power loss of 30-50%. Alternatively, integrating lasers directly onto the chip offers improved optical coupling, provided thermal management and laser reliability are effectively addressed.
• Enhanced bandwidth and reduced latency: CPOs deliver higher bandwidth and lower latency by minimizing DSP requirements and eliminating lengthy copper traces. Additional components like DSPs and parasitic effects in copper traces introduce delays that are mitigated in CPO solutions.

?Challenges:

• Thermal dynamics: Placing the photonic integrated circuit (PIC) inside the electrical package increases the risk of thermal interactions. Thermal energy from heaters and laser sources on the photonic die impacts the package's thermal profile, while the heat generated by electrical dies and managed by the overall cooling system affects the PIC's thermal behavior. A thorough thermal analysis is essential, spanning from individual dies to system-wide implications.
• Optical coupling: Achieving effective alignment of optical signals from the fiber array to the package presents a significant technical hurdle. Factors such as alignment precision (using passive or active methods), angular alignment considerations, structural integrity, thermal management strategies, manufacturability, and serviceability all demand meticulous attention. Design optimization of optical coupling mechanisms is paramount.
• Production feasibility and testing: The economic viability of any design hinges on low production costs and high yield rates, especially within a diverse supplier ecosystem. Ensuring consistent quality and implementing robust testing protocols remain ongoing challenges that evolve alongside market demands and investment priorities.
• Electrical and photonic integration: Ensuring the integrity of both signal transmission and power delivery necessitates comprehensive transient simulations across the entire system. This involves integrating cohesive electrical and photonic circuit simulations, accounting for additional parasitic effects introduced during the packaging phase by various electrical interconnect configurations.

We anticipate exploring additional opportunities in co-packaged optics during 2024-2025, given its potential to significantly reduce power consumption currently expended on short data movements within switches. As we progress into the 800G and eventually the 1.6T era, co-packaged optics will become increasingly critical. Presently, Broadcom has initiated customer sampling of its co-packaged optics solution.

References:

1. https://www.synopsys.com/blogs/chip-design/what-are-co-packaged-optics.html
2. https://www.ansys.com/en-in/blog/what-is-co-packaged-optics
3. https://www.broadcom.com/info/optics/cpo
4. https://www.electronicdesign.com/technologies/embedded/article/21248744/electronic-design-broadcoms-first-switch-with-co-packaged-optics-is-cloud-bound
5. https://www.servethehome.com/broadcom-now-sampling-51-2t-co-packaged-optics-switch/