How to Review a System Channel Layout and the Complications in Completing the Overall Channel Layout

Recently, I was busy binging to understand the system layout in detail and ended up watching the system layout series done by Robert Feranec from Fedevel academy ( This is neither a sponsored post nor he recommended his videos). I genuinely, felt it was a beautiful series on how to review a system layout design. Although there were many interesting points he revealed in the series, I want to add few more points to the series which will bring furthermore value addition. After reading this blog post, I seriously suggest watching those 7 episodes and enhance further your system knowledge. Each episode has many subtopics on which "n" number of posts can be created.

The idea is to throw a gist about how one can analyze these boards and add some more guidelines to their analysis. This post talks about my baby steps into the system design projects and about the above YouTube series.

Please find the link for the YouTube series: https://www.youtube.com/watch?v=FXZJ6jrpIKU.

SI Engineer life in a system design project:

First and foremost thing, if you are the responsible SI engineer for any system project, you are going to end up communicating with many cross-functional teams and this can be overwhelming. If you are leveraging on the previous designs of the project, then the process of completing the layout will help to meet your deadlines before it goes to the fabrication.

If it is a new design/project you are getting ready for a joy ride.

SI engineer role in a system project:

1. Designing the stack up, choosing the right connector, pin mapping, analyzing the routing done by the ECAD, performing SI simulations, power ac simulations, and so on and on and dozens of meetings. As said this could be quite overwhelming if you are handling multiple projects. You need definitely Camille Vasquez (Johnny Depp's lawyer) to support your back to handle the pressure you undergo sometimes.

YouTube Series:

Let us now delve into the YouTube series and learn from Robert's offering! Please keep in mind the layouts designed by Microsoft were pretty straightforward. For example, usage of FR4 material, low layer count, and simple routing. These may not be possible in our/your projects. These layouts should serve as a guide but not like a holy grail!

1. Episode 1: Project Olympus is Microsoft's next-generation rack-level solution that is open-sourced through Open Compute Project. Oh boy, currently these files are not available anymore for download. It is such a disappointment. I hope Robert keeps the material in some drive.

Lesson learned in this episode:

Try to add a block diagram of the whole architecture. This makes anyone's life easy. Let us say if you are working with a new CPU architecture and this CPU is connected to multiple cards (NPU, MPA) for data transmission purposes, it will be so easy to navigate and find the things that are needed for your purpose. I am going to implement it for sure in my new upcoming projects if the team agrees! Does your team follow this approach? If so can you comment if there is a better approach than the current one offered here? It will help the community.

Episode 2 (Platform Controller Hub):

This episode dealt with the server PCB layout analysis. In this episode, the author went over the schematic design and explained the sequences used for booting up the board. USB 3.0 connector schematic is explained along with the demonstration of the connector.

Lesson learned in this episode:

1. The designer in this board has connected thermal reliefs to connect the power pins to the BGA. The routing of the BGA is pretty clean and routed within 4 signal layers. The speaker advocates that he would not place capacitors around the BGA as against many chip vendors' suggestions because he felt there are no significant issues observed by him till now.
2. Discussion on SATA interfaces and how nicely they are routed together rather than scattered into different layers. The way length has been matched between P and N using the serpentine routing.
3. An important point that I learned and was always bothering me when I review the layout is where should I do the serpentine routing when I am laying the track between points A and B. It is always good to do the serpentine routing close to the area where there is a difference in the track length as highlighted with a green color arrow. You can see that trace P and trace N have different starting points. Better do serpentine routing at the start of those traces.

4. What is your assumption or thoughts about Microsoft engineer routing those serpentine traces pointed by those red arrows? Take a minute and comment on the post with your expertise! The reason here is to make sure the edges of the signal travel together (both P and N).

Additional Points:

1. USB will always be routed as differential pair with 90 ohms and from my understanding, the skew will be kept to be less than 300 picoseconds. Till now, none of my colleagues spoke about compliance testing for USB in-house.
2. Most of the time chip companies would have met with compliance specified by the standards. All, I did till now was to make sure to read the data sheet of the USB3/4 and make sure the chip from where I buy is meeting the compliance mask/target.
3. Slightly disagree with the author regarding the capacitor placement with its connection to PDN. It depends on the effective decoupling radius, and mutual inductance, and important is the fact that you can't rely on capacitors only, instead of the entire PDN in complex high-speed designs. In our discussions with some of the chip vendors, we have been told sometimes, that placing the capacitors underneath the BGA has yielded the worst performance in terms of impedance optimization and other power integrity parameters. So, it is something that needs to be understood and designed carefully.

Episode 3: Baseboard management controller board (BMC):

A blue screen on a laptop/desktop would have been seen by anyone who would have a laptop/desktop for some years. Once you experience this mayhem, you will be then routed to BIOS setup. What exactly BMC does is that it is a specialized service processor that monitors the physical state of a computer and communicates through an independent connection. It is usually contained within the motherboard or main circuit board of the board. In this episode 3, the discussion was around BMC controller design.

Lesson learned in this episode:

• Understanding the BIOS setup and how to read those BIOS setup connections.
• Understanding the DDR3 routing setup, and suggestions on the DDR3 signals breakout patterns. Remember the DDR3 used in this design is a low-power CPU and hence timings can be relaxed. However, timing in DDR3 and 4,5 are very critical and requires a good amount of understanding.

Additional Points:

1. Typically if you are using Intel's/ Broadcom DDR interfaces, most of the time they provide the routing guidelines, and the SI engineer can ask the ECAD team to implement the same without any deviations. However, if you are working on your OWN ASIC design, then the SI Engineer needs to do a pre-layout analysis on the DDR banks and see whether the suggested guidelines are working well and within the margins. Post layout analysis is mandatory in both conditions.
2. Additionally when there are power planes involved, the SI engineer tries to simulate the power integrity simulations to understand if the copper that is poured on the layout is sufficient enough and if capacitors that are provided in the layout can meet the target impedance.

Episode 4: PCIe Express Analysis

Points Discussed:

1. Importance of the BIOS post 80H codes on the motherboard and how these codes could be used for debugging analysis. Zig Zag Routing that discusses fiber weave effect. Look at my last post where I discussed in detail about Fiber weave effect.
2. In this board, the PCIe signals are routed on the bottom (Rx) and Top (Tx) layers and there is a significant amount of stub as the signals are routed via through hole connectors. Usage of stitching vias near the signal vias helps to control the return current path.
3. Mini SAS connectors routing guidelines and the discussion about how the breakout is done using the CFP2 Interface Optimization document.

Additional Points:

Let us understand whether stub impacts the PCIe signal performance. If you observe the below analysis done by Zachariah Peterson, it is about a MiniPC board in question that uses an Arria 10 FPGA with a PCIe interface, which is routed to a slot connector, as shown below. If we observe the insertion loss profile, we observe a resonance dip at 14GHz. The dip in this graph limits the data transfer rate to any value corresponding to a Nyquist frequency of approximately 8 GHz (or 16 Gbps for 2-level/NRZ signaling). This would be fine for PCIe Gen4 but not Gen5 PCIe. So always calculate your system budget for PCIe and decide whether back drilling is needed or not. I think you can check one of my favorite SI guru Lambert Simonovich papers on stub and I don't think you need any more references once you read his blog posts.

Episode 5: SFP + Connector

Analysis of SFP+ (Small form-factor pluggable) Connector:

Points Discussed :

1. Discussion on OCULINK. I am hearing this for the first time and after I googled it out, I figured out it is a competitor to Thunderbolt 3 in a different form factor.
2. SFP+ Connector: There was discussion on the removal of the copper beneath the connector but there was a reason for doing it and I will emphasize the technical aspect in the additional point discussion. SFP+ connector applications are typically used?in 10Gbps Ethernet, 8.5 Gbps Fibre Channel, 10.51 Gbps Fibre Channel, 10 Gbps Ethernet with FEC and in Telecom (SONET OC-192 and G.709 "OTU-2").?
3. Back drilling for the signals layer breakout has not been done and now this is something not great. Even now, I have seen people avoid back drilling because of the speed at which this application gets operated. However, avoiding back drilling is not a great option! We will see a few pointers below. Even for PCIe signals, they have not done back drilling!
4. Discussion on power supply management connector.

Additional Points:

I think enough has been said about back drilling vias. As a simple example, I had chosen a 10GBps channel simulation to demonstrate the performance of a 10 mil and 20 mil back drill stub performance. S-parameters used in the simulation are based on a PCB breakout with a simplified version of the SFP connector. You can observe how the eye opens for a 10 mil and 20 mil stub. Again, it depends on how much margin is available to you and how you define your channel loss budget.

I did not get the exact SFP connector but let us understand with a similar profile. The CFP2 host connector layout optimization reduces the impact of discontinuity at the differential pair to the CFP2 connector interface.

If we observe, there are oval anti pads and ground return vias placed closed to signal vias. All these are done to keep the impedance profile neither too capacitive nor too inductive. A reference plane cutout is provided beneath the connector pads and larger oval anti-pads are used for the signal vias. Four nearby ground return vias are provided to help reduce the connector interface discontinuity.

Episode 6: DDR 4 Memory Layout and CPU Power:

Points Discussed :

1. Coupons placed on the board - Typically these are needed when we want to test specific trace configuration of the board. In a connector design, we?typically enable calibration traces of 2X and 1X spacing to understand the impedance of the differential traces by specifying +5% tolerance to the fab vendor.
2. Discussion about the power controller chip. Honestly, I am a novice in power integrity analysis. Most of the time this analysis is taken care by the power integrity engineers as the scope of SI engineer mostly gets limited to the validating the impedance targets provided by the manufacturers/engineers.
3. DDR4 design and the reference to the power planes. I will write a separate post on DDR when i completely digest and swallow this whale topic!

The last episode is based on the miscellaneous part and can be seen online as I didn't see a reason to extend further.

Thanks for reading.