Challenges of testing FPGA boards in production

Today, FPGAs populate all the market segments globally and their application is ever growing. Being a very popular and often the only viable choice due to their inherent flexibility, streamlined design start, and decent processing power FPGAs offer quick time-to-market even for very demanding and complex projects. Moreover, an FPGA-based product is seemingly easy to test in production as most of FPGAs support classical JTAG based test access for Boundary Scan.

The reality is quite the opposite: FPGAs imply high density of traces, multiple BGA component packages high-speed interfaces and peripherals sensitive to signal integrity issues. Hence, even a combined application of common test techniques like In-Circuit Test (ICT), Boundary Scan (BST), and Flying Probe (FP) would not guarantee a complete coverage of structural defects, which may lead to serious consequences such as system failures, operational instability, or even safety hazards - all due to test escapes.

Many companies respond by developing a thorough Functional Test (FCT) solution for their FPGA-based products. Unfortunately, the steam is often too quickly over, resulting in a simple repurposing of an existing application firmware for production testing needs. Moreover, a factory line often sees a much simplified version of the test setup, while more thorough testing is only applied after a final product assembly. Is this good or bad? It is definitely a more expensive and risky approach compared to best practices used by leaders.

Let's quickly dive into all these aspects in search for an efficient test methodology, which applies to the whole product cycle, starting from the prototyping stage, to mass production, to troubleshooting and field return processing.

What is a typical FPGA board today and how to test it?

Due to their versatility, on-board FPGAs would nowadays host most of the product's or unit's functionality migrated into this large programmable device. Hence, the FPGA would be a centric unit the board's architecture is built around, becoming a hub connected to most of buses, peripherals and other infrastructure.

Enjoying a high degree of integration and miniaturization, the board's inventory is mostly comprised of complex digital components with small footprint and BGA packages (incl. FPGA itself), which on the contrary complicates testing. This is further escalated by strict signal integrity requirements of high-speed devices and gigabit links and a limited number of test points board's surface can host. Hence, the efficiency of traditional structural tests like ICT or BST taken alone has been significantly compromised.

On the other hand, the FPGA device itself offers an excellent access to the rest of the board's infrastructure. Moreover, most boards have standard periphery like Ethernet and USB PHYs, DDR memories, an SPI flash, I2C / SPI devices, I/O connectors, etc. This offers an excellent opportunity to build an effective Functional Test (FCT) strategy.

Indeed, our experience in numerous customer projects shows that the ability to leverage the functional test to target all the high-speed interfaces, as well as simplify testing of standard peripheral devices and buses, is the key to success.?

Obviously, the functional test has to be implemented in a way that carefully addresses these integration trends, elevated quality requirements and demanding physical limitations.

In the past, a typical FCT strategy was relying on a simple on-board firmware accompanied by T&M instrumentation (digital I/Os, signal generators/analyzers, DMMs, oscilloscopes, etc.) hosted in a bulky 19" rack chassis. The greater the speed the equipment can support, the higher the quality the test would deliver. This is not sufficient today, since FPGA concentrates lots of features inside offering no access to them from the outside.

Hence the ability to efficiently leverage the processing power of on-board FPGA is the key to success for a contemporary functional test strategy. This involves two components: a) the FPGA configuration (which functions the FPGA will host) during the test and b) the test program (test sequence) that used the FPGA internal resources during the FCT.

Who should develop the FCT firmware: board designer, test engineer or FPGA developer?

No good test quality is really possible without the FCT, but then, who is responsible and capable of developing both the FCT program and the test firmware to be hosted inside this FPGA?

The board designer knows the board by heart. He knows very well how it works: which working modes, timing requirements, and non-functional requirements are there. He knows how to drive bits and bytes, and what kind of data to squeeze through high-speed interfaces. Basically, he knows pretty well what needs to be done to the board to have good test coverage, but there is one obstacle now - the board has an FPGA! The board designers can probably do some simple software parts, but the FPGA world is completely different. So, usually, the board designer is not the right person to develop a test program.

What about a test engineer? Of course, the test engineer, just by the job title, has to be capable of doing this. But he faces the same issue, he can write some simple code, he knows how to execute tests, how to put them into a test sequence. He knows how to diagnose, how to troubleshoot, find a defect. He thinks in terms of shorts, opens, tombstone defects, missing components, wrong terminations, etc. He knows well how to measure the quality of the FCT that has to be developed to test the board. Still, he has very little understanding of how to program the FPGA.

Hence, we have no other choice than to turn to the last candidate, which is the FPGA developer. He knows very well how to handle devices like FPGA, he knows very well what to do. But normally FPGA developer is a very limited workforce and he has a lot of other tasks. His essential task is to close all application requirements that the product and firmware need to fulfill. On the other hand, there is no other choice for the product manager rather than adding this task to the FPGA developer on top of other responsibilities.

To minimize the time and effort, the FPGA developer usually takes the application firmware and tries to convert it to a test firmware. But we all need to remember that a test is not an application, these are completely different worlds. While the target of application firmware is to fulfill the product specification and make the board work as supposed to, the target of a test procedure is to check that there are no physical defects on a board that may prevent application firmware from correct operation. During a test, we are checking the nets for shorts, opens, crosstalk, missing components, etc. And what is important, the test firmware should be executed in an automated way and provide a pass/fail result with a good diagnostic resolution.

Sounds familiar? In the next edition, we will continue this study, bring some common examples that often arise when this approach is used, and analyze the consequences. Stay tuned!