SoC debugging

Many issues arise at SOC-level across the Clock domains , IP Blocks ,FPGA prototyping, multi-socket ASICs, HW, FW and SW layers. Certus enables debugging system and chip-level issues by enabling efficient instrumentation of tens of thousands of signals, with the ability to capture very deep traces. Using software tools like the GNU Debugger , breakpoints can be set in the software code.

If this point in the program reached, the program execution is stalled and the program control is handed over to the debugger. Using the debugger, a developer can now read register or memory values, print stack traces, and much more. To be efficient, run-control debugging functionality needs hardware support to stop the program execution at a given time. In addition, run-control debugging on SoC platforms is usually done remotely.

Currently OSD supports timestamps which are full numbers of configurable width. Some debug modules can be configured to enable or disable timestamp generation. Timestamps are added to the trace data at the source, as opposed to (e.g.) adding timestamps when the data is received by a debug adapter hardware between the SoC and the host PC. Many smaller single-core designs traditionally only support run-control debugging through custom JTAG-based debug infrastructure.

Concurrent software distributed across multiple CPUs and hardware accelerators, interacting with complex I/O interfaces and strict real-time requirements is the new normal. This results in new classes of bugs which are hard to find, like race conditions, deadlocks, and severely degraded performance for no obvious reason. To find such bugs, run-control debugging is not applicable: setting a breakpoint disturbs the temporal relationship between the different threads of execution.

Tracing avoids these problems by unobtrusively monitoring the program execution and transferring the observations off-chip. There, the program flow can be reconstructed and the program behavior analyzed. OSD comes with components to enable tracing for not only CPU cores, but also for any component in the SoC, such as memories, hardware accelerators, and interconnects.

Memory Access Reading and writing memories is an essential tool during bring-up and debugging of a SoC. A typical use case is to write software to a program memory from the host PC, to avoid writing it for example to a SD card or flash memory and then resetting the system. OSD ships with a module that can be attached to a memory to support reads and writes from and to memories

Setting up a debugger on a host PC to communicate with the hardware often requires obscure configuration settings, secret switches and a bit of magic sauce to make it all work. OSD is designed to be plug-and-play. All hardware components are self-describing. When a host connects to the system, it first enumerates all available components, and reads necessary configuration bits..

Timestamps enable correlation of events in different parts of the chip with each other. Additionally, they can be used to restore order to events which are (for some reason) out of order when they arrive. While timestamps are useful in many cases, adding them to all events generated by the debug system can significantly increase the overhead of such events.

The integration of Imperas Multi-Processor Debugger enables UltraDevelop 2 to support multi-core, multi-threaded platforms, including a blend of cores utilizing different processor architectures, enabling the development of complex heterogeneous systems which are becoming increasingly common as developers mix and match CPUs to suit specific functions and applications.UltraDevelop 2 is built to give SoC designers exactly this optimal blend of functionality and flexibility in their choice of development platform, with the potential for real-time run control of over 20 processor architectures from vendors including Arm, MIPS and RISC-V. Developers can deploy third-party tools from existing UltraSoC partners such as Lauterbach, with support for the underlying UltraSoC hardware capabilities; or they can opt for a pre-integrated configuration supplied by UltraSoC. The fully featured configuration, UltraDevelop Pro, includes full multi-core debug based on the Imperas MPD debugger; visualization via Percepio’s Traceable tool; and library-based and (Python) scriptable data science .It includes a library of debug adapters (based on OpenOCD) to enable real-time run control of more than 20 processor core architectures from multiple vendors, including Arm, MIPS and RISC-V amongst others.

For simpler systems, the tools are provided with the open source gdb debugger. For more complex multicore systems, MPD (from Imperas) allows UltraDevelop Pro users to simultaneously debug multiple application processors, including single core, multi-core and multi-threaded variants. Peripherals can be debugged at the same time as the application, letting the developer see the peripherals operating in the context of the platform and the application code, and further extending the hardware/software co-development capabilities within UltraDevelop Pro.UltraDevelop 2 makes use of industry-standard interfaces such as the Eclipse Target Communication Framework (TCF), the GDB Remote Serial Protocol (RSP), Common Trace Format (CTF), and MI, the machine interface layer commonly used to communicate between a debugger’s backend and the IDE front end.

Modular, consisting of over 30 different parameterized and configurable modules that can be used to debug third-party IP cores or custom code; it also includes an on-chip messaging network. The system architect controls which points they wish to monitor, to what level of detail, how information is passed around the system and communicated externally. many cores such as ARM Cortex (v7 and v8 ISA) and MIPS, along with Cadence (Tensilica) XTensa and CEVA Teaklitea are supported.We can also monitor bus and memory structures and custom logic: in fact, most common on-chip structures are supported.We support many, including: AXI, AXI-4, ACE and OCP3.0. Our approach is easily extended to proprietary packet-based protocols and buses.First, in respect of processors UltraDebug is multi-vendor and non-proprietary, so supports any heterogeneous mix of processors from different vendors. Combining an application processor from one vendor, GPU from another and DSP from a third can often be a challenge, with multiple different incompatible development systems, each operating in a silo. In contrast, UltraDebug is a single, coherent platform.UltraSoC can either be used to replace CoreSight, or can be used as a “wrapper”, interworking seamlessly with CoreSight and providing greatly enhanced functionality.Secondly, UltraDebug was also designed for advanced symmetric multi-core systems, so it scales seamlessly to support the large clusters or arrays being designed at 16nm, 10nm and beyond.First, it is much faster than JTAG or CoreSight SWD. Debugging over serial interfaces can be painfully slow; using USB can save a lot of time.

Second, in many systems the debug interface is not exposed in the production form. This allows developers to continue to debug systems in final form factor.Finally, pins have a cost. By re-using existing pins, it eliminates the need for dedicated test pins, saving money.UltraSoC is fully message-based and has been designed to work with practically any interface including USB and Ethernet.

Ethernet is in development, based on a standalone IP component, enabling high performance and can enable independent run-control of CPUs and other system components. Such a component will have a comparable gate-count to a similarly specified Ethernet system IP.Other system interfaces (PCIe, SATA, HDMI, CAN etc).UltraSoC is supplied with verification IP to drive and where appropriate monitor UltraSoC interfaces.The components have optional fine-grained dynamic clock gating to lower dynamic power significantly when components or specific features are not in use. This enables UltraSoC monitors to be co-located with the system components they observe and used as part of the final product. UltraSoC can be disabled or powered off in part or as a whole as it supports multiple power and clock domains.We are using constraint random testing for our blocks with standard UVM. For each block we have a verification plan which lists the functional coverage bins to be developed.

When a block gets frozen, we stress test it, running tests for many days and gather functional and code coverage, which gets reviewed. All tests in this run must pass. Target is 100% for code, branch, condition, FSM and functional coverage using excludes (which are reviewed too). Functional coverage might contain coverpoints which are a cross of many elements to indicate any unexpected bias in our tests.

Nightly regressions are run for all blocks to ensure stability

It is supplied as class-based System Verilog and can be used in SV UVM environments and Specman/e environments.

Legacy Verilog environments are supported using an adapter module.

Try to maintain SV compatibility for 4 tools such as Verilator, VCS, Vivado simulator and Vivado Synthesizer.

Guidelines

Avoid using arrayed interface any where. (port or inside a module) Existing issues in: ISim, Vivado , avoid using functions/tasks in interfaces. Avoid using simple names, such as “length”, “size”, “out”, “in”. They can be mistaken into functions. , avoid “assign” member elements of a struct, use always_comb instead. Existing issues in: ISim ,Avoid using interface as data buffer inside a module. Avoid using interface to connect multiple hierarchical ports, avoid using interface as top-level ports. Interfaces are flattened after synthesis, which causes port mismatch between behavioural DUT and post-syn DUT,Arguably avoid using interface modport. Some tool cannot correctly check the modport input/output definition anyway. Existing issues in: ISim(no check), Verilator(no check) Use always_comb rather than always_comb @(*). Even wild-cased sensitive list is an error in VCS. Existing issues in: VCS