Verification - open source - what is what? Part 1/x.

Open source hardware has always been a struggle, much more than the software equivalent.

Patents, IP, licenses, ... you name it, we got them to deal with.

OpenCores was (and probably is) a great site for open source RTL.

For verification, we do have some options as well, but there seems to be some confusion around the terms and names floating around in this space.

Neil Johnson's post, June 2020.

My earliest memory from the nineties is simulation with VHDL testbenches.

Then co-simulation of the HDL simulator (Mentor, Cadence, Synopsys) via a Foreign Language Interface (FLI) started getting traction.

Back then the chip was first modeled in C/C++ and later SystemC. Not cycle accurate, it was a high level model that was refined in steps to RTL. The model generated golden reference output. Those golden reference files were used in top-level verification.

The boundary between the software process that interfaced with the HDL simulator was the IO of the RTL under test. In case you had an embedded processor, the RTL could be replaced with a model. That model connected via the FLI to the firmware in C/C++. Of course real RTL was used too with the C/C++ compiled to the ISA (Instruction Set Architecture) of the processor. But using the model, the C/C++ would run in a seperate thread in parallel with the HDL simulator thread.

Years later, I came across a Python based co-simulation with RTL. Python interpreted (could potentially be compiled too) code testbenches to generate and monitor the RTL.

The benefit of this co-simulation with a software (verification) side and the hardware (RTL) simulator:

• Verification is very close to the software world, hence use software people to do verification. There are 10x, 100x more software engineers in the world than hardware engineers.
• Bare metal software/firmware can be done as soon as the RTL is in good shape.
• The actual silicon and the prototype (FPGA, emulator) use the same software testbench. No EDA simulation license needed to run those testcases. A big issue in the cost of a tape-out (ASIC specific) is stupidity. Let anyone in charge choose between a cheap resource and a more expensive resource. Let them choose in hiring a human or throw money at more server time. We all know the choices excel sheet managers make. Ssshhhttt.

Monthy Python's "The Spanish Inquisition": Nobody expects the Spanish Inquisition!

(Probably unwise to mention this, because it will be soon renamed to something less condescending towards our Spanish Brothers in Arms.)

Back then, inquisitions and all, there was a famous test for witchcraft, if you burn on a pyre you weren't a witch. If you survive, they have proof you are a witch, the devil's offspring and you should be killed in another way. In both cases, you have lost your life.

In business, it is quite similar, you speak out and get burned or you stay silent and nobody ever mentions the fact that you could actually verify in other ways than the license (cost) heavy methodology that today rules the globe. It is clear that veterans whose memory doesn't falter and that aren't paid pundits, have lean ways to save millions of USD/EUR in the development cost of a digital circuit (ASIC or FPGA). You see, I do apply for verification roles often, the PARETO principle tells us that 20% of the time is spent in design and 80% in verification. A designer is a verification resource, you can't design an interrupt controller and expect it to be correct by construction. That is more the way software engineers think (correct by construction), hardware engineers need to be paranoid because a bug could cost millions.

VERIFICATION FRAMEWORK: Verification IP and TestBench

• SVUnit, SystemVerilog Unit: (AgileSoC.com).
• OSVVM, Open Source VHDL Verification Methodology: (osvvm.org).
• UVVM, Universal VHDL Verification Methodology: (bitvis.no).

REGRESSION FRAMEWORK: self-checking test automation

• VUnit, Unit Testing for VHDL and verilog/systemverilog, default uses OSVVM (see above) but can use any VHDL or (System)verilog verification framework (vunit.github.io).

CONCLUSION

Self-checking testbenches: PASS/FAIL are the only acceptable way.

Verification requires automation for:

• Regressions: verify the verification is still all passing after a RTL change.
• Code coverage.
• Assertions.

And we appreciate ReUse (reuse) of existing protocols (Verification IP).

The latter is difficult because in hardware, SPI is just a general term for some kind of Serial Parallel Interface. It is not a well defined standard. The number of IO pins differs, the addressing (in case of register control and status) and data definitions has many flavors. Plus, if we are going to verify USB 4 in 2020, we will have a hard time finding any (even commercial) VIP that supports the version 4 standard since the hardware IP has been announced or is just recently made available.

I feel that a comparison matrix of features could be a good idea to see how all open source frameworks compare.

Just like you, just searching my way through the jungle ...

#verilog #systemverilog #VHDL #verification #UVM #opensource #asicdesign #fpgadesign

Second article: Open source Verification, what is what Part 2 of x.

Third article: Open source Verification, what is what Part 3 of x.