OSVVM 2020.08: Requirements Tracking + Updated Scripting

It has been a busy summer for OSVVM.  With the world working out of home offices, all of our training has shifted to on-line - which fortunately we started doing 7+ years ago.  Not having to travel has freed up additional time that I have devoted to OSVVM development.  

Updates started with adding PASSED counts to the AlertLog data structure in 2020.05.  AXI Full verification components (master, responder (transaction), and memory (also a responder)) with bursting was added in 2020.07.  A methodology defined set of transactions (which we call Model Independent Transactions) for Address Bus Interfaces and Streaming Interfaces was also added in 2020.07.  

Now after a long month of single-minded focus, release 2020.08 adds requirement tracking and updated scripting to OSVVM.  It is hard to believe it is September already – August went by so fast.  

Requirements Tracking

Requirement tracking has been a long-term project – something that I have talked to a number of training clients about for the past couple of years.  

Requirements are a little bit like coverage modeling and a little bit like affirmations. Like coverage modeling, they keep a count of the number of occurrences of an event.  Like affirmations, they track both PASSED and FAILED conditions.

In early 2020 I did the first experiment toward requirements tracking using coverage modeling to track passed and failed events.   This worked well, but it did not seem to be the right answer.  So, it was set aside temporarily while other solutions were explored.

As it turns out, there is more commonality between affirmations and requirements than coverage modeling.  The first step toward implementing requirements tracking in the form of tracking PASSED counts was added in 2020.05. Passed counts work well with either requirements tracking or self-checking tests.

Next PASSED goals were added.  To track requirements, PASSED goals need to be grouped together, so they are kept under a new built-in level of hierarchy in the AlertLog data structure – a Requirements level (REQUIREMENT_ALERTLOG_ID).  

A dedicated requirements level of hierarchy simplifies reporting requirements (ReportRequirements) as well as writing them out as a CSV file (WriteRequirements). ReadRequirements reads and merges requirements information collected from separate simulation runs - in OSVVM terms these are test cases implemented in a separate architecture of the test sequencer (our TestCtrl).  

To correlate requirements with specifications, ReadSpecification was added to pre-fill the requirements bins with items that must be tracked.  By default, if requirements are not met with ReportRequirements, then it results in a test failure.  

Requirements are tracked with AffirmIf, the same as we do for self-checking tests.  Since each requirement is only checked a couple of times within a given test, a new form of AffirmIf was added that looks up the requirement by its name rather than its AlertLogID.     

At this point, OSVVM requirements are ready for review by a wider audience.   Comments and suggestions for improvement are welcome.  Documentation is currently part of the AlertLogPkg User Guide that is in the documentation repository (https://github.com/OSVVM/Documentation).

Updated OSVVM Scripting

OSVVM scripting is another big feature update for 2020.08.  OSVVM Scripting provides a simple way to create and activate libraries, compile designs, and run simulations – all in a simulator independent fashion.  The intent is to write one set of simulator scripts, that is as easy to read as a compile order list, that references files relative to the current directory (so you don't have to think about where the script is being called from), and that provides control capability (such as if then) when it is needed. 

To keep the implementation of OSVVM scripting light weight, rather than parsing our scripts, such as one would do with some form of a descriptor file, we have created an abstraction layer on top of TCL. With this abstraction layer, OSVVM scripts are executable.  This means that OSVVM scripts inherently have all of the power of TCL available.  Keep in mind though the idea is to simplify.  So, use the abstractions where possible and only use TCL when necessary for flow control. Currently only the OSVVM utility library requires the extra TCL capability.   

OSVVM scripts currently support Aldec and Mentor simulators. Looking for other simulator support – feel free to reach out to me directly for help.   

Ready to get started with OSVVM scripting.  The README.md in the scripts repository (https://github.com/OSVVM/OSVVM-Scripts) provides a step by step guide to using the OSVVM scripting to compile all of the OSVVM models.   

About OSVVM

Open Source VHDL Verification Methodology (OSVVM) is a methodology and libraries that simplify the creation of structured testbenches that are readable and powerful. OSVVM supports the same capabilities that other verification languages (such as SystemVerilog + UVM) support – from a structured testbench framework using transaction level modeling, to functional coverage and randomized test generation, to scoreboards and FIFOs, to error handling utilities, to synchronization utilities, to memory modeling, and to verification components - All you need is OSVVM.

OSVVM verification components are a growing set of open source verification components.  They now include:

• AXI4 Master – supports single word and burst transfers
• AXI4 Memory – supports single word and burst access to an internal memory
• AXI4 Lite Responder – supports single word transfers
• AXI4 Lite Master
• AXI4 Lite Memory
• AXI4 Lite Responder
• AXI Stream Transmitter
• AXI Stream Receiver
• UART Transmitter (with error injection)
• UART Receiver (with error injection)

Testbenches and OSVVM simulation scripts for each model are in the Git repository, so you can run a simulation and see a live example of how to use the models (directions are below). 

Documentation

There is extensive documentation on the OSVVM utility library in the documentation repository (https://github.com/OSVVM/Documentation).   

On the other hand, formal documentation for the verification components is a work in progress.   For now, you can learn about OSVVM verification components by seeing the following two presentations and by running testbenches. 

• Creating an AXI4 Lite, Transaction Based VHDL Testbench with OSVVM (https://www.aldec.com/en/support/resources/multimedia/webinars/2083)
• Creating Better Self-Checking FPGA Verification Tests with Open Source VHDL Verification Methodology (OSVVM) (https://www.aldec.com/en/support/resources/multimedia/webinars/2094)
• A great way to learn OSVVM Verification Components is to run the testbenches and then review the test cases (see next section)

Getting and Running the testbenches

The OsvvmLibraries repository contains all other OSVVM repositories as a submodule.  Hence, when you get OsvvmLibraries, you get everything you need. You can get OsvvmLibraries by calling git clone with the "--recursive" option:

$ git clone --recursive https://github.com/osvvm/OsvvmLibraries

A zip file is available at https://osvvm.org/downloads.

Once you have OsvvmLibraries, follow the steps in the README.md in the OSVVM-Scripts repository (https://github.com/OSVVM/OSVVM-Scripts/) to compile the verification components and run the simulations.

What Would You Like Next In OSVVM?

In the comments, let us know what you would like to see next. What verification components do you need?  What extensions in the utility library would you like to see?

Want to help adding features and/or verification components to OSVVM?  Join us.  See the next section for details.

OSVVM + IEEE Open Source project.  

OSVVM is now an IEEE Open Source project.  Join us in further developing OSVVM.  More details about participating are in my OSVVM Call for Participation blog post (https://www.dhirubhai.net/pulse/osvvm-call-participation-jim-lewis/) and the contributing guidelines are in https://opensource.ieee.org/osvvm/OsvvmLibraries/-/blob/master/CONTRIBUTING.md.  

Don’t let the IEEE login intimidate you, you get that for free (ie: no membership required) – just like at GitHub.

Getting Started with OSVVM

The fastest way to get started with OSVVM is SynthWorks' Advanced VHDL Testbenches and Verification which is available world wide either on-line or on-site (once we can travel again). Here is our current class schedule.  

Funding OSVVM

OSVVM is free, open source software. With this changing world, we are looking for funding sources for OSVVM.  If you know of any organizations who can support our work, please reach out to me.