An Ah-Ha Moment for Testbench Assembly

Sometimes we miss the forest for the trees, and I’m as guilty as anyone else. When we think testbenches, we rightly turn to UVM because that’s the agreed standard, and everyone has been investing their energy in learning UVM. UVM is fine, so why do we need to talk about anything different? That’s the forest and trees thing. We don’t need to change the way we define testbenches – the behavior and (largely) the top-level structure. But maybe there’s a better way to assemble that top level through a more structured assembly method than through hand-coding or ad-hoc scripting.

A parallel in design assembly

This sounds just like SoC design assembly. IPs are defined in RTL already, and you also want the top level in RTL because that’s the standard required by all design tools. But while top-level designs can be and should be defined in RTL, that is a cumbersome representation for assembly. Which is why so many design teams switch to spreadsheets and scripts to pull it together. Creation and updates are simpler through spreadsheets and scripts that handle the mechanical task of generating the final RTL.

UVM top levels present a similar problem for a different reason. The UVM methodology is very powerful and amply capable of representing a testbench top level. But it is a methodology defined by and for software engineers, full of object-oriented design and complex structures. All of which is foreign to the great majority of hardware verifiers who are not software experts. Worse still, UVM is sufficiently powerful that it does not constrain how components – VIPs, sequencers, scoreboards, etc. – define their interfaces. Which makes instantiating, configuring and hooking up these components a problem to be solved by the testbench integrator. Redundantly repeated between verification teams. This problem is well known. Verification teams typically spend weeks debugging testbenches before they can turn to debugging the design.

SoC verification requires that many testbenches be generated in support of the wide range of objectives defined in the test plan. Those objectives will be farmed out to multiple verification teams, often distributed across the globe. Most of whom are production verification engineers, not UVM experts. It is easy to see how effort you can’t afford to waste is wasted in support of an unstructured approach to testbench assembly.

Testbench assembly cries out for standardization

The Universal Verification Methodology is the foundation of any modern verification strategy. But few would deny that UVM, as a complex methodology designed around class-based design, is mystifying to the great majority of hardware verification engineers who are not experts in modern software programming concepts. A small team of UVM experts bridges the gap. They know how to construct the complex functions needed in SoC verification while also hiding that complexity behind functions or classes to make them more accessible to non-UVM-experts.

Complexity hiding is logical but is compromised by the diversity of sources for modern VIPs. Without a standard to align packaging methods, disconnects at the integration level are inevitable. In design assembly, the assembly problem has been significantly alleviated through the IP-XACT standard, defining a constrained and unified interface between components and the top-level assembly. Design and testbench structure have much in common, therefore IP-XACT should also be a good starting point to assemble testbench top levels.

Potential problems and solutions

One drawback is that there is no accepted standard today for packaging testbench components. Not that we don’t try. The in-house UVM team will develop components with interfaces to the in-house standard. Commercial VIP developers will each develop to their in-house standard. And there are legacy VIPs developed before any standard was considered. All well-intentioned, but this is a tower of Babel of interfaces. Some UVM teams go further, wrapping all VIPs in an interface following their protocol. A practical solution, though obviously, it would be better if we were all aiming at a standard and that redundant rework could be avoided. IP-XACT would be an excellent starting point, already well established for IP packaging.

A second potential problem is that IP-XACT has been defined for design, not testbenches. This is not nearly as big a problem as it might seem. Top levels should be mostly structural; IP-XACT already handles this very well through instantiations, ports, interfaces, connections, parametrization, and configurations. A couple of exceptions are class information exposed at the block level and SystemVerilog interfaces, both of which can be managed through vendor extensions. In the upcoming 2022 release, interfaces will be incorporated in the standard, leaving only class support to be handled in a later release, a goal which Arteris IP continues to push in the working group.

Interesting idea. What’s next?

Standardizing (and automating) testbench assembly is the only way to go to bring scalability to this task. UVM experts can work on building standard VIPs, leaving assembly (with much easier to understand scripting) to all those diverse teams in support of their needs.

Arteris IP has been developing a solution around this concept, with feedback from key customers. The result is Arteris Magillem UTG (UVM Testbench Generator). If you are intrigued and wonder if this approach could accelerate your SoC verification efforts, contact Arteris IP.

Also read:

Business Considerations in Traceability

Traceability and ISO 26262

Physically Aware SoC Assembly