Moving up Verification to Scenario Driven Methodology

Verification complexity and volume has always been on the rise, taking significant amount of time, human, and compute resources. There are multiple techniques such as simulation, emulation, FPGA prototyping, formal verification, post-silicon testing, and so on which gain prominence in different situations and at different stages of SoC design. Testbench and test creation is one of the major tasks of verification; it’s not possible to do it manually for the kind of SoCs we have today. The advent of HVLs such as VERA, e, SystemVerilog, and others, and standard verification methodologies such as UVM (Universal Verification Methodology) eased the tasks of test generation and coverage closure to a large extent by enabling automated procedures for verification tasks. However, UVM is suitable at IP level and is targeted at RTL simulation. When the same IP gets into an SoC, it needs to be re-verified, along with other IPs, multiple processor cores, custom blocks, and the overall design in the context of applications running on the SoC. The UVM no longer remains a suitable methodology at the SoC or system level. So, what’s the alternative?

The case of verification at system-level really gets complicated because at that level it needs not only simulation, but also hardware emulation, virtual prototyping, and FPGA prototyping. Also, an SoC can have lot of embedded software in it which does not provide automated stimulus generation like SystemVerilog in case of IPs. This, at the system level, leaves us in a state where there are large, unbounded cases to be verified under different scenarios without any standard, automated, and reusable test methodology. The situation is alarming and EDA companies are already working towards this paradigm shift in verification at the system-level, of course the IP-level verification stays well with UVM and SystemVerilog. However, there is an early need for a common standard to provide interoperability between different solutions at different levels, vertically (i.e. IP, sub-system, SoC and System levels) as well as horizontally (i.e. different engines such as simulation, emulation, virtual prototyping, and so on).

Towards standardizing verification methodology at system-level, the way UVM has worked at IP-level, this year Accelleraannounced the formation of Portable Stimulus Working Group (PSWG) with participation from the EDA and semiconductor industry. The charter of this group is to develop a Portable Test and Stimulus Standard which will define a specification to permit the creation of a single representation, usable by a variety of users across different levels of integration under different configurations, enabling the generation of different implementations that run on a variety of execution platforms, including, but not necessarily limited to, simulation, emulation, FPGA prototyping and post-silicon.

This week, three EDA companies, Cadence, Mentor, and Breker, who have significantly worked in this area and have good verification solutions at the system-level jointly announced about their collaborative technology contribution to PSWG. I got a chance to speak with Frank Schirrmeister, Sr. Group Director, System and Verification Group at Cadence. It was a great opportunity to know about how the competitors are collaborating (while remaining at the competitive edge) for this novel, industry level initiative. Before getting into that, for audience’s ease of understanding, let me provide a glimpse of what I learnt about Cadence’s solution for automated verification of use-cases at system-level.

By using Cadence’s Perspec System Verifier flow, system level actions can be specified graphically by using UML (Unified Modeling Language) representation, abstract testcases generated and mapped to different verification platforms, and the test results debugged in Incisive Debug Analyzer. The use-case testing thus defined is software driven as it applies tests through ‘C’ code running on embedded processor models. The ‘C’ code tests are portable vertically at all levels, horizontally at different platforms, and for different use-cases’ reuse. The Perspec System Verifier uses SLN (System-Level Notation), an object oriented language for defining scenarios in the abstract model of a use-case.

The SoC design houses like STMicroelectronicsand Samsungare already using the Perspec System Verifier solution. ST has been using it since long; ST’s technical paper in this year’s DVConconference stated that they used this approach at sub-system level and improved the test generation productivity by 20x with lesser code written by the test engineers. The methodology applied at the SoC level can definitely provide an order of magnitude higher productivity. ST also presented in this year’s DAC about their methodology of verifying complex system scenarios by using Perspec System Verifier where they illustrated the methodology an example of combining a PCIe IP and a CPU models to generate IO coherency tests.

A more recent update from CDNLive at Bangalore last month is that Samsung is using the Perspec System Verifier to generate testcases that run several billions of cycles each week. They concluded that the portable stimuli generated through this methodology are language agnostic and can be reused in all environments from pre-silicon to post-silicon.

Also, Mentor has inFact verification tool for testbench automation solution and Breker has TrekSoC and TrekSoC-Si products for such software-driven verification platforms at the system-level. Also, there may be other companies working towards this new initiative; I heard of Vayavya Labs as well.

Coming back to the collaboration part; how Cadence, Mentor, and Breker can contribute and compete. They can come up with a standard language for specifying use-case scenarios at abstract level that can include tests, coverage, and result checking. To comply with this standard language, they will have to do some tweaking in their tools. This standard should allow multiple implementations for different tool environments with consistent behaviour at simulation, emulation, prototyping and post-silicon verification. By doing so they can differentiate at their tool level and at the same time be interoperable through the common standard for stimulus and test scenarios.

This initiative will bring a major shift in paradigm for SoC verification at the system-level through use-cases and test scenarios. This is the right time an industry wide standard for Portable Stimulus is established for system-level scenario-based testing. The standard can provide a major boost in SoC verification productivity through test automation, horizontal and vertical interoperability, and reuse. The joint contribution by Cadence, Mentor and Breker will accelerate the establishment of this standard.

Read these for more details:
The Press Release is HERE
ST’s DAC 2015 presentation is HERE
Frank’s blog after CDNLive Bangalore is HERE; Samsung’s reference about Perspec is in this.
A few of Frank’s other blogs are HEREand HERE
Richard Goering’s Q/A with Cadence Fellow, Mike Stellfox is HERE

Pawan Kumar Fangaria
Founder & President at www.fangarias.com