System VIPs are to PSS as Apps are to Formal

In the formal world the core technology is extremely powerful, and specialist users need full access to tackle difficult problems. But for many applications, teams prefer canned solutions built on the core technology yet scalable to non-experts. A similar dynamic appears to be playing out between System VIPs and PSS. PSS, the system level simulation standard, is an extremely powerful approach to define portable system level tests. Powerful but different from the more familiar UVM standard for lower-level testing, which can be a speed bump for adoption. Dave Kelf (CEO of Breker Systems) told me that his customers have been telling him that while they like Breker’s PSS tests, they are looking for more canned system VIP solutions, for example around coherency testing. Hence my observation in the title. PSS technologies are still essential as the core, but teams now want more scalability through complete out-of-the-box system VIP solutions for standard system verification needs. While still being able to develop their own specialized tests as needed.

What is a System VIP?

The intent is the same as for an interface IP (USB, etc) – a method to generate traffic to drive design testing, along with checkers and scoreboards to check and monitor interaction between the VIP and the design. However rather than checking protocol compliance, System VIP checks system level behaviors such as cache coherence and interoperability between cores/clusters and the rest of the system.

There are some other important differences. An interface IP acts as a proxy for a design block or an external source. System VIPs by their nature don’t have a design equivalent. They act instead as test overseers for their area of interest. A coherency system VIP is a good example. This will at minimum generate traffic for end-point memory requests on a coherent network, checking and monitoring some level of cache controller and directory behavior, and compliance with the coherence protocol (e.g. MOESI). They probably also should provide some kind of scoreboard to accumulate coverage stats against target objectives.

The basic minimum for System VIP

As a relatively new field, an obvious question is “what should such VIPs provide at minimum?” Breker has developed a top-10 list of requirements over and above the basic VIP expectations, which I find quite useful. I’ll start with a zeroth requirement, implied for their list yet fundamental in PSS implementations.

System level tests scale up very fast. Unit tests (for a mesh network say) may run on a simulator but larger tests demand emulation or prototyping together with a virtual interface for the testbench, then virtual models to run software loads and interfaces to interact with other simulation platforms such as a network traffic simulator. The bigger the design gets (e.g. multi-die) the more varied and distributed the simulation platform becomes, each aspect bringing its own constraints to the test flow. PSS is designed to scale across these diverse modeling systems which is one of the reasons System VIPs are built on PSS.

The first 5 must-have’s start with a no-brainer: targeting system level operational scenarios rather than functional components. Of course, you need a configurable model for the VIP, scalable between different system designs and allowing for reuse. The VIP should be built on a detailed understanding of the domain, in the coherence case all the possible options and protocols, abstracting away detailed implementation choices. It should be scalable from unit (system) tests up to multi-die systems and it should support portability between simulation platforms so if I can abstract some part of the DUT in a virtual model if I want, or use a more accurate RTL model running on an emulator. All still under control of the same System VIP.

Dave and Adnan Hamid (Executive President and CTO) see the next 5 must-haves as areas where Breker offers differentiation. One is the ability to intermix verification objectives, so you can combine coherency checking with power management variations for example. A second is the ability to run multiple scenarios concurrently for stress testing. A System VIP should have self-checking tests and a scoreboard because that’s the kind of complexity hiding users want to see in System VIP. Breker also argue that System VIPs should be user extensible. And they say that these tests should support firmware and UVM code components.

Possibly some more must-haves could be added, but this list looks like a good starting point.

The Breker System VIP portfolio

Cache System Coherency is an obvious VIP since Breker have been working in this domain for a long time. Arm SoCReady provides testing against the hardware sections of Arm’s SystemReady certification, verifying compliance against the system-level hardware requirements. RISC-V SoC Ready provides a similar service design based on RISC-V cores. RISC-V CoreAssurance is an interesting complement for teams building RISC-V cores, who want to get a general assurance that the core will be compliant with a RISC-V ready checkout without having to build a surrounding SoC – or rather many such SoCs to get better coverage.

Breker also provides System VIP class solutions in a number of other domains, listed in the graphic above. I didn’t talk about these with Dave and Adnan but I’m sure they would be happy to share more details. You can learn more HERE.

Also Read:

Qualitative Shift in RISC-V Targets Raises Verification Bar

Breker’s Maheen Hamid Believes Shared Vision Unifying Factor for Business Success

Scaling the RISC-V Verification Stack