Scaling the RISC-V Verification Stack

The RISC-V open ISA premise was clearly a good bet. It’s taking off everywhere, however verification is still a challenge. As an alternative to Arm, the architecture and functionality from multiple IP providers looks very competitive, but how do RISC-V providers and users ensure the same level of confidence we have in Arm? Arm run 10¹⁵ cycles of verification per core, with years of experience baked into core and system level regression suites. Equally problematic is verifying the custom instructions you can add in RISC-V. How can a core or system builder measure up to the Arm-level of confidence? A part of the answer, according to Breker, is much higher levels of automation. Makes sense, but what can be automated? Breker start with a verification stack, with layers from early liveness testing in IPs all the way up to system level performance and power profiling.

Core Verification – Part I

Maybe you don’t think you need help testing a RISC-V core, but if you’re just starting out with this architecture, or you’d like to accelerate test suite development (test suite generation is the biggest time hog in the 2022 Wilson survey on verification), or you’d just like to add an independently generated suite of tests to make sure you covered all the bases, Breker’s FASTApps might be worth a look.

Remember how the Breker technology works. Building on one or more PSS (or UVM)-compliant test scenario models, the technology generates usage-centric graphs, then automatically builds a suite of test cases as paths traced through those graphs. These include stimulus, coverage models and expected results. Scenarios can be single-threaded or multi-threaded, even on a single core. The Apps are a layer over this fundamental test synthesis capability. These build important tests for load store integrity, random instruction testing, register-to-register hazards, conditionals and branches, exceptions, asynchronous interrupts, privilege level switching, core security, exception testing (memory protection and machine-code integrity), virtual memory/paging and core coherency.

A noteworthy point here is that custom instructions added to the core become new nodes in these graphs. When you synthesize test suites, custom instructions are added naturally to test suites during scenario development. They will be covered as comprehensively as any other instruction, to the extent possible given the nature of those instructions.

Tests developed through the Breker technology are portable across simulation, emulation, and prototyping platforms and from IP verification to system level verification, maximizing value in the overall test plan. They even have a UVM handle for those allergic to PSS 😊.

SoC Verification

The same approach can be extended to system-level verification apps, here the upper 3 levels of the stack. Breker is already well-known for their dynamic coherency verification, a major consideration at the system level. To this they have added dynamic power management checking. Think of the state machine for a power domain controlling startup and shut down. That can be mapped to a graph, then graphs for each such domain can be squashed together, allowing test synthesis to explore all kinds of interesting scenarios across power switching.

For security verification, scenarios can be defined to test that access to different parts of the memory map. At a more detailed level, test suites can cover system level interrupts, typically even more layered, complex and asynchronous than at the IP level. More possibilities, tests for atomic instructions, in the presence of interrupts for example. System memory virtualization and paging tests. And so on.

What about performance bottlenecks, in the network, in interrupt handling, in external memory accesses, in all the places performance can be dragged down? The best way to attack this problem is by running a lot of synthetic tests concurrently. Like all those tests Breker built for you. That’s a great way to increase confidence in your coverage.

Core Verification – Part II

Core developers know how to deliver coverage for the first 4 levels in the stack, but how do they test system-level correctness, the upper 3 levels? Improving coverage here is just as important. Arm especially has excelled at delivering high confidence for integration integrity. If your RISC-V is being developed in parallel with a system application and use cases, obviously that system will be at least one of your platforms. If you don’t yet have a system target or you want to extend system testing further, you might consider the OpenPiton framework from Princeton. This is a framework to build many-core platforms, offering an excellent stress test for RISC-V system verification.

Running system integrity tests against a core isn’t overkill. I attended a talk recently touching on issues found in system software. Software that has been proven to work correctly on a virtual model of hardware often uncovers bugs when run on an emulated model. A significant number of those bugs are attributable to spec ambiguity, where the virtual model developer and the hardware developer made seemingly harmless yet different decisions.  Difficult to point a finger at who was at fault, but either way expensive disconnects emerge late in design. The Breker solution also allows firmware to be executed early in the verification process, on designs where the processor has not yet been incorporated. You might not catch all these problems in the upper 3 layers of the verification stack, but robust system testing will catch more than you would otherwise.

Worth it?

I really like the approach. No verification technology can do more than dent the infinite verification task, but some can do it more intelligently than others, especially at the system level. Breker provides an intuitively reasonable way to scale the RISC-V verification stack with meaningful coverage. Some blue-chip and smaller customers of Breker also seem to think so. Among customers willing to be cited, Breker mentions both the GM of the SiFive IP business unit and the CEO of Nuclei technology (a RISC-V IP and solution provider). You can learn more about Breker FASTApps HERE.