Closing Coverage in HLS

Coverage is a common metric with many manifestation. During the ‘90s, both fault and test coverage were mainstream DFT (Design For Testability) terminologies used to indicate the percentage of a design being observable or tested. Its pervasive use was then spilled over into other design segments such as code coverage, formal coverage, STA timing analysis coverage, etc.

Motivation of having a code coverage
While the term coverage may provide the management team with a sense of how much testing was done on a targeted code or design, to the code developers or system designers it offers an additional measure on how stable are their incepted codes –to be fully realized or used in the production mode. In software development domain, code coverage measures the percentage of source code being exercised during test using a collection of test suites –the underlying axiom here is that a high percentage value indicates a lower chance of undetected bugs.

The primary code coverage criteria include function coverage (measures frequency of a program function is called); statement coverage or line coverage (measures the number of statements that are executed); branch coverage (measures the amount of branches being executed, such as if-else constructs) and condition coverage or predicate coverage (measures whether the Boolean sub-expression has been assessed either true and false).

Although it is hard to achieve a full coverage and the relationship between an attained code coverage level versus probability of a program being bug-free is non-linear, code coverage data offers many benefits. This includes improving the completeness and robustness of the existing test suites (such as to guide generation of missing test cases, to minimize a test suite for runtime reduction, to guide fuzz testing, etc.) and enhancing regression sensitivity of a test suite.

SoC design, HLS and Catapult
Emerging applications such as 5G, automotive and IoT have introduced more embedded sensors, specialized processors and communication blocks such as visual processing, AI neural networks and high-speed communication units. In order to speed-up and lower the inception cost of such products, more system designers are migrating their code abstraction from RTL to high level synthesis (HLS) as shown in figure 1. The overall development time can be cut by half with higher level synthesis. While traditional software coverage tools can be run on the C++ source code, the generated results are inaccurate and may mislead designers for a good coverage as discussed in more details in the next paragraph.

Mentor’s Catapult® HLS Platform provides a complete C++/SystemC verification solution that interfaces with Questa® for RTL verification as shown in figure 2. The platform consists of a design checker (Catapult Design Checker or CDesign Checker), a coverage tool (Catapult Code Coverage or CCOV), a high-level synthesis (Catapult HLS) and a formal tool (Sequential Logic Equivalence Check).

To gain a better clarity of how these platform sub-components handshake, we should track the code as it is being processed. Initially, the code (containing Assert and Cover statements) is applied as input to CDesign Checker for linting and formal analysis to uncover any coding or language related bugs. A subsequent CCOV run is done on the checked code to provide hardware-aware coverage analysis. Once the code is cleanly analyzed, HLS can be performed to produce power-optimized and verification ready RTL. HLS propagates assertions and cover points to the output RTL and generates the infrastructure using SCVerify for RTL simulation using Questa –allowing the reuse of the original C++ testbench to verify the RTL. As a final step, SLEC-HLS formally verifies that the C++ exactly matches the generated RTL code.

Catapult vs traditional Code Coverage
Code coverage analysis in CCOV had been enhanced to be more hardware-context aware. The intent is to also enable designers to achieve the same coverage level for both high-level source code and post HLS RTL. The four main coverage techniques used to analyze the C++ code are statement, branch, focused expression, and toggle coverage. At first glance they look similar to those of the generic coverage tool version, but in actuality are not the same, as captured in the (left) table of figure 3. The right table shows differences between CCOV and traditional code coverage such as GCOV (Gnu Coverage).

Furthermore, mainstream software coverage tools are not hardware-context-aware in nature as highlighted below:

Using Catapult Code Coverage
A complete HLS to RTL coverage flow starts with stimulus preparation for HLS model in either C++ or SystemC. It is followed by running CCOV to assess if targeted coverage is met, otherwise more tests get added. Optionally, designer can exclude areas of the design from coverage analysis. CCOV captured all the coverage metrics into the UCDB and HLS is done to produce power-optimized RTL.

The generated coverage results captured in Questa Unified Coverage DataBase (UCDB), can later be used within the context of the proven Questa verification management tools as shown in figure 5. This CCOV integrated with the UCDB provides an assurance for a complete and accurate RTL coverage analysis.

If the Questa simulator is used with code coverage turned on, all coverage metrics are added into the UCDB. For those cases requiring further verification such as any unreachable code segment, designer can use Questa CoverCheck to help formally prove unreachability and apply the necessary directives to the simulator (or in UCDB) for its exclusion. Prior generated SCVerify settings and makefile can be used to simulate the RTL. The process gets iterated with more tests until RTL code coverage is reached.

In conclusion, migrating to HLS is a proven cost-savings and significantly shorten RTL development time. Mentor’s Catapult hardware-aware code coverage is a key metric in HLS flow and bridges coverage of higher abstraction codes (C++/SystemC) with RTL –enabling a fast convergence to verification sign-off.

For more details on Catapult HLS please check HERE.