Bronco Debug Stress Tested Measures Up

I wrote last year about a company called Bronco, offering an agentic approach to one of the hottest areas in verification – root-cause debug. I find Bronco especially interesting because their approach to agentic is different than most. Still based on LLMs of course but emphasizing playbooks of DV wisdom for knowledge capture versus other learning methods. Last year, setup required playbooks developed in a collaborative approach between DV experts and the Bronco debug agent. Unsurprisingly as a young company in a fast-moving field, learning methods have evolved considerably to the point that the debug agent can now build its own reusable playbook to cover a substantial range of bugs. Proprietary expert DV refinement can then be added on top. David Zhi LuoZhang (CEO, Bronco) suggests that this expert step is a 10–15-minute task, hardly challenging. He adds that now generated playbooks and memories from debug runs are combined with specs and user-provided documentation. Together this institutional knowledge coalesces in a customer-specific library he calls the Bronco Library (to be covered in a later blog perhaps).

How well does it work?

An industrial strength test

David can’t share a name for a recent successful eval but he can say it is a major public company who evaluated on a big SoC. Lots of sensors, suggesting probably several modalities of AI processing plus the usual multi-core CPU, memory management, etc, etc. Also sounds like it may be safety critical. The eval task was to find a significant number of known bugs discovered during active development on a design. Also important, these were SoC-level bugs, the hardest to root-cause. Examples they cite include performance problems in AI accelerators, deadlocks in power mode transitions, nasty UVM race conditions and assertion-firing issues.

Bronco was able to isolate an exact root-cause location for about 50% of those bugs without help, some in UVM testbenches, some in the design. Just by looking at the RTL, UVM testbench, design spec and playbooks. For another 25% the agent was able to localize the root cause to a file (out of 10k files). After initial setup, analysis was hands-free and completed in 15 minutes. A human debugger would surely have taken hours if not days to get to similar closure. That’s a pretty significant advance on reducing the largest overhead (debug) in DV.

Integration with regression flows

AI-based methods can have challenges integrating into regression flows. Apparently, Bronco have found a way to coexist very neatly with these flows. Their debugger can be instrumented into an overnight regression, triggering whenever a failure is found., launching an investigation of each failure in turn. For each, the debug agent creates a ticket, showing not only the root problem but also the steps that lead to this diagnosis. The tcket is then ready for DV to review in the morning. When an engineer analyzes a ticket the following morning, they can submit it to Jira directly if they choose.

On that note a sample ticket is worth a look, just to understand the detailed analysis this debug agent is able to generate:

This example is based on bug analysis on a block rather than one of the SoC bugs to protect proprietary details. Nevertheless, the quality of analysis suggested by this ticket is pretty impressive. If this example is representative for even 50% of the bugs exposed in a regression, I imagine debug technologies like this are going to take off fast.

What if the debug agent delivers an incorrect analysis or doesn’t get close enough in isolating a root-cause? Working through the list of tickets, in such a case the DV engineer can provide extra guidance, and/or suggest the debugger go deeper. That re-analysis continues in the background, so that well before the engineer has finished checking subsequent tickets, updates are ready for re-analysis.

Big step forward in agentic debug. You can learn a bit more about Bronco HERE.

Also Read:

Verification Futures with Bronco AI Agents for DV Debug

Superhuman AI for Design Verification, Delivered at Scale

AI RTL Generation versus AI RTL Verification