Many moons ago in the Innovation series we explored techniques like spectrum analysis to root-cause bugs. While these methods provide some value they don’t get as close as we would like to isolating a root-cause. In hindsight given what we know about the complexity of conventional debug it is unsurprising that we can’t root-cause in one shot. Hence the rise of agentic debug solutions from companies like ChipAgents and ChipStack. Agentic systems can reason through a root cause analysis in multiple steps just as we do in human-based analysis. Following is a very intriguing parallel from our sister field (software debug) posted as a YouTube session from the C++ conference.
(Image courtesy of Cppcon)
Background and bugs
This event was a joint presentation between UnDo.io (who provide time-travel debugging for C++ and Java, think something like gdb with full context replay) and Anthropic. Their goal was to explore live (not a canned demo) what agentic debugging would look like. Gutsy move because the reality was messy though still very informative. They test on a couple of cases in parallel: A segfault in the Python interpreter and unexpected behaviors (which prove not to be bugs) in Doom.
The Python bug should attract the interest of hardware designers: effectively a cache coherency issue in software. The code caches pointers to objects allocated in memory and entries in the cache can be tested without incrementing reference counts for those objects. The coherency risk is that a referenced object may be freed without clearing the cache reference, a worthy test for the value of agentic debugging. The Doom exploration is primarily interesting for how it influences the debugging process in localizing a behavior within a playback to get close to whatever triggered that behavior. This case may be even more interesting for hardware debug, where unexpected behavior is much more likely than anything comparable to a crash.
My takeaways from the demo
The Python debug demo is, as far as I can tell, hands-free apart from the initial setup. Analysis starts with the crash and iterates backwards and between multiple types of agents, trying different hypotheses, testing with different techniques to eliminate possibilities. UnDo added an adversarial “bug diagnosis validator” agent (Claude Code) provides support for this). As agentic analysis progresses, discoveries start to converge towards the right area ultimately getting get pretty darn close to the root cause.
As expected, Claude builds a ToDo list of tasks it believes it needs to perform to work towards a goal (e.g. find when the second zombie was killed in the Doom debug, see below), and checks these off as it progresses. An interesting revelation is that it apparently can lose the plot periodically, at which point it needs to be reminded to revisit the list. This didn’t seem to happen in this Python case.
The Doom analysis is more collaborative, I imagine because they don’t have a bug to target. Instead, they are trying to understand unexpected”behaviors. For example, why did the player get stuck in the map room after killing the second Zombie? Here the demo guy asked, “when was the second zombie killed during this playthrough (recorded playback)?” Claude got him to this point, from which he could ask it to drill down further. Note the value of being able to use a high level reference (second zombie) in prompting next steps.
The demo often ran into system problems (“repeated server overload with “Opus model” – Opus is Claude’s model optimized for coding) which seem to reflect server busy problems on the Claude side. These issues are now apparently fixed (or at least improved) – this demo was running with a pre-release of the Claude Code API.
There was question from the audience about token costs. The UnDo speaker suggested single-digit dollars for the Doom example (56k LOC), much higher costs ($$ numbers not cited) to track down the Python interpreter bug mentioned earlier (350k C LOC, 800k Python LOC).
Long video (about an hour) but well worth watching all the way through for the insights it provides. You can find the video HERE.
Also Read:
Why PDF Solutions Is Positioning Itself at the Center of the Semiconductor Ecosystem
Gate-All-Around (GAA) Technology for Sustainable AI
Beyond Transformers. Physics-Centric Machine Learning for Analog
Share this post via:
The Risk of Not Optimizing Clock Power