LLM-assisted schematic test-report analysis and root-cause investigation

By Piyush Bag

Intro: In this AI era, building data centers has become crucial not just for hyperscalers, but for many companies as well, as they build the infrastructure for AI. At a high level, a data center includes compute, storage, networking, and other infrastructure. This article is about the design lifecycle of network hardware.

Definition: What is a schematic test report generator?

Broadly, when a data center network switch is manufactured, designing the PCB board is a crucial part of the initial phase of the hardware development lifecycle, as it determines which components are required and compatible with customer requirements. The Hardware Design Engineer designs the schematic diagram in CAD software like Cadence Allegro. The schematic is a symbolic connectivity model showing components, nets, pins, ports, hierarchy, and design relationships. The schematic design database is maintained in its native format and can be exported as a PDF for review. The BOM and netlist are stored separately. To verify the schematic diagrams are correct, they run a set of topology tests to ensure basic to advanced design configurations are done correctly. In organizations where git practice is not followed in the hardware development lifecycle and work iteratively, they tend to rely on manually running the topology test, aggregating the log results from the CLI, then viewing the errors, consolidating and manually triaging the errors in a Google Doc with log links and headers only, and placing this doc in the drive for the team to pick up. This process continues until the hardware design engineers ensure all topology tests have passed based on their schematic design. This process matters because it saves a lot of money in the early phase before manufacturing, since errors are caught before production, but it also slows the development lifecycle.

Problem: Why was the schematic test report generator not enough?

In this iterative approach to designing schematic diagrams, running topology tests works fine until the scale is small. Still, as the organization grew, faster iteration alone wasn’t enough, as it could lead to more errors and more back-and-forth in the design phase of the cycle, compared with the hardware requirement document. Another problem was manually aggregating logs from various databases, grouping failures, and matching errors to schematic diagrams in PDF format stored in a centralized repository like Mercury or Perforce; then pasting one error result at a time into a document, formatting it properly, and filling in comments by the hardware design engineer. This solution takes a lot of time, and at this scale it’s hard to iterate quickly as design requirements get bigger and more complex.

Solution: Framework to write evals for LLM-based schematic test report analyzer

One managerial solution is to pivot the development approach rather than follow a classic waterfall/iterative hardware development model, and to stay agile / more agentic with modern automation. However, changing the entire development lifecycle is difficult for organizations that have operated in a specific way for years. However, being AI-native from day one is not the only way for hardware engineering to speed up development; integrating and automating with AI plays an important role in such organizations.

The existing SRG collects schematic revisions, topology test results, job information, raw logs, nets, pins, and component references, and feeds them to an LLM that analyzes and helps explain and organize grouped topology-test results. This information is normalized into structured context and examples that guide the analyzer on what to group and how, written in Markdown with examples and exceptions.

Step two in the system is to create a separate context for every failure group, like “missing connection on the VOUT net”, “unconnected power pin”, “repeated topology mismatch” ie. several components such as U29, U57, and U33 may fail the same connectivity rule because they share a common control signal or expected circuit structure. The best part of the LLM-based analyzer is that it explains the result and recommends how to fix the root cause; for example, the engineer might need to inspect the schematic connection, verify the component library, or rerun a specific topology test. To ensure the LLM output is correct, it is compared with a golden dataset of previously reviewed schematic test cases in JSON format.

Conclusion: An LLM-assisted schematic report generator can help hardware design teams reduce the manual effort involved in reviewing topology test results. It can organize information from the schematic revision, test logs, and component references and convert it into a report that is easier for the engineer to use, increasing designer productivity. The hardware design engineer still needs to make the final decision about the schematic design and the electrical root cause after reviewing the original schematic and test results. With LLM analysis as just phase one, the process becomes more agentic if traditional CAD tools start providing official MCPs to take actions in the tool based on topology test results; just as software engineers are “vibing,” hardware design engineers will soon “vibe design and test” too.

Piyush Bag is a software engineer who builds test automation and software infrastructure for hardware validation, diagnostics, and manufacturing tests. His work spans electronic design validation tooling, platform software, and data-center switch systems.

Also Read:

Agentic AI Is Reshaping Semiconductor Design

Webinar – ChipAgents Shows You How to Break the Timing Closure Bottleneck with AI

Podcast EP366: How Cognichip is Changing Chip Design with Faraj Aalaei

Share this post via:

• 
• 
• 
•