Python in Verification. Veriest MeetUp

Veriest held a recent meetup on a topic that has always made me curious – use of Python in verification. The event, moderated by Dusica Glisic (technical marketing manager at Veriest), started with an intro from Moshe Zalcberg (CEO of Veriest) and talks by Avidan Efody (Apple verification) and Tamás Kállay (Team leader, Veriest). I know Moshe is a fan of this concept as an example of extending gains in SW development to the HW world. This meetup digs deeper into Python in Verification.

Flows and stupid verification tasks

Avidan has background as a verification expert from Amazon, Intel and Apple, which makes him a serious authority in my view. He was careful to stress that none of what he talked here should be interpreted as methodologies at his current employer. Here, he was simply synthesizing his know-how gained over many years in using Python in his day-to-day verification activities. He also stressed that he is a verification expert using Python, not a Python expert drafted into verification.

This talk was an excellent introduction to “Why Python?” in verification. Consider Python’s assets. Many of us, not just in hardware design, already know and use the language., Python supports version control systems access and readers and writers for virtually any formats. It has increasing support from EDA companies and is already used in many production CAD flows. It has support for databases, CI/CD flows, etc etc. and is widely understood and supported for questions on e.g. StackOverflow.

From an applications point of view, Avidan cited 5 (the last one a stretch). First for building production flows such as tool wrappers and regression runners. Second for what he called stupid verification tasks: connectivity checks, clock/power gating, register checks. He made the point that tests of this type require design knowledge and spreadsheets but really don’t need SystemVerilog testbenches or randomization. Python can drive all of this. He pointed to the fact that Python can read RTL directly. There is a nice package called cocotb for testbenches, good enough for these purposes. And Python can read waveform files.

Python for designers who hate the verification team!

I really liked his next point – “Python for designers who hate us”. His point here is that a couple of decades ago, designers were doing verification themselves but stopped because verification split off into a separate team and became very complicated. Designers stopped verifying, not because they wanted to but because the whole process with UVM etc became too complicated and too slow to respond. Python provides them a way to return to unit/block testing, again using coctb etc, without having to wait on the verification team.

Avidan mentioned using Python to boost UVM flows by isolating stuff that can change quickly – sequences, configuration, assertion, checkers, etc, minimizing recompile requirements. The final application he mentioned is the “one language to rule them all” concept – that Python could replace UVM. He’s not a believer but he does know smart people who are pushing this direction 😎.

Developing bringup tests

Tamás described another interesting application – developing bringup tests before silicon arrives. In this context he needs to be able to support multiple platforms such as simulation, emulation, FPGA prototyping and of course silicon when available. What is important here is a unified development interface, supporting communication over standard hardware interfaces such as PCIe, JTAG and UART. In the early stages of development this supports development and debug of the tests, and later I would guess in support of post-silicon debug.

UVM obviously plays a role in test development but needs to sit under a superstructure which can span all these platforms. And which especially will work equally well with first silicon. For this reason the team built a client-server structure in which the servers are the various simulation platforms or silicon. These communicate through sockets with a client written in Python and running Python tests. The rationale for using Python was that the low-level SW team were already using Python to write tests in pytest. Also they found that many HW engineers already have at least some Python expertise. Which made adoption quite painless across both teams.

Tamás includes more detail on how they architected this system. He wrapped up by saying the approach worked well for them with some limitations. Perhaps not surprising for an in-house development to serve a custom purpose.

My takeaway

A few dreamers aside, production verification engineers are not aiming to replace UVM with Python. There will always be many clever things that UVM can do that Python cannot (easily). The purpose of Python development and usage around verification is to plug the holes in mainstream verification methodologies. For stupid tests and to support designers running their own verification. To speed up standard verification flows and to support silicon bringup test development. Could you do all that in standard UVM (or PSS) flows? Perhaps as an exercise, but would it have the flexibility of Python for these often-custom applications. With minimal learning across diverse HW and SW teams? That would be a stretch too, I think.

You can watch the meetup replay HERE.

Also read:

5 Talks on RISC-V

Ramping Up Software Ideas for Hardware Design

Verification Completion: When is Enough Enough?  Part II