How to Overcome HW Project Release Nightmares

Is a software development release methodology a “square peg in a round hole” when it comes to hardware design? To answer this question we have to look at how exactly hardware design projects differ from their software counterparts. Intuitively we know they are fundamentally different. Let’s take a second to dig deeper to understand why development methodologies for software do not map well into the hardware space, and then discuss solutions that do work.

HDL’s are a lot like the source code found in software projects, but downstream in the flow we encounter many binary files. These include custom layout files, place and route data bases, and models for simulation, etc. The presence of binary files breaks the ability to use diff and to merge two separate but parallel versions of a file. Really only one person at a time can work on a layout cell, for instance.

Hardware projects involve large teams that often are using or modifying the the same data. It’s common for sub-teams to have tens of members and for the entire project to include hundreds of members. These teams are arrayed over partitions of the design and they also cross functional boundaries, such as front end, simulation, verification, physical, physical verification, test, DFM, silicon engineering, etc.

When it comes to the number of files that need to be managed and the amount of data, we see tremendous numbers. A single workspace may consist of tens of thousands of files, or even more. Not just the number of files, but the total amount of data is enormous. It can head up into the tens or hundreds of gigabytes.

Lastly, the flow steps themselves can take days, or weeks. Place and route steps, verifying physical design rules, extraction for LVS or simulation, and many other steps easily can take days or longer.

All of these characteristics make it hard to overlay the popular branch/merge release flow onto physical design projects. In this approach each designer or developer creates a branch of the main trunk and can work on any or all of the files. It is isolated from the work of others, and down the road it will be merged back into the trunk. For software projects, files can be diff’ed and separate versions can be merged, based on the specific edits.

Because each team member has their own copy, it means that there are potentially many branches. In a hardware design team of 20 or 50 people, you can see that this can create a lot of branches, and eventually it will be some unlucky team member’s job to try to recombine them into something like a workable release. The lengthy work cycles mentioned above add to this by causing the branches to live long lives before any attempt to merge them back into the main trunk. This means that the baseline will have moved along significantly and increases the amount of data that needs to be reconciled.

Methodics has posted an interesting summary of best practices for data management used in hardware design projects. Their EDA focused data management solution is agnostic when it comes to the underlying revision control system that is used. This makes it easier for large enterprises to combine systems and achieve better integration. As a result of this a lot of people use Perforce based data management systems for their design data. The summary that Methodics provides talks about the alternative to the branch merge release process as used in Perforce. This recommended solution is referred to as Release on Trunk.

The Methodics white paper goes into more detail, but the main points are that each user creates a workspace based on a known good release. When they make changes they return them to the main trunk. They also keep current on releases that are updating the main trunk. Finally, when it is time for them to create a release based on their updates, they ensure they have the latest release updates in their workspace and run full regressions there before making the release.

Methodics also briefly also covers how to deal with the massive workspaces that are generated in hardware design flows. They point back to underlying capabilities in Perforce Helix and show several examples of ways they can be used to minimize the storage requirements.

The entire white paper is on their website and is interesting reading. It’s good to know that enterprise data management solutions can be applied to EDA design flows. But at the same time it is critical to understand where and how differences arise and how to adapt to them.