This blog is a follow-on piece to an earlier one titled NetApp’s ONTAP Enables Engineering Productivity Boost. If you have not had an opportunity to read that blog, please do so for context. Using real life examples, it showcases how customers could improve real-world design engineering productivity as much as 10%.
For the engineers within all of us, this blog will begin by explaining how the Snapshots and FlexClone features of ONTAP system work. It will then describe where you can immediately deploy this into your design flow for productivity boosts. Although EDA design flow is used for illustration purposes, this works well for software development workflows too.
As a rehash of the first blog, the premises was the following.
What If you could…
How would you reimagine your EDA and SW development workflows if your design DATA were Agile?
What Makes Instant Data Copies Possible?
NetApp’s ONTAP storage operating system has two key data acceleration features you could build into your design flow with simple RESTful API calls.
- Snapshots, which makes “Instant volume level READ-ONLY back-ups or copy” possible
- FlexClone, which makes “Instant volume level READ/WRITEABLE copy” possible
The ONTAP storage operating system takes simple API calls and operates on massive data sets without the data leaving the storage controller. The concept is similar to how graphics accelerators work. They take simple API calls from the CPU and manipulate gigantic numbers of pixels in the video framebuffer to create complex video images.
Snapshots can be derived from existing volumes of data to create Read-only copies. The snapshot names are specified through the respective API calls. The Figure below shows an active volume of data (Vol1) where each of block A, B, C, and D represents a file (or logical block of data). When a snapshot is created, the physical blocks of data are not copied, only the pointers to the data are copied. This makes instant data copies possible. An added value of this approach is the storage efficiency gained, as the data itself is not copied.
FlexClones can be derived from existing volumes of data to create fully READ/WRITEABLE copies. Clones are created just as snapshots are, by copying just the pointers to the data. Clones provide the same added value of storage efficiency as snapshots do. A clone takes up incremental storage only when modifications are made to the data.
That is “Data Accelerated”
Snapshots and clones are very fast and very storage efficient. The API calls make it very easy to create them and the operating system makes it very easy to use. And it typically takes seconds to either snapshot or clone 1GB, 1TB or 1PB of design data. To an application, both snapshots and clones look and behave just like a normal storage volume.
Using Snapshots and FlexClones in Your Design Flow
Let’s look at where and how in your design flow you can benefit by using snapshots and flexclones. We will divide the design phase into development, tapeout and post-tapeout ECO.
During Development Phase
Most chip design flows use modern Continuous Integration (CI) build processes. Design commits (code check-in) go through a CI smoke test verification process before the file is “integrated” into the branch or mainline code. Integrated code is run through larger Nightly build and test suite to further verify code integrity and quality. Then Release builds create Release Candidate builds until a final build meets all the feature quality goals and is ready for tapeout.
- Fast workspace creation
Engineers spend a considerable amount of time checking files out of source control tools like SOS or Perforce and then building views of the chip. As designs get bigger, the time taken to checkout and build a development workspace grows. To speed up this process, teams try different implementations.
NetApp’s ONTAP storage system allows you to snapshot each CI, Nightly and Release candidate build volume with a label. Dev and verification engineers can then create flexclones of the latest “Good” CI or Nightly build. Within seconds, they would have a personal copy of the source code for that build including all the build artifacts from that build. Build artifacts might include simulation views, synthesis .db/.gate files, log files, etc.
Hours of checkout and build time can be reduced to minutes.
- Faster bug fixes and design closure
If a bug is found in a CI, Nightly or Release candidate build, there is a lot of time overhead in both setting up the workspace before debugging and in committing the change.
Using ONTAP’s FlexClone function, an engineer can instantly create a clone of the failing CI, Nightly or Release candidate build for use in their debugging effort. Time taken to reproducing the bug, fixing it and achieving design closure is accelerated. This results in better utilization and efficiency of two of the most expensive resources (engineers and EDA licenses).
- Faster Continuous Integration (CI Builds)
Modern DevOps processes ensure design integrity by checking each and every design commit as soon as the code is committed. The processes do not wait until a Nightly build to test build quality. These modern DevOps practices have dramatically reduced the time to detect build errors and rework time. During busy commit periods right before a milestone build, critical changes can get stuck in a log jam of CI builds.
DevOps teams have tried to accelerate the CI process by various means such as building parallel CI build pipelines, checking out multiple baseline code directories and applying and testing multiple code commits in parallel.
ONTAP data acceleration can improve modern DevOps process flows.
Refer to the Figure below.
A FlexClone can be created from the snapshot of the last known good build, new code can then be integrated and retested. When a new baseline CI build is established, a new snapshot can be created, and the process repeated. This approach parallelizes the CI process and eliminates bottlenecks while shortening the time for code integration and retesting.
With API driven snapshot names, you can quickly and easily create checkpoints and restore points for your project throughout the development phase. This allows for increased innovation as engineers can freely try bold changes without worrying about main database corruption.
Tapeout
At the end of the development phase, the entire database is massive in size. It includes everything from the RTL stage to GDSII. It is standard practice to make a golden copy of this entire database at the time of backend release (tapeout). But how to ensure that the golden copy stays golden? How to prevent inadvertent modifications of files inside the final project directory?
Simply add an API call to your design flow to create a volume level snapshot with a name that speaks for itself. Example names for this snapshot could be “ProjA_final_v20210922” or “ProjA_tapeout_<build_id>”, where the build_id relates back to an ICManage/P4 integration ID.
During Post-Tapeout ECO Phase
What about an ECO modification post tapeout? When a post-tapeout ECO needs to happen, engineers are under more intense time pressure. You want to speed up the ECO fix/test cycle with the ability to easily rollback if a mistake creeped in during the bug fix.
Simply make a clone of the golden copy snapshot made at the time of tapeout. If the ECO is to fix a bug, you may decide to assign parallel teams to speed up the process. As the clones consume disk space only for modifications, you can create multiple clones, one for each team. Each team can test their changes in their own clone copy. Once tested, the changes can be merged back into the original volume.
When the ECO is released, simply apply a new snapshot name such as “ProjA_final_v20210922_ECO_A” that is self-descriptive. Now you have a golden copy of the ECO version and the original golden copy is still intact.
As you can see, use of snapshots and clones in an ECO flow can dramatically speedup iterations and enable rapid and easy rollbacks if and when needed.
Ready to transform your engineering environment?
If you are already using NetApp for your chip or software development, you already have what you need to get started. If you’re already familiar with how to make ONTAP system’s API calls, you can immediately start benefitting by implementing as described above. Otherwise, stay tuned for the next blog in this series.
The next blog will delve into the technical details of NetApp’s Open Source DataOps Toolkit. Gaining familiarity with this toolkit will shorten the learning curve for implementing Snapshots and FlexClones in your Design flow.
If you’re not currently using NetApp, you may want to seriously explore leveraging it, going forward.
Share this post via:
Intel – Everyone’s Favourite Second Source?