“NoC, NoC” – Are You Listening to nVidia’s Dally?

Recently Bill Dally, nVidia’s Chief Scientist & SVP of Research, and a professor of electrical engineering and computer science at Stanford University, has been out speaking quite a bit including a “short keynote” at the Design Automation Conference and a keynote at ISC 2013. The DAC audience is primarily EDA tool users and EDA tool developers. ISC’s attendees are high performance computing (HPC) experts. While the DAC talk focused on designer productivity, the ISC talk honed in on the challenges of exascale system design. The topics, in fact, are highly interrelated.

As Dally pointed out in his presentation at DAC, the key to greater design productivity is to work at higher levels of abstraction. While this should seem obvious, it does run a bit counter to the behavior of many engineers. Most designers work hard to squeeze out every bit of power, performance, or area (PPA), depending on the importance of each design constraint for their end product. This often means starting with a proven piece of IP in an RTL form and then changing it to “perfect it” for their needs. That perfection takes time, not just on the editing, but on the subsequent implementation and verification efforts. It would be much faster if the designer used all the hardened IP they could find. While the design will not be quite as good on overall PPA it would be ready for tape-out many months sooner.

Of course, the same argument applies to getting designers to move up from RTL to higher level design languages such as SystemC. High-level synthesis (HLS) technology became available more than 10 years ago, yet the migration has been slow. Many designers have built a career out of RTL design – it is what they know, and it is where they see as their value-added to their employers. In the late 1980s and 1990s there were many IC custom layout engineers who felt the same way about their skills, but unless they were analog experts the number of jobs and the relative value of those jobs has not grown at the same rate as the overall semiconductor industry thanks to what is today are pervasive technologies, standard cell libraries and place and route tools.

But, according to Dally, moving up from RTL to HLS is not sufficient for getting the chip design time back to a couple weeks, nor is that enough for productive exascale design. To make that leap, Dally says to look at PCBs – fixed components you don’t modify. The challenges to use this approach in semiconductor are primarily twofold – having all of the IP you need available on the process you want, and having a standardized way of connecting them. The former problem is simply a business economics problem which I will leave for a later study. But the latter is simple a call for a standardized network on chip architecture, a NoC.

Dally’s ISC keynote seemed to conclude that the biggest challenge in exascale computing was not performance, but power management. For me, this is where it gets really interesting. Sonics has been working hard on putting power managementsupport into their NoC tools and architecture. After all, the NoC is aware of what data is being sent where and when which means it should know when to wake up or shut down whole system elements. Of course using a NoC is the best way to connect IP blocks. So, a NoC is a great tool to help the designer move up a level of abstraction and to reduce power – the two main points that Dally is making.

There is yet one remaining point I should make though. In modern chip designs using a NoC, the system architect picks which NoC features/options they want to implement. Even in standard protocols, there may be optional features. In chip design now, the RTL included in the NoC is only what is needed for the options that will be used, thus minimizing gate count. To fully move Dally’s PCB-like model, those feature need to always be present, yet programmable, not just in the NoC controller elements, but in the system block component interfaces as well. That means giving up some area for improved design productivity. I am not sure how many designers are ready to do that, or how much area they would really be giving up. In any event, modern system design requires a NoC, and one that can simplify chip connectivity and reduce power consumption, like Sonics’ on-chip networks can. This type of IP can help a lot in completing Bill Dally’s vision.