Leveraging AI to help build AI SOCs

When I first started working in the semiconductor industry back in 1982, I realized that there was a race going on between the complexity of the system being designed and the capabilities of the technology in the tools and systems used to design them. The technology used to design the next generation of hardware was always lagging behind while it was being used to build generationally larger and more complex systems. I liken it to a dragon chasing its own tail. Designers have always really wished they had the next generation computing power available to design the next generation of hardware.

The situation has been this way ever since those days so long ago. However, perhaps the advent of Artificial Intelligence may change that dynamic. AI has an uncanny ability to solve complex problems that cannot addressed by more processors, more memory and more networking. It represents a fundamentally different way of solving problems that have large numbers of variables and complex performance surfaces.

It’s not surprising then to see machine learning making its way into the software and tools used to design SOCs and complex systems. The endgame of this is using machine learning to design machine learning systems. There you have it, AI inception.

One of the most complex and non-deterministic problems in SOC design is interconnect. Long ago the ability of hardwired interconnect to keep up has slipped away. As a result, the application of Network on Chip (NoC) for interconnecting blocks has become more prevalent in SOC designs. Still, even with top down, requirement driven tools for designing NoC structures, there are great challenges in designing efficient NoC interconnect implementations. These days NoCs have routers and they dynamically manage traffic.

I recently had a chance to talk to Anush Mohandass, VP of Business Development at Netspeed, a leading provider of NoC IP and development tools. We talked about their announcement of Orion AI which is NoC technology that now incorporates machine learning algorithms. Right off the bat he pointed out that the challenges of designing AI chips has led to dramatically shifting requirements for SOC design. AI chips have significantly different interconnect needs. They have a large number of computing elements with their own local memory stores that are connected in a largely flat topology. No longer does data move to and from central memory to be processed by a central processor. This is a peer to peer system that requires low latency, high bandwidth and incredible flexibility.

The NoC for AI must support multicast and broadcast data transfers. It is also needs to have non-posted and posted transactions. Comprehensive QoS is also necessary and it must be non-blocking. In effect AI applications require software defined NoCs. Netspeed accomplishes this with a multi-layered protocol that created levels of abstraction between the physical and functional implementations.

Netspeed ‘s Orion AI is a leap forward in NoC technology. It offers scalable data widths that are significantly larger than its predecessor. It can operate at speeds of 2-3GHz with bus widths of 1024 bits. It can support the interconnection of thousands of elements. The AI algorithms built into Orion AI efficiently optimize the final implementation. This naturally means that it is the ideal technology to implement AI systems.

Maybe we haven’t reached the level of robots building robots, but we definitely have reached the age of using AI to help build SOCs. Netspeed’s Orion AI is an excellent example of how this technology can be applied. For more detailed information about Netspeed’s Orion AI visit their website.