2024 Outlook with Steve Roddy of Quadric

Quadric Inc. is the leading licensor of general-purpose neural processor IP (GPNPU) that runs both machine learning inference workloads and classic DSP and control algorithms.  Quadric’s unified hardware and software architecture is optimized for on-device ML inference. I have know Steve Roddy for many years, he is a high standard in the IP business.

Tell us a little bit about yourself and your company.
Quadric is a startup processor IP licensing company delivering a unique general-purpose, programmable neural processor (GPNPU) IP solution. In a marketplace with more than a dozen machine learning “accelerators” ours is the only NPU solution that is fully C++ programmable that can run any and every AI/ML graph without the need for any fallback to a host CPU or DSP. With more than 25 years of marketing and management experience in the IP business, I lead the marketing and product management teams at Quadric.

What was the most exciting high point of 2023 for your company?
2023 was exciting for Quadric because it marked the debut of our first licensable IP product in May 2023 – both first production RTL deliveries of the Chimera GPNPU and the launch of our online Quadric DevStudio. In the seven months since we’ve been expanding or our sales & FAE team around the world. 2023 was an eventful and successful year, indeed.

What was the biggest challenge your company faced in 2023?
The biggest “news” in 2023 in the market for NPUs/GPNPUs was the dramatic upsurge in interest in Large Language Models (LLMs) in devices, rather than running purely in the cloud. Whether it is the rise of the so-called “AI PC” or the embedded of LLM-based voice assistants in countless end products, the surge in user demand for transformer-based LLMs dramatically impacted the NPU IP market. Many existing NPUs could not efficiently run LLMs, putting stress on the silicon ecosystem.

How is your company’s work addressing this biggest challenge?
Unique among NPU vendor offerings, the Quadric Chimera GPNPU is 100% programmable. As a result, we tackled the on-device LLM wave by demonstrating the Llama-2 LLM less than 5 weeks after it was published. Meanwhile our competitors were announcing new cores that won’t be available until mid-2024 (or later). The rate of change of LLMs has only accelerated since mid-2023 with a myriad of new language model type and topologies. This rapid pace of change demands flexible hardware that can run as-yet not invented machine learning models.

What do you think the biggest growth area for 2024 will be, and why?
The ML inference market that we serve is rapidly changing. The dominant ML algorithm styles of three and four years ago were classic convolution-based CNNs, such as the Resnet and MobileNet and Yolo families of networks. Today, newer structures leveraging transformer topologies – such as LLMs and ViT models – are rapidly displacing the older CNNs. That is turn is causing silicon design teams to respin older devices to be positioned to support these newer algorithms in the coming years.

How is your company’s work addressing this growth?
Quadric is continuously adding ports of new algorithms to our processors. Adding demonstration of a new ML model is a pure software effort for us, and we are focused in 2024 on widening the array of models further with each periodic software release. Today we support all the major modalities of ML inference, including a variety of leading-edge transformers.

What conferences did you attend in 2023 and how was the traffic?
In 2023 we attended smaller, focused technical conferences (Embedded Vision Summit, DAC, Design Solution Forum). We avoided the bigger mass-gathering shows in 2023 (CES, MWC, Embedded World) because those shows were not all the way “back” to pre-pandemic attendance levels. However, I did go to CES 2024 this month for some very interesting meetings and to take a pulse of the marketplace.

Will you attend conferences in 2024? Same or more?
I think the world has fully returned to “normal” in 2024. CES in Las Vegas this month was a good indicator – full crowds reminiscent of 2019. As a result we will be attending both the small, focused IP-centric conferences this year as well as the broader shows.

Also Read:

Fast Path to Baby Llama BringUp at the Edge

Vision Transformers Challenge Accelerator Architectures

An SDK for an Advanced AI Engine