How does TensorFlow Lite on Tensilica HiFi DSP IP Sound?

In all the hubbub about AI/ML, it’s easy to see why visual ML gets more attention. It’s got appeal because of applications such as autonomous driving. Because of this it’s easy to overlook the importance of audio ML. I own a Tesla and putting it into autopilot is very cool, but even it has voice recognition built in as an important feature to reduce driver distraction. At home I have numerous Google Minis that we use every day for controlling our heating, lights, etc. When all is said and done I am sure I use our voice ML appliances more often than the Tesla Autopilot.

Audio ML is most useful when it runs on the edge, not in the cloud. This enhances security because the audio stream does not need to travel to the cloud. Local ML processing also improves latency and lowers network loading. With this in mind, Cadence just announced support for TensorFlow on their Tensilica HiFi DSPs. Along with the announcement on their website, Cadence also gave a presentation, titled “Efficient Machine Learning on HiFi DSPs Using TensorFlow”, at the Linley Processor Conference in March. This year, of course, the conference was held online due to Covid-19.

Yipeng Liu, Technical Marketing Director at Cadence for Tensilica IP, started off the talk with a discussion of how audio ML was recently found useful in China to help prevent the spread of the Covid-19 virus. Once elevator buttons were found to be a transmission risk, a team of engineers rapidly developed a voice activated system that worked in the acoustically difficult elevator environment to let riders control the elevator.

Embedded edge devices present several constraints for deploying ML, including small memory sizes and limited processing capabilities, both of which can lead to difficulties in coding effective solutions in a timely manner. There are optimization methods that can be employed; however, they make the process more manual and ad-hoc. Edge-based hardware usually has fixed-point math units and also requires C/C++ rather than Python code.

Cadence is offering TensorFlow Lite for Microcontrollers (TFLM) targeted at its Tensilica HiFi DSP IP, which effectively deals with these limitations. Included in TFLM is the HiFi Neural Network library (NN lib), HiFi Nature DSP library (NDSP lib) and 8/16/32 bit SIMD and VFPU support. The libraries are optimized for all Tensilica HiFi DSPs. The libraries are also framework agnostic.

Yipeng also talked about the Tensilica XAF middleware that allows for faster system integration. It handles memory allocation and management. It also has the ability to install and uninstall components to save memory. Yipeng said that XAF middleware simplifies integration of ML and also traditional audio components from multiple providers.

We live in a world where audio information plays a huge role. Even though image processing gets the limelight, audio processing enables many important tasks. These range from voice processing, to song recognition, and a host of other important tasks. Now with the ability to easily add ML to devices at the edge, the applications and uses for this technology will expand further. Cadence is now providing a way to develop and deploy these applications with much less effort with the widely used TensorFlow platform. Full information about Cadence TensorFlow Lite for Microcontrollers can be found on the Cadence website.

Also Read:

Ultra-Short Reach PHY IP Optimized for Advanced Packaging Technology

Cadence Dives Deeper at Linley Fall Processor Conference

Leveraging Virtual Platforms to Shift-Left Software Development and System Verification