It’s popular to use DSP chips for vision processing in diverse applications like ADAS, security cameras and AR. Tensilica has been designing DSP chips and IP since 1997, and their technology was successful enough that Cadence acquired Tensilica back in 2013. At the IP-SoC Silicon Valley 2022 event in April I had the pleasure to watch the presentation from Amol Borkar, Product Marketing Director, IPG TIP, from Cadence Design Systems, and his topic was titled, Designing the Next Ultra-Low-Power Always-On Silicon.
In the past 25 years there have been over 40 billion processors shipped by Tensilica customers, 19 of the top 20 semiconductor vendors use Tensilica IP, and three recent customers include: UNISCOC (Application Processor), Light (Auto vision) and Xvision (AR). The trends for processors and sensors can be shown across five categories, and the ultra-low-power segment requires a long battery life:
Microphones and ultrasonic sensors are common approaches to wake up electronic devices, and cameras are now being used for people detection, face detection, even gestures. Amol focused on vision and speech processing for always on embedded and IoT devices. An always-on (AON) device runs continuously, reads inputs from sensors, and often dissipates under 1mW of power, while using a microcontroller.
To wake up a device requires three stages: Detection, Authentication, Processing. For AON, the focus is on the Detection phase, where a low-power MCU or DSP is typically used to do something like keyword spotting, face detection, visual wake words, or handling gestures. An AON subsystem could be designed with both a Vision AON processor, and Audio AON processor; or with a combined IP for both workloads.
For audio and speech dominant AON processing tasks there is a processor family called the Tensilica HiFi DSP, then for image and camera dominant AON processing, there is a processor family called the Tensilica Vision DSP. The low-energy DSP for vision and AI tasks is the Tensilica Vision P1 DSP.
Always-On Face Detection
The earliest approaches used a camera connected to a CPU or App processor, although the neural network algorithms on a CPU are not very energy efficient. A more energy efficient approach is to have the camera connected to a vision DSP first, then alert the CPU or App processor after detection. The most energy efficient approach is shown below, where the P1 DSP reads a low resolution image and detects a face, then a vision DSP reads the high resolution image to detect, recognize and authenticate while connected to a CPU or app processor.
The DSP and CPU only wake up after the P1 does the face detection, so this approach takes the least energy. Even after the face is detected, you can choose to have the P1 perform co-processing tasks, or have the P1 go into standby. Benchmark results compared a competitor to a Vision P1 system using TinyML v0.5 for visual wake word, image classification, keyword spotting and anomaly detection. The Vision P1 system performed higher inferences per second, and much lower energy numbers than the competitor.
Some customers use the Vision P1 DSP with their smart image sensor to classify objects, support bounding boxes and segmentation. AR applications for foveated rendering have detected pupil center and gaze detection with a Vision P1 DSP. Aside from AON, Vision P1 is a fully functioning Vision DSP and there’s plenty of support that comes along with the P1:
- Imaging and vision DSP tools and libraries
- Supports AI flow of XNNC, NNAPI, TFLm
- 100+ model zoo networks
Summary
The Cadence Tensilica products for both audio and vision are quite popular, as they’ve been in the DSP business for decades now. The Vision P1 DSP has been optimized for AON applications, and performs tasks like face detection quite efficiently. There’s a 30 minute video of this presentation online.
Related Blogs
- Battery Sipping HiFi DSP Offers Always-On Sensor Fusion
- On-Device Tensilica AI Platform For AI SoCs
- Cadence Tensilica FloatingPoint DSPs
- Cadence Extends Tensilica Vision, AI Product Line
- How does TensorFlow Lite on Tensilica HiFi DSP IP Sound?
- Cadence Dives Deeper at Linley Fall Processor Conference
- Tensilica DNA 100 Brings the AI Inference Solution for Level 2 ADAS ECUs and Level 4 Autonomous Driving
Next Generation of Systems Design at Siemens