Qualcomm’s New Spectra ISP and Camera Modules Enable Nextgen AR

The mobile VR and AR space has been evolving rapidly, with many different players innovating in recent months. Companies like Apple and Google have been innovating on the software and hardware front, but others have been working diligently to support some of these efforts as well. One of those companies is Qualcomm with their support of technologies like Google’s Tango platform. However, Tango has had some challenges in taking off with the complexity of the hardware requirements proving to be a challenge for OEMs to ship in volumes.

In the past, Qualcomm had a camera module program co-branded with their ISP (image signal processor) which processes the image data from the cameras. This program was designed to make it easier for Qualcomm’s customers, the smartphone OEMs to quickly and reliably implement dual camera setups with a wide angle and telescopic zoom camera. That was what the market needed in the past, but now the market needs new capabilities which Qualcomm’s new Spectra Module Program and ISP will support.

The new Spectra Camera modules include an Iris Authentication Module that has latency as low as 40ms and features an Omnivision 1080P IR sensor for high resolution iris image capture. The real focus with these new camera modules, however, comes from the computer vision capabilities which include both passive and active depth sensing capabilities. The entry-level solution for value-tier devices will feature two cameras and passively calculate depth, allowing for more coarse measuring and lower cost. The high-end active depth sensing camera solution will feature three cameras including an IR emitter and IR camera for high resolution depth sensing at distances up to 4 meters and 0.1 mm accuracy.

The Spectra Camera modules and some of their capabilities will be supported by the newest generation of Spectra ISP inside of the next generation Snapdragon SoC. This new ISP will support features like multi-frame noise reduction in hardware which is like what Google already does in software with their HDR+ algorithm and make it available to anyone that uses their ISP. Qualcomm’s new Spectra ISP also supports motion compensated temporal filtering (MCTF) and accelerated EIS (electronic image stabilization) which cleans up the noise in low-light video and helps to sharpen the image as well, making low light video quality significantly better. Last but certainly not least is the implementation and support for 6-DoF and SLAM with 16ms motion to photon latency for inside-out tracking at room scale and collision avoidance. This paired with the new Spectra Camera module will enable highly precise AR solutions that can rely on the sub-mm accuracy point cloud generated by the Spectra Camera module and Spectra ISP inside of the Snapdragon.

Qualcomm’s new camera modules, especially their active depth sensing camera module, seem like no-brainers for Android OEMs to adopt to compete with what many expect that Apple will implement in the new iPhone 8. Qualcomm’s new ISP features are also sure to elevate the camera experience for Android smartphone users and can help to rise the tide that lifts all boats. Right now, the more integrated and simple an AR solution, the more likely OEMs and developers are to pick it up and build towards it. I believe that this new camera module helps to bring the Android ecosystem closer to where we expect Apple will be with the iPhone 8.