AI on the Edge

A lot of the press we see on AI tends to be of the “big iron” variety – recognition algorithms for Facebook images, Google TensorFlow and IBM Watson systems. But AI is already on edge-nodes such as smartphones and home automation hubs, for functions like voice-recognition, facial recognition and natural language understanding. Qualcomm believes there are good reasons for functions like this not only to stay on the edge but to continue to evolve there. I talked with Gary Brotman, director of product management at QTI to understand what’s driving this trend.

Part of the reason is availability. Carrier claims notwithstanding, there are still plenty of places you can’t get cellular or WiFi coverage. That might not be a huge deal for image recognition in Facebook photos, but it becomes a very big deal if you use biometric id(s) to unlock your phone or perform other critical functions. Which makes it a big deal in the rural/ mountainous/ heavily wooded areas that still account for the great majority by area of the US. Even urbanites accustomed to gigabit access can feel this pain when travelling any distance across country.

Part of the reason is privacy. If your dermatologist wants to use a mobile diagnostic device to check a possible melanoma, you have a right to expect that data will be handled with extreme care and especially that it won’t be shipped off to the cloud for analysis.

And part of the reason is security. No matter how great your hardware security may be, there are plenty of holes in software, and traditional signature-based approaches to malware detection are too cumbersome, too power-hungry and too slow to change to be effective against zero-day threats.

This is not an academic concern. Gary mentioned an IDC survey reporting that while less than 1% of applications use cognitive (aka AI) technologies today, more than 50% are expected to have that capability by 2018. The demand for cognitive-enabled functions is rocketing and if at least some of that capability has to be able to work untethered from the cloud, effective local solutions become essential.

Of course this doesn’t mean that everything has to be done locally. Training for deep-learning and related methods still happens in the cloud. But once training is downloaded, recognition should be able to function independently. If permitted, new data to enhance the training dataset can be uploaded when feasible, as Tesla does in gathering data from customer vehicles.

What powers this local analysis? Gary repeated a point he and others made on a panel earlier in the day. While there are now commonly-used hardware platforms for cognitive applications (CPU, GPU and DSP for convolutional and recurrent neural nets, along with frameworks like Caffe and Cuda), the bulk of application know-how today is still in software, not least because the domain is evolving so rapidly. Qualcomm sees platforms like their Machine Learning Platform as the best way to deliver a foundation for application developers. An SDK and frameworks offered within that SDK hide the gory details of implementation from the developer and can provide some level of future-proofing from changes in the underlying technology.

One example application can be found in Snapdragon™ Smart Protect. This is malware detection which uses not signatures for malware but rather machine-learning-based behavior triggers to protect against multiple types of attack and particularly against zero-day attacks. This is clever stuff. Signature-based approaches are impossibly clunky for mobile devices, are too easy to fool through mutating malware and cannot defend against zero-day attacks. Smart Protect behavioral detection looks instead at ~360 low-level behaviors which are harder to hide if the malware wants to achieve its intended objective (some examples cited include sending text messages when the user is not interacting with the device or taking photos when the display is off).

Finally, Gary noted that, to further support this trend to more processing (including AI) on the edge, Apple recently announced their position on “differential privacy” – the need to keep customer personal data out of their hands. Whatever you may think of Apple’s announcement, the principle they support is important. What we would consider personal used to be logins, passwords, bank data and other forms easily reduced to text. But increasingly we need to worry about information for facial recognition, typing behaviors, voice recognition and other biometrics which seem more abstract but could be just as damaging if leaked beyond our devices. I like what Qualcomm is doing; I might lose my phone or it might be stolen but I still have a better sense of control over something I can hold than over what ever might be happening in some distant cloud.

You can learn more about the Qualcomm Machine Learning Platform HERE.

More articles by Bernard…