Deep Learning and Security

Deep learning based on neural nets is most commonly thought of as a very promising approach to scene recognition, text recognition and similar areas. Now there is increasing interest in applying this technology to security objectives. There were a number of papers on this topic in a 2015 BlackHat conference and there are several ventures either investing in or built around this method.

Deep Instinct, one such venture, was founded in 2014 to build security methods on deep learning, and to coin another cool company name around the word “deep”. (I’m hoping someday to see a venture called Deep Doodoo – the founders won’t be stuck for a description of their problem when they get into trouble).

The philosophy behind Deep Instinct and deep learning in general as applied to security is to build automatic adaptability into threat detection and prevention. The dominant approach to detection today requires human involvement at some level, which is why zero day attacks are such a problem – you can’t defend against an attack-type you are seeing for the first time. This presents a general problem for defenses because it’s not difficult to modify a virus in a relatively minor way, or even to make viruses self-modifying to confuse standard methods of detection.

But a neural network initially trained on threat examples can potentially continue to self-train (as Deep Mind’s AlphaGo does for Go games). That could mean that Deep Instinct could significantly improve defenses. Variants of known viruses could be detected and blocked without human intervention and only truly new attacks could hope to succeed.

Deep Instinct has already partnered with FireLayers on security for cloud platforms, which could add further weight to cloud suppliers’ claims that they are actually more secure than in-house systems. All of which is good, but I feel it may be premature to declare victory quite yet. I still see a number of potential issues, though I’m very willing to admit that I may not have a sufficiently deep (there it is again) understanding of the application.

First, training is presumably central but can (post-training) self-learning updates be shared, or does each installation become an island after training, learning on its own but with no ability to share learning? Perhaps there is a way to share but I’m not quite sure how you merge training weights from multiple neural networks. Might some locally-appropriate defenses be lost in that merge or is each merge guaranteed to preserve all subset detections?

A plus to the neural network technique is that it is not easy to hack in the conventional manner – you can’t search for weaknesses in antiviral code for example. But a minus is that neural networks, especially self-training networks, can be hacked by driving them with biased data. Neural net conclusions after all are probabilistic. If I drive data at a neural net which biases to some class of threat examples, I can reduce sensitivity to a different class. Biasing is not an abstract possibility. Microsoft had to withdraw their Tay chatbot recently after it had been gamed into expressing racist and other offensive views.

Finally, one reviewer of the BlackHat papers was somewhat skeptical – not necessarily of the general direction but of lack of depth in published research, also challenges in getting to a sufficiently representative corpus of virus examples for training. Perhaps Deep Instinct has answers to these questions.

You can learn more about Deep Instinct HERE. The BlackHat review is HERE.

More articles by Bernard…