WEBINAR: Adnan on Challenges in Security Verification

Adnan Hamid, CEO of Breker, has an interesting background. He was born in China to diplomat parents in the Bangladesh embassy. After I’m sure an equally interesting childhood, he got his BSEE/CS at Princeton. Where, like most of us he had to make money on the side, in his case working for a professor in the Psych lab on artificial intelligence projects. This was before deep learning got hot. They were working on planning algorithms, more of a rule-based approach to AI. You tell the tool what endpoint you want to get to, and it figures out how.

Adnan joined AMD as a new grad, assigned to create testbenches, a task he quickly recognized would be mammoth and never-ending. He thought back to the work he’d done in the psych lab, wanting to try planning as an approach to test generation. Tell a tool this is the result you want to get to, give it a bunch of strategies and let the tool figure out reasonable test cases. From that, Adnan started Breker, where they pioneered portable stimulus starting from graph-based models because they’re easy for us simple humans to understand. Which of course evolved further into an even broader standard for system-level verification.

Breker has, among other capabilities, an app for security verification based on this standard. I wanted to get his take on why security verification is hard and what characteristics a solution needs to have, independent of any product viewpoint. That led to an interesting discussion.

Where Security is Most Important

We started with the business motivation. For all the advantages of the IoT, the disadvantage is that nothing now is a closed system. Everything by default can be hacked, traced, identities stolen, caused to misbehave. This is a huge problem for the DoD who face very sophisticated attackers. It’s a huge problem for the government and infrastructure, given the antiquated IT equipment on which they depend, it’s a huge problem for financial institutions, for businesses whose reputation rests on the security of their systems, and on and on.  So #1, security now come with a societal price tag and a financial price tag, both huge. Much of it demands six-9’s (99.9999%) levels of security which somehow we have to prove.

Defining the Requirement

The second problem is not so much a technology problem as a people problem. How do you know you’ve tested enough? This is of course a classic verification problem but the classic solution – hit some level of coverage through randomized testing – doesn’t work. For security we need to check all the vulnerabilities of a particular target. It would take forever to get to six-9s. Plus a lot of what you’d be testing would be meaningless. What you really want is to test exhaustively within a realistic range of possibilities – like a semi-formal approach to dynamic verification. In fact, formal is already good at doing this at the IP level and in some sub-systems. What we need is to carry that concept over into full system testing.

How do you get there? Definitely not by writing a test, then another test, ad infinitum. Or by randomizing those tests. First, the range of possibilities needs to be captured in a way we can visualize easily. What are the possible master/slave paths in this system? For all the possible access and privilege combinations on those endpoints, which are allowed? Then in the memory map, which regions are accessible to which devices and under what conditions?

This might all seem rather trivial but Breker finds that simply constructing these access rule tables, no tools required, will often reveal gaps in understanding. Architect/designers have to fill out each entry and when they do, they realize they actually don’t know the right answer for some cases. Because we really are human after all.

Testing Compliance

Once the tables are filled out, tools can get involved. Now you’re back to that question of whether you want to hit six-9s or just do the best you can within a schedule. Which will of course depend on the application of the product. If you do have to hit six-9s, you have to do exhaustive testing, that semi-formal kind of testing at the system level. AI planning algorithms, starting from PSS models, are actually pretty good at that.

That’s Adnan’s view. If it’s critical to business or national security, you have to hit a high level of confidence. If you have to hit a high level of confidence, you need to use a semi-formal (realistic exhaustive) approach. And the best way to do that is through planning algorithms to generate exhaustive tests over a PSS graph.

You can learn more by REGISTERING TO WATCH THE BREKER WEBINAR, scheduled for June 16, 2020 at 10am Pacific.

Also Read

Breker Tips a Hat to Formal Graphs in PSS Security Verification

Verification, RISC-V and Extensibility

Build More and Better Tests Faster