More than just mobile phones for Mali

ARM TechCon 2015 was another tour de force for ARM and its ecosystem. Besides some of the developments in mobile, IoT, and security (more coming soon in the Epilogue of “Mobile Unleashed”), there were two topics that I found very educational and will cover in blogs this week. One was how the Mali family is powering more than just mobile phones.

Two separate presentations, one by Dan Wilson of ARM and the other by Henk Pruim of INSIDE Secure, looked at two different segments of Mali multimedia technology – both in markets for consumer electronics.

Wilson covered one of the hottest areas of consumer electronics: the Android-powered wearable device. Mali GPUs are built to run OpenGL ES. In the mobile phone environment, that has meant a drive for more advanced high-end GPUs to run Open GL ES 3.2 and the Vulkan API, subsuming the Android Extension Pack (AEP). Vulkan unifies an API across Windows 10, Android, Tizen, and some Linux distributions.

However, the use case for wearables is very different from that of phones or tablets. Unlike a phone or tablet where users aren’t too distressed if they have to charge daily after heavy graphics usage, wearable users are looking for something closer to a week between recharges. Displays on wearables are too small to run sophisticated games. In some modes, such as just moving an analog second hand on a watch, frame rates are inherently 1 fps, but in others the expectation is 30 fps or higher with complex shading and minute interactions.

For user interface design without the advanced gaming components, Open GL ES 2.0 provides the necessary per-pixel control and energy efficiency, integral to both Android Wear and Tizen. With knowledge gained from many projects, ARM turned back to the basic Utgard architecture and engineered a new core, the Mali-470 GPU introduced just a couple weeks ago, for lower power use in wearables.

The strategy involved analyzing a set of representative frames. Mali GPUs have performance monitoring units to help determine performance in fps and bandwidth in MB/frame. Challenges included exploring memory subsystem power, not just the GPU core, and preserving the driver stack. Once the data and command lists were determined, they were converted into a testbench for simulation and optimization.

As Wilson put it, “… every design change contributed to power reduction.” Dynamic power was reduced through steps including optimizing quad-thread scheduling, better clock gating including L1 cache, and use of fixed point arithmetic where possible. Static power reduction amounted to attention to flexible, independent power domains for implementor configuration, and removing features not needed for OpenGL ES 2.0.

The result is the Mali-470 is exactly – to three decimal places on an average of benchmarks – 2x as efficient in FPS/mW compared to the Mali-400. One to four pixel processors can be implemented, supporting up to 1080p at 60fps and 32bpp. This is a great example of how designers are pursuing chips optimized for wearables, not just knocking off mobile phone designs and underclocking them.

At the other end of the spectrum is a segment we don’t discuss as much, the Mali video processing cores. The second generation Mali-V550, introduced a little over a year ago, is state-of-the-art for set top box design. Pruim launched into a deep dive on the subject of content protection for 4K UHD. With dramatic increases in over-the-top (OTT) content, digital rights management (DRM) is an end-to-end proposition.

Obviously, the bit rates of 4K UHD are much higher. Resolution increases are just part of the story; frame rates have doubled to 60fps, and color depth has increased to 10 bits. Compression helps, and H.265 is twice as efficient as H.264. Ultimately, 4K UHD streaming is 12x faster than HD streaming.

Content protection demands are also on the rise, mandated by the content providers and imposed on any medium of distribution. One interesting concept is protection is time varying; first-run content may be highly protected, while syndicated content in reruns less so. Requirements for 4K UHD typically include protection against hardware and software attacks, trusted execution environments plus hardware crypto, HDCP/DTCP output protection, and watermarking to prevent screen captures.

That is just the overview diagram. Pruim wrapped up his presentation with a data flow slide (hard to read even full size) showing how encryption ripples through an Anrdoid-based system in 25 steps from reading in content to outputting it on HDMI. Central to the concept is the trusted execution environment, in the Mali-V550 based on ARM TrustZone and a secure OS, keeping both content and keys safe.

I didn’t realize just how sophisticated the DRM environment has become. INSIDE Secure is offering cutting-edge IP complimentary to the Mali-V550 and tuned to Android, helping ARM add DRM to their Juno reference platform (with critical IP in an Xilinx Virtex-7 expansion daughtercard). Fortunately, power is limited only by cooling in the set-top box environment, and measures to increase performance with security allow for very sophisticated designs.

Further reading:

ARM Mali-470 GPU IP
ARM Mali-V550 Video Processing IP
INSIDE Secure Content Protection

These are two great examples of how the ARM ecosystem customizes IP for the job, and just how far IP for consumer electronics has come since ARM7TDMI launched 20 years ago. Later this week, I’ll be taking a look at more ARM TechCon news from the software side – mbed.

More articles from Don…