Arm Client 2024 Growing into AI Phones and AI PCs

I wrote last year about the challenge Arm Client/Mobile faces in growing in a saturated phone market and how they put a big focus on mobile gaming to stimulate growth. The gaming direction continues but this year they have added (of course) an AI focus, not just to mobile but also to other clients, notably PCs. It would be easy to be cynical about this direction but there are now indications (see below) that AI in client applications is moving beyond promises and is translating into real products. While there is undeniable debate around risks of AI in personal devices, I believe real value with safety will ultimately win out over both hype and apocalyptic claims. Given existing momentum, product builders must be in the game to have a chance of reaping those benefits.

What’s new in Arm Client 2024

Think AI at the edge requires a dedicated AI accelerator? Think again; according to Chris Bergey (SVP and GM for the Arm Client line of business), 70% of Android 3^rd party ML workloads run on Arm CPUs with no plan to move elsewhere. In support of this preference Arm continues to advance CPU and GPU platforms, this year introducing CSS – compute subsystem wrapping CPU and GPU cores, optimized and hardened now down to 3nm processes.

At the core IP level, Cortex-X925 delivers a 36% performance uplift for single-threaded processes, 41% uplift in AI (time to first token for tiny-Llama), while the Immortalis-G925 GPU offers 37% better performance over a range of graphics tasks and 34% improvement in inference performance over a wide set of AI and ML networks. Raytracing, first introduced in 2022, now delivers 52% improved performance on complex objects. For power saving architectures, Cortex-A725 provides 35% improved power efficiency over Cortex-A720 and the “LITTLE” CPU, Cortex-A520 has been further optimized for 15% improved power efficiency. Meanwhile Arm is showing 30% improvement in GPU power for games like Fortnite.

The Arm Client LOB have also introduced new software libraries to squeeze further application performance from these CSS-based designs. The first such libraries are Kleidi CV for computer vision and Kleidi AI for AI applications, exploiting Arm’s SVE2 and SME2 extensions. Based on a little digging around, Kleidi CV offers support for saturation arithmetic, color conversions, matrix transforms, image filters and resize with interpolation. Details are sparser on Kleidi AI but what I can find suggests support for what they call micro-kernels which allow say optimized matrix multiplications to be split into different threads across an output tensor. I think the key takeaway here is that for CSS implementations, just as hardware can be maximally optimized, low-level software functions (say for ONNX) can also be maximally optimized, which is what the Kleidi libraries aim to offer especially in signal processing and AI operations.

Enabling AI phones and PCs

Good story but where’s the market demand? Before I get to AI, Arm has been co-optimizing for Android with Google for improved performance in Chrome, also rippling through to handset OEM browsers, YouTube performance and lower power. Apparently, this collaborative effort is paying off as reflected in a trend back to OEMs building on the Google distribution rather than their own Android variants. (On an unrelated note, did you know that YouTube now ranks as the most popular streaming service on TVs? Food for thought in growth potential for Google and Arm.) Another example of Arm widening the moat, here again through CSS and collaborative development around a market they already dominate.

A downside for AI and CV is increasing complexity in corresponding pipelines and stacks. Proprietary ISPs and AI accelerators are appealing for added performance of course, but if a standard platform can be tuned to offer enough performance at low enough power, I can equally see a cost and time to market case for sticking to hardware platform evolution rather than revolution.

For example, the new Samsung Galaxy AI provides real-time multi language translation among other AI-based features, building on top of Google Gemini. Other phone OEMs like Oppo, Vivo and Xiaomi are introducing their own AI assistants and LLMs in search of differentiation. All sitting on Arm processing.

On the AI-enabled PC front, I’ll first refer you back to my write up on the Cristiano Amon (CEO of Qualcomm) chat at Cadence Live 2024. There he made a big deal about the convergence between phone and PCs and particularly the opportunity to reinvent the PC and the PC market through the Qualcomm Snapdragon X-Elite processor. This processor has already been barnstorming the automotive industry; if you buy a new car, chances are high that the chip behind your infotainment system is Snapdragon X-Elite. Now it’s also taking off in laptops. You can buy such a laptop from Lenovo, Samsung, Dell, HP, Microsoft and ASUS and perhaps others. Given Microsoft support for Arm-based platforms I’m sure other semiconductor systems players are looking hard at this opportunity.

Speaking of Microsoft, Satya Nadella was recently interviewed by the Wall Street Journal and was very excited. He sees AI CoPilot+ PCs besting Macs which is quite a statement given that laptops in general have had little new to offer for quite a long time. He says the new Surface platforms are 58% faster than the M3 MacBook Air and have 20% better battery life. Together with lots of opportunities to AI-enable all sorts of apps locally on the PC. (To be clear, he is talking about at least some of the AI happening on the PC, not in the cloud.) Satya name-dropped Arm multiple times during this interview, so yes, if this AI PC transition is real and market-changing, Arm Client products will also benefit from that transition.

Exciting ideas and trends. Much still to prove of course, not least around safety/privacy implications. As an inveterate optimist I believe issues will shakeout over time and useful/beneficial innovations will survive. You can read the Arm release HERE.