Arm Deliver Their Next Step in Infrastructure

Arm announced their Neoverse plans not long ago at TechCon 2018. Neoverse is a brand, launched by Arm, to provide the foundations for cloud to edge infrastructure in support of their vision of a trillion edge devices. To a cynic this might sound like marketing hype. Sure, they’re widely used in communications infrastructure and certainly in edge devices, but they never really cracked the datacenter, or so conventional wisdom held. They put that concern to rest not long after TechCon when AWS announced immediate availability of EC2 A1 instances in their services. These are built on Arm-based Graviton processors, developed by AWS Annapurna Labs.

These instances offer what marketeers like to call value-pricing since they’re cheaper though lower performance than Xeon and Epyc servers. But don’t snicker; AWS wouldn’t put them in the lineup if they didn’t serve a valuable market. Web-services (kind of a big thing these days), containerized micro-services (important to the IoT) and applications caching are all examples I have heard where the A1 instances fit well.

Not bad, but Arm has no plan to sit on their laurels. The Neoverse announcement listed a progression of architectures diverging from the Cortex family. This started with Cosmos (a modest extension on Cortex as I understand it) on which Graviton was based, then evolving progressively to Ares, Zeus and Poseidon architectures. These are slated to appear on an annual beat with around 30% improvement in performance and features per architecture.

This doesn’t mean, as I understand it, that they are chasing Intel and AMD in the server biz. A revelation for me is that one size no longer fits all in datacenters. We’re used to thinking of a few oddball additions like GPUs, TPUs and FPGA-based systems but mainstream datacenter needs are already much more heterogenous, from serving trending videos to servicing infrequent updates from an edge device. Cloud providers are finding they can offer better differentiated services more cost-competitively using a range of platforms optimized for different classes of workload. There’s another consideration also. The cloud needs to be optimized, but so does the “fog”, those intermediate layers of data capture, consolidation and reduction that will be vacuuming up exabytes of data from all those IoT devices and forwarding on just what’s needed to the cloud.

So Arm announced last month availability of the first products in the Ares architecture – Neoverse N1 for compute and Neoverse E1 for throughput. N1 delivers an estimated 60% improvement in SPECint performance over Cosmos and is designed to support server-class features such as virtualization and remote access services (RAS). At the application level, ARM has benchmarked performance for heavy-duty web serving, multi-machine caching and Java at around 2-2.5X Cosmos performance. Not bad – the core beat the 30% guideline and even these complex cloud applications approach that range.

The E1 platform is aimed at very fast data filtering and throughput. Think of fog applications, say low-power (<35W) base stations in the 5G infrastructure, also multi-100GB routers in hyperscale datacenters. Arm didn’t reveal much about architecture here, other than to say it is scalable. They have however shared benchmark compute performance at over 2X Cosmos, throughput efficiency at 2.4X and throughput performance at 2.7X.

As usual, all of this is backed up by partners and a rich ecosystem: networking, OS, languages and libraries, virtualization and containers, and applications. Arm expects first deployable platforms based on these cores to start appearing later this year

Bottom line, Arm is aiming squarely for datacenters and IoT/5G infrastructure. In the datacenters, I’m not sure they care about the Intel/AMD use-cases. They can build healthy growth around all these evolving “non-traditional” use-models. Of course that doesn’t bode so well for Intel/AMD growth opportunities. Maybe someone else can comment on that front.