Data Outgrowing Datacenter Performance

Last week I attended the Linley Datacenter Conference. This is not the conference on mobile which is not until April. However, a lot of the growth in datacenter is driven by mobile, with the increasing dominance of the model where data is accessed by smartphones but a lot of the backend computing and datastorage is in the cloud.

From 2012 to 2017 smartphones have grown 20% and tablets 46%. There are now more than 10 billion internet-connected devices. It takes about 400 smartphones to drive one server in a datacenter. During that same 5 year period, traffic per mobile device has increased between 4 and 8 times, with video being a big driver (growing fast and with the highest bandwidth requirements). This has driven a cloud computing growth rate of 23.5% from 2013 to 2017. As a specific example, Amazon Web Services (AWS) adds enough servers daily to support a $7B company.

One them running through the first part of the conference is that the growth in the amount of data is overwhelming the compute power. The standard way to build a datacenter is with a two level network, a top-of-stack (ToS) router at the top of each stack of servers, and then another level to link the stacks and connect to the outside world. The ToS may not use ethernet, RapidIO and PCIe are options for the stack communication. The problem is that the data rates are now so high that it is taking more and more of the compute power on each server just to run the network stack, leaving less and less compute power available to do actual work.

More of the cost and more of the power dissipation is in the networking, and as a result there are solutions from companies like Cavium, Netronome and Tilera that can be used to offload the servers and free up more compute power for actual processing. Also, specialized memory architectures such as the Micron Hybrid Memory Cube are also targeted at improving the power and performance.

Another area that still seems to be coming soon rather than here is scaling out rather than up, using (mostly) ARM-based servers that have a much lower cost-of-ownership. Intel/AMD cores have very high single thread performance but this comes at a significant cost in terms of power dissipation. Highly integrated ARM-based chips are much lower power but at have less single threat performance. However, aggregate performance can be much higher for the same budget in $, W or Size. Solutions are shipping but the big guys like Facebook don’t seem to yet be building ARM based datacenters. But there are products shipping:

• Broadcom/Netlogic with up to 8 cores
• Cavium Octeon II family with up to 32 cores
• Freescale QorIQ P4 with up to 8 cores
• LSI Axxia ACP3448 with 4 cores
• Tilera Tile-GX with up to 72 cores

More articles by Paul McLellan…