The Journey of Interface Protocols: The Evolution of Interface Protocols – Part 1 of 2

Prolog – Interface Protocols: Achilles’ Heels in Today’s State-of-the-art SOCs

June 30 was only a week away when Varun had a sleepless night. The call from the datacenter manager the evening before alerted him on a potential problem with the training of a new Generative AI model. Six months earlier Varun’s employer installed the latest generation of a leading-edge ML accelerator that promised to cut the training time of the largest GenAI models by 50% via stellar processing bandwidth and reduced latency. Previously, training the largest LLMs, boasting over a trillion parameters, on the then state-of-the-art accelerators took approximately one year to achieve a satisfactory level of confidence. This process involved leveraging high-quality training data to minimize hallucinations. On paper all was perfect if not for a little secret unknown to most: the loss of even a single data packet during training could necessitate retraining the entire AI model. The secret became a mantra in the Gen AI community: “Start and Pray.”

The grim scenario described above is just one of many with potentially catastrophic consequences unfolding across various cutting-edge industries today. From autonomous vehicles making split-second life-or-death decisions to AI-driven medical systems managing critical diagnoses, the stakes have never been higher. Yet, these industries share a disturbingly common vulnerability: malfunctions in interface protocols.

Part 1 of 2. The Evolution of Interface Protocols: From Supporting Components to Critical Elements in Modern SoC Designs

Part 1 presents a comprehensive overview of the evolution of Interface Protocols, tracing their journey from auxiliary support components to indispensable pillars in cutting-edge HPC SoC designs. These protocols now underpin not only everyday technologies but also mission-critical, complex AI applications. The section explores the key drivers behind the rapid advancement and frequent upgrades of existing protocols, as well as the innovation fueling the development of entirely new standards.

Brief Historical Perspective of Interface Protocols

Interface protocols have their origins in the early development of computer systems, dating back to the 1950s and 1960s, when computers transitioned from isolated, monolithic machines to interconnected systems. For most of the time since their inception, the design of Interface Protocols has been driven primarily by the need for connectivity between different components of a computer system or between different systems themselves. As the electronic industry expanded, Interface Protocols facilitated the interoperability of components sourced from different vendors.

Over the past decade, the landscape of Interface Protocols has undergone a profound transformation. Technological advancements have created new demands for higher performance, shorter latency, greater power efficiency, improved reliability, and enhanced security, all of which have driven significant changes in interface protocol design and development. These evolving requirements are now central to a wide range of applications, from consumer electronics to industrial systems, automotive, and more.

As a result, modern System-on-Chip (SoC) development priorities have shifted dramatically. Performance is no longer the sole focus; energy efficiency, data integrity, and robust security are equally critical.

See Sidebar: Seven Decades of Interface Protocols Development.

Key Drivers Shaping the Evolution of Modern Interface Protocols

The evolution of modern interface protocols is shaped by two major, yet opposing, industry trends. On one hand, the rapid growth of software development has shifted System-on-Chip (SoC) functionality from hardware-centric implementations to software-defined solutions. On the other hand, the meteoric rise of artificial intelligence (AI)—especially predictive and generative AI—has reintroduced heavy compute and data manipulation demands, moving the focus back to SoC hardware.

Software’s Dominance: Transforming SoC Development

In his influential 2011 article, “Why Software Is Eating the World,” Marc Andreessen predicted the transformative power of software across industries. Over the past decade, this vision has materialized, driving profound changes in SoC design.

The shift to software-defined solutions has revolutionized the development process by offering greater flexibility, faster time-to-market, and simplified post-release updates. Developers can now enhance and scale SoC functionality without requiring costly hardware redesigns. These advantages have significantly streamlined the SoC lifecycle, enabling rapid responses to changing market demands.

However, this transition has brought its own set of challenges:

1. Data-Intensive Operations: Software’s growing reliance on large datasets demands substantial memory capacity.
2. Energy Consumption: The continuous transfer of data between memory, processing elements, and interfaces consumes significant power.

AI’s Impact: Redefining SoC Hardware Requirements

The rise of AI has compounded these challenges while introducing a third. Predictive and generative AI applications require processing engines capable of handling massive data loads with minimal latency.

Traditional CPUs often fall short, as their architecture struggles with the bottlenecks of data movement between memory and compute units. To address these demands, the industry has embraced GPUs, FPGAs, and specialized AI accelerators, which excel at handling high-throughput workloads.

Yet even the most advanced processors face limitations if data delivery speeds cannot keep up. When memory and I/O protocols lag, high-performance processing units risk underutilization, idling while waiting for data. This highlights the critical importance of modernizing interface protocols to meet AI’s escalating data demands and fully leverage advanced SoC capabilities.

Implications on Interface Protocols by Key Industry Trends

As AI and software continue to drive innovation, balancing these opposing trends will require advances in memory and I/O protocols, leading to the rapid evolution of existing protocols and the emergence of new protocols.

Implication on Complexity

As modern SoCs designs have grown in complexity, the embedded Interface Protocols that interconnect them have also progressed at an extraordinary pace.

High-performance protocols, such as the latest iterations of PCIe—from Gen 4 to the cutting-edge Gen 7—have evolved into highly complex systems. The PCIe Gen 7 specifications alone now encompass over 2,000 pages, underscoring the complexity needed to enable advanced functionality. Furthermore, implementation complexity continues to escalate as data transmission speeds push the physical limits of what is achievable, challenging both design and manufacturing processes.

Implication on Performance

Cloud infrastructures, AI algorithms, and generative AI applications are fueling an unprecedented demand for data, both in volume and in processing power. This surge drives the need for massive memory capacities, higher communication bandwidth, lower latency, and significantly enhanced throughput.

Traditionally, achieving faster data transfer rates in new protocol generations was accomplished by physically positioning on-chip components closer together. However, modern protocols must now support connections over longer distances, where both bandwidth and latency become critical challenges.

Evolution of Existing Interface Protocols and Emergence of New Interface Protocols

In the fast-changing landscapes of AI, machine learning, and big data analytics, established protocols such as PCIe, Ethernet, and memory interfaces have undergone significant evolution to meet the growing demands for larger capacity and higher performance. As AI software workloads generate vast amounts of data, traditional data transfer mechanisms have faced challenges in keeping pace, resulting in inefficiencies affecting processing power, latencies and power consumption. Research highlights that moving data via DRAM consumes up to three orders of magnitude more energy than performing arithmetic operations on the data, making memory-related power consumption a critical bottleneck in high-performance computing environments.

The surge in demand for memory bandwidth and capacity has begun to exceed the capabilities of existing protocols. Consequently, well-established technologies like PCIe have had to evolve continuously, leading to innovations such as PCIe 7.0 and beyond. Meanwhile, new solutions like Compute Express Link (CXL) have emerged to address these limitations, offering greater flexibility in how memory and accelerators are connected. CXL enables cache coherency and shared memory resources across CPUs, GPUs, and other accelerators, enhancing efficiency and cost for workloads like AI inference and data analytics.

Simultaneously, multi-die architectures, which integrate multiple dies or chiplets within a single package, have introduced transformative improvements in data movement between processing units. By bringing these dies physically closer together, communication between them becomes faster and more power-efficient, significantly reducing latency.

Evolution of Existing Protocols

Existing protocols have been evolving at an increasingly rapid pace. One prominent example is PCIe, which has now advanced to PCIe Gen 7, set for release in 2025. Even as Gen 7 approaches, the PCI-SIG (Peripheral Component Interconnect Special Interest Group) is already discussing the specifications for PCIe Gen 8, highlighting the urgency of keeping up with growing performance needs. See Table I

The evolution of Ethernet has been even more dramatic. Ethernet standards, particularly those supporting speeds up to 10 Gbps, have undergone more than a dozen amendments in the past five years alone, with the rate of updates accelerating. Ultra-Ethernet, currently under development by the Ultra Ethernet Consortium (UEC), which includes leading companies in AI and networking like AMD, Intel, and HPE, is specified to support transfer speeds of up to 224 GB/sec. For context, this is nearly twice the speed anticipated from PCIe Gen 7 and positions Ultra-Ethernet as a direct competitor to Nvidia’s NVLink[i], which offers a bandwidth of 480 GB/sec.

Memory protocols are advancing at a similarly rapid pace. DDR (Double-Data-Rate) memory has reached its sixth generation, while HBM (High-Bandwidth Memory) is now in its third generation, offering a bandwidth of up to 800 GB/sec. These developments in memory protocols are crucial for supporting the growing data needs of AI and high-performance computing (HPC) environments.

Emergence of New Protocols

In parallel with the evolution of existing protocols, entirely new protocols are being designed to address the unique demands of AI accelerator engines, where high bandwidth and low latency are critical. Some of the most groundbreaking new protocols include UCIe (Universal Chiplet Interconnect Express), UAL (Ultra Accelerator Link), and UEC (Ultra Ethernet Consortium). These protocols are specifically engineered to ensure interoperability across diverse ecosystems while maximizing performance, increasing data bandwidth, and improving power efficiency. They also are designed with a emphasis on security to ensure reliability and security of data transfer critical in AI and cloud-based systems.

In summary, the rapid evolution of existing protocols, coupled with the emergence of new ones, is driving the technological infrastructure required to support the next generation of AI and data-intensive applications like autonomous driving vehicles.

Conclusions

The increasing complexity of SoC software, along with the rapid evolution of SoC hardware and rising performance demands, is pushing the design community to continuously innovate and extend the boundaries of interface protocols, all while ensuring efficiency and reliability. Modern interface protocols are designed with flexibility in mind, allowing them to adapt to diverse applications and workloads. This ongoing evolution fosters deeper integration between hardware and software, enabling SoC designs to deliver highly optimized solutions that balance performance, efficiency, and security.

Verifying the functionality and performance of these advanced protocols in sophisticated, software-driven systems demands a blend of high-performance hardware-assisted verification and protocol verification technologies built on proven protocol IP implementations. The rapid pace of protocol innovation necessitates aggressive roadmaps that match IP advancements with verification technologies, ensuring alignment with the tight time-to-market schedules critical for HPC market leaders.

As these protocols evolve, they will play a critical role in shaping the next generation of interconnected systems, expanding the possibilities in fields like artificial intelligence, autonomous systems, and cloud computing.

—

Sidebar – Seven Decades of Interface Protocols Evolution

The origins of Interface Protocols date back to the early days of computing in the 1950s and 1960s, when they were initially developed to enable communication between different components within a computer. These early protocols were often proprietary and hardware specific. However, over time, they evolved into standardized systems designed to facilitate seamless connectivity, communication, compatibility, interoperability, scalability, and security across devices and systems from multiple vendors.

As technology advanced, Interface Protocols became more sophisticated, secure, and universal, playing a crucial role in ensuring the smooth operation of increasingly complex computing environments. Over the course of seven decades, these protocols have been integral to the evolution of communication technologies.

1950s: Early Computer Systems and Proprietary Protocols

• In the early days, computers like the UNIVAC and IBM 700 series used proprietary protocols to communicate with peripherals like punch card readers, printers, and tape drives.

1960s: Rise of Serial Communication and Early Networking

• With the proliferation of peripherals and modems, the need for a protocol to transfer data over simple connections led to the development of the RS-232 standard, one of the most widely used serial communication protocols.

1970s: Networking and Early Standardization

• In 1973, Xerox PARC developed the Ethernet protocol for local area networks (LANs) that quickly became the dominant standard for connecting computers within an area, enabling faster and more reliable communication.
• Around the same time, ARPANET conceived the TCP/IP suite to provide a scalable protocol for interconnecting different networks. It set the stage for the Internet.

1980s: Expansion and Global Standardization

• In the 1980s, the Small Computer System Interface (SCSI) standard was developed to connect peripherals as hard drives, scanners, and others to host computers.

1990s: Internet and Security Computing Peripherals Protocols

• With the rise of the World Wide Web, HTTP (Hypertext Transfer Protocol) and HTML (Hypertext Markup Language) became fundamental to web communication. HTTP facilitated the transfer of web pages between clients (browsers) and servers.
• The Universal Serial Bus (USB) standard, introduced in 1996, supported data transfer rates of 1.5 Mbps (Low-Speed) and 12 Mbps (Full-Speed), significantly improved over serial and parallel ports. It became a crucial protocol for connecting devices such as keyboards, mice, and storage drives to computers, offering plug-and-play functionality.

2000s: Wireless Communication and Computing Peripherals Protocols

• Wi-Fi: Wireless communication protocols, particularly Wi-Fi (based on the IEEE 802.11 standard), became increasingly important in the 2000s as mobile computing and smartphones gained popularity.
• Bluetooth: Bluetooth emerged as a short-range wireless protocol for connecting personal devices such as headphones, speakers, and wearables.
• The USB has seen more than 10 upgrades since its inception. The latest USB4 v2.0 released in 2022 supports a max bandwidth of 80 Gbps.

2010s-Present: High Performance and Secure Data Transfer

• PCIe, Ethernet and Memory protocols underwent several upgrades in rapid progression and emerged as the de-facto standards for AI and datacenters.

[1] The NVLink Switch is the first rack-level switch chip capable of supporting up to 576 fully connected GPUs in a non-blocking compute fabric. The NVLink Switch interconnects every GPU pair at an incredible 1,800GB/s.

Also Read:

Metal fill extraction: Breaking the speed-accuracy tradeoff

How Arteris is Revolutionizing SoC Design with Smart NoC IP

CEO Interview with Ido Bukspan of Pliops

Share this post via:

• 
• 
• 
•