2D NoC Based FPGAs Valuable for SmartNIC Implementation

Smart network interface cards (SmartNICs) have proven themselves valuable in improving network efficiency. According to Scott Schweitzer, senior product manager at Achronix, it has been shown that SmartNICs can relieve up to – and perhaps beyond – 30% of the host processor’s loading. SmartNICs started out taking on simple functions to supplement the host processor. With advances in SmartNIC design and architecture they have taken on much more complex roles and provide a high degree of flexibility with their re-programmability. I recently watched an on-demand webinar replay from Achronix where Scott talked about five important aspects of SmartNICs. The webinar is titled “5 Reasons Why a High Performance Reconfigurable NIC Demands a 2D NoC”.

According to Scott there are three fundamental architectures for SmartNIC design: bump in a wire, Von Neumann Sidecar and single chip. All of these except single chip require multiple chips with chip-to-chip interfaces that create bottle necks. With 100GbE, and above, packet rates are staggering, reaching 2,400 Mpps on dual port 400G. Each packet will typically be touched multiple times when transiting the NIC. Thus, the slower PCIe transfers within multi-chip SmartNICs will hinder throughput. FPGAs are attractive for SmartNIC operations because they are reconfigurable for different workloads depending on the application. All of this points to single chip FPGA based solutions dominating the market.

SmartNICs need high internal bandwidth to handle the increasing external bandwidth they are seeing. Some estimates suggest that internal data movement in a SmartNIC needs to be 10x the external rate in order to smoothly handle the functions they are asked to perform. The 2D network on chip (NoC) that is used by the Achronix Speedster7t has 2 vertical NOC lanes for each Ethernet controller. These lanes each operate at 512Gbps, servicing an Rx/Tx pair (400Gbps/ea).

For receive, network traffic moves easily through the onboard Ethernet SerDes and PCS/MAC layer onto a 2D NoC column.  In the FPGA fabric there is a receiving Rx engine that processes the packets and forwards them along a horizontal NoC row to the matching engine. After this, packets are moved via NoC to a DMA engine for conversion to PCIe buffers. After moving through another vertical NoC column, the packets move to the PCie controller and SerDes.

Virtualization and SD Overlay Networks add complexity to Rx/Tx and matching engines. There can be larger block sizes in these environments. With all this comes increased on-chip traffic. While the overlay network may appear less complex, the data movement on the underlay network can become quite complex. Physical SmartNICs will see heavier loads and more throughput as a result.

Scott talks about the reasons that security, filtering, encryption and key management make single chip SmartNICs more attractive. Each of these activities is necessary and growing more challenging in networks today. For instance, filtering in the matching engine requires deep packet inspection, tagging, rewriting packet headers and unwrapping & wrapping packets, etc. At the same time the SmartNIC needs to offer full support for key management and encryption/decryption for VPN tunnel termination.

Scott also touches on the changes coming with CXL and NVMe during the webinar. He also makes the case that the continuing move to higher bandwidth network interfaces and changes in applications, such as VMs will call for higher throughput and flexibility. All of the above factors play an important role in driving the preferred architecture and the specific implementation for SmartNICs. Achronix’s use of a 2D NoC with their programmable FPGA fabric offers impressive data handling capabilities to meet these needs.

Their 2D NoC offers 20 Tbps aggregate on-chip bandwidth. Each vertical and horizontal bus handles 512 Gbps in a matrix that covers the FPGA fabric. There are numerous Network Access Points (NAPs) for on and off-loading data to the NoC. Scott points out that if each packet moves through 4 processing blocks, 3.2 Tbps would be needed with 4 x 400 GbE. Scalability and future proofing could call for 10x that.

This webinar offers a stark view of the needs of SmartNICs today and in the future. Historically they might have started off as handy assistants to simplify operations on hosts CPUs. It is clear that SmarNICs are becoming more and more the center of gravity for complex network applications. The full webinar is available for viewing on the Achronix website.