What NoCs with virtual channels really do for SoCs

Most of us understand the basic concept of a virtual channel: mapping multiple channels of traffic, possibly of mixed priority, to a single physical link. Where priority varies, quality of service (QoS) settings can help ensure higher priority traffic flows unimpeded. SoC designers can capture the benefits of virtual channels inside a chip with network-on-chip (NoC) strategies.

In theory, a NoC structure maintains a low latency while allowing IP blocks to initiate different classes of traffic to various destinations. A prioriknowledge about traffic certainly helps design of any network topology. A small SoC with a NoC handling separate traffic classes – real-time, best effort, and mixed – might look like this at the conceptual level:

In practice, two problems develop as SoC complexity increases. The first is deadlocking. If the NoC is an addressable mesh, and there are multiple paths between IP blocks, a deadlock can result when one block reaches around the mesh and modifies something another block needs for before proceeding. According to CTO Drew Wingard of Sonics, that was the original value proposition for virtual channels, incorporated in the first release of Sonics NoC technology in 1997. One could use virtual channels to avoid deadlocking, perhaps by doing something as simple as setting up north-south traffic on even channels, east-west traffic on odd channels.

The second and bigger problem in the real world is floorplanning, which Sonics has also studied carefully. A NoC takes up actual real estate, and requires actual retiming logic. (So much for “it’s just software.”) In real-world scenarios, the longest wires other than clock and reset signals are those within the NoC. If one were very lucky, routers could be placed strategically next to IP blocks of the same traffic class:

There are more considerations. IP blocks aren’t the same size, some are in hard macros, many have to be placed next to I/O pins and other blocks for optimum signal routing, some live in separate power and clock domains, and other details. Let’s look at what happens when just the simplified blocks are jostled around without these other factors involved:

By putting NoC routers out nearer IP blocks, congestion takes over the center of the chip with very long wires. Compared to the “lucky” floorplan, wire length went up over 50% and retiming gates more than doubled. The tendency for designers in fixing that is to push the routers closer together, what Wingard calls “the big hairy golf ball”. This simplifies the router interconnect somewhat but does not do much for reducing retiming logic – essentially forcing it back to each IP block, even more so if their output is not serialized.

The solution is to keep the NoC routers back out nearer the IP blocks, hooking them to their nearest router neighbor. Then, sort out traffic classes between the routers using virtual channels, keeping the arbitration simple and deferring decisions until the last hop. This eliminates much of the retiming circuitry required – back to the “lucky” case. It only increases wire length slightly over the “lucky” case, with the wire length contained in the NoC. The savings over the realistic case: 25% less wire length, and 60% less retimer area.

These NoC virtual channel advantages scale rapidly in larger, more complex designs with perhaps 100 IP blocks, where the retiming real estate adds up quickly. Consider the issues without using virtual channels. Adding routers increases latency with more hops, and it also can set off a vicious cycle of added retiming stages that cascades into problems with real estate and closure. If the floorplan topology iterates, or congestion arises in the NoC itself, more retiming circuitry is needed and adjustments propagate across the chip.

Uncharacteristically good results with a NoC in the early stages of a SoC design can be deceiving as the floorplan grows and evolves. After a lot of fiddling with retiming and little luck, the last resort to get a NoC design without virtual channels to close is often to turn down the clock – a bitter compromise.

Virtual channels separate traffic in time instead of space. A Sonics NoC with virtual channels simplifies interconnect and reduces retiming requirements, while keeping both real-time and best effort traffic flowing.

More articles from Don…