Improve Your Memory the Sonics Way

There is never enough memory bandwidth. Well, occasionally there is but many SoCs have lots of blocks that communicate through memory, typically off-chip DRAM. In 2001 Sonics created their first solution to this problem with MemMax technology that was incorporated into their SonicsSX product. This has been used in over 100 designs including the major video games and the largest manufacturer of digital TVs. Flagship products.

Since then Sonics have released their SonicsGN product and today they have incorporated their memory access technology into this newer technology. I talked to Drew Wingard, Sonics’s CTO about it.

The challenge with memory access is not just that you want to keep high throughput to the memory but that there are lots of blocks communicating with memory that have different requirements. GPUs need to have high throughput or there will be flicker in the graphics, communications need timely access or packets will be dropped, these are quasi-real-time deadlines. With the main processor latency is the big issue. By the time the processor tries to access DRAM, it has already missed in one or more caches and the processor is probably stalled until the memory line arrives.

So there need to be quality of service (QoS) guarantees not just at the level of overall access to memory but for individual blocks. The alternative is to have additional buffering so that, for example, instead of that graphics frame appearing right now it appears 10ms later. You won’t notice but it wastes chip area and power. This is analogous to the way video streaming like Netflix buffers at least a few seconds to cope with unpredictable internet performance so hopefully the screen never freezes, except at the chip level. The key to this is that different memory accesses have different requirements, plus they are not issued in the order in which they need to be services.

To make things worse, the number of subsystems on an SoC that need access to memory has exploded. This is implemented through virtual channels between the blocks on the chip and the off-chip memory. Flow-control is handled on a per-channel basis so that one virtual channel doesn’t block another. For example, bursty traffic on a PCIx shouldn’t make the video flicker.

There are three different traffic classes for channels with different QoS requirements:

• high (low latency)
• guaranteed latency (audio, video, isochronous data)
• best effort

Of course if there are too many requests then eventually the NoC is being asked to do the impossible. But when the hardware detects that something is past its guarantee it doesn’t simply block it but demotes it to best effort.

Of course doing QoS at the network level is equally valuable for other forms of shared memory. In some applications, where there is not enough bandwidth to off-chip memory, it becomes attractive to add embedded SRAM to offload some of that bandwidth to on-chip memory, but this still needs the same type of end-to-end QoS guarantees.

These new capabilities mean that SonicsGN supports the latest DDR4 and LPDDR4 memories with the highest performance. It also fully supports the multi-threading capabilities of the Open Core Protocol (OCP) interface that reduces contention and increases performance.

The lower power demands of SonicsGN means that this scales down to very low-power small-footprint chips in internet of things (IoT) type applications such as medical and wearables.