Semidynamics: A Single-Software-Stack, Configurable and Customizable RISC-V Solution

Founded with a vision to create transformative, customizable IP solutions, Semidynamics has emerged as a significant player in the AI hardware industry. Initially operating as a design engineering company, Semidynamics spent its early years exploring various pathways before pivoting to develop proprietary intellectual property (IP) around 2019. With financial support from the European Union, they began by creating highly efficient Core and Vector Units, receiving recognition from the tech ecosystem.

Over the past year, Semidynamics has made several announcements highlighting their technology advancements and partnership engagements to support many fast growing market segments. Their signature value proposition is a versatile “All-in-One IP” solution equipped to meet the demands of modern AI applications.

During the RISC-V Summit 2024, I sat down with Roger Espasa, Founder and CEO of Semidynamics to receive a holistic update. The following is a synthesis of that discussion.

A Unified, Programmable Solution

The heart of Semidynamics’ innovation lies in its commitment to a single software stack approach. In an industry where heterogeneous SoC (System on a Chip) architectures often combine CPUs, GPUs, and NPUs from multiple vendors, each with its own software stack, Semidynamics offers a streamlined alternative. By uniting Core, Vector, and Tensor processing units under a single software stack, they eliminate the inefficiencies commonly associated with multiple software stacks that rely heavily on data orchestration through Direct Memory Access (DMA) operations.

This unified solution is built on the RISC-V open-source architecture, ensuring adaptability and control. Semidynamics’ RISC-V-based architecture enables seamless communication between the Core and specialized units, allowing everything to run smoothly as a cohesive program. This differs from traditional designs where data is sent, processed, and returned in a fragmented sequence, leading to latency issues. Customers have responded positively to this innovation, appreciating the streamlined programming experience it provides.

Key Components of Semidynamics’ IP Solution

Core, Vector, and Tensor Units

Semidynamics’ “All-in-One IP” integrates three essential processing units—the Core, Vector, and Tensor units—working in harmony. While the Core handles general-purpose processing, the Vector unit manages 32-bit precision activations, and the Tensor unit is optimized for smaller data types, crucial for AI tasks like matrix multiplications in transformer models. The system dynamically balances workloads across these units to maximize performance.

Gazillion Misses™ IP

The Gazillion Misses IP is a specialized feature within their CPU Core that ensures high data availability for AI applications. With AI models requiring vast amounts of data, caches alone cannot keep up. Gazillion Misses IP addresses this challenge by continuously requesting data from the main memory, ensuring that both the Vector and Tensor units remain active and data-ready, a capability essential for managing complex models like transformers.

Out-of-Order Processing with Atrevido

In response to the growing demands of transformer-based models, Semidynamics offers an Out-of-Order processing architecture, dubbed “Atrevido.” This architecture ensures that, even as data demand surges, the processing units do not suffer from data starvation, maintaining smooth and efficient operation.

Configurability and Customization

Recognizing that each customer’s requirements vary, Semidynamics offers both configurable and customizable IP. Configuration involves selecting from standard options like cryptography and hypervisor support, while customization entails crafting special instructions based on customer requirements. This flexibility allows Semidynamics to serve a broad range of applications, from high-performance computing (HPC) to low-power security cameras.

RISC-V: The Backbone of Semidynamics’ Approach to Open Standards

Semidynamics’ choice of RISC-V as the foundation of their technology aligns with a broader industry shift towards open-source architectures. Similar to the freedom Linux brought to software, RISC-V liberates hardware developers from proprietary constraints. However, with the high costs associated with hardware tapeouts, choosing a solution partner becomes critical. Semidynamics not only brings flexibility and control to hardware but also future-proofing by grounding their technology in a general-purpose core that can adapt to new algorithms as they emerge.

Practical Engagement with Customers

Beyond IP delivery, Semidynamics ensures their clients have hands-on access to the technology. Once the RTL (Register Transfer Level) is delivered, customers can begin working immediately, with the option of testing on their multi-FPGA emulation platform. This engagement model accelerates integration and allows clients to adapt the IP to their needs in real-time.

Business Model

Semidynamics employs a straightforward business model that includes a licensing fee, a maintenance fee, and royalty options. This flexible structure ensures that customers pay for what they need, aligning the financial model with the technical customization Semidynamics provides.

A Blended Talent Pool

Based in Barcelona, Semidynamics boasts a team that combines industry veterans from companies like Intel, DEC, and Broadcom with young talent trained through rigorous immersion. This blend of experience and fresh perspectives ensures that the company remains innovative while drawing on deep industry knowledge.

Future-Proofing AI Hardware

AI hardware is undergoing rapid evolution, driven by emerging algorithms and models that challenge traditional computing frameworks. Semidynamics’ approach—anchoring Vector and Tensor units under CPU control—ensures that their IP can adapt to future AI trends. This scalability, combined with their focus on programmability, positions Semidynamics as a forward-thinking solution provider in the AI hardware space.

Summary

Semidynamics “All-in-One IP” solution strategically combines CPU, GPU, and NPU processing capabilities into a unified RISC-V architecture to meet the increasing demands of AI, machine learning, and edge computing. By implementing a single software stack, Semidynamics enables seamless control over Core, Vector, and Tensor units, minimizing the need for fallback to the CPU. This approach ensures efficient task distribution across specialized units and directly addresses the performance limitations highlighted by Amdahl’s Law, which focuses on bottlenecks from tasks that cannot be parallelized.

To prevent memory access issues that can slow down AI applications, Semidynamics developed Gazillion Misses™ technology. This technology continuously feeds data to the Vector and Tensor units from main memory, reducing idle time and supporting high-throughput processing, even for large, complex AI models. By combining a unified software stack, advanced memory management, and a customizable architecture, Semidynamics delivers an adaptable solution for various AI and HPC workloads, providing efficient, scalable, and future-ready performance.

To learn more, visit https://semidynamics.com/en

Also Read:

Gazzillion Misses – Making the Memory Wall Irrelevant

CEO Interview: Roger Espasa of Semidynamics

Semidynamics Shakes Up Embedded World 2024 with All-In-One AI IP to Power Nextgen AI Chips