Based on Dylan Patel’s SEMI Industry Strategy Symposium (ISS): Tokens to Infrastructure presentation, one of the most important themes is the emergence of the AI Economic Stack, where every layer of artificial intelligence—from semiconductor manufacturing to cloud infrastructure, model providers, and applications—is becoming tightly coupled through compute economics. The presentation argues that AI is no longer simply a software business; it is an industrial-scale infrastructure business driven by power, silicon, memory, networking, and datacenter investment.
AI Infrastructure: The New Industrial Revolution
The AI industry is increasingly defined by the conversion of capital expenditure into tokens, the fundamental unit of AI output. SemiAnalysis describes this process as a “token factory,” where GPUs, networking equipment, memory systems, power infrastructure, and software stacks work together to generate AI inference and training tokens. The economics begin with expensive accelerator systems such as NVIDIA H200-class servers, where capital costs, depreciation, electricity, and colocation expenses determine the cost per generated token. As models become larger and user demand increases, infrastructure efficiency becomes a primary competitive advantage.
The presentation highlights that inference costs are rapidly becoming the dominant expense for AI companies. Unlike traditional software businesses, where serving an additional user is nearly free, AI companies incur a direct compute cost for every token generated. This creates a unique economic model in which revenue growth is closely linked to compute consumption. OpenAI, for example, is projected to see inference expenses rise nearly linearly with revenue growth, making hardware optimization and datacenter efficiency critical to profitability.
The Importance of Memory and HBM
A major bottleneck identified in the report is High Bandwidth Memory (HBM). SemiAnalysis forecasts that HBM shortages could persist through 2027 while demand continues to exceed supply. At the same time, the transition of memory manufacturers toward HBM production may worsen shortages in conventional DRAM markets. This “double-shortage dilemma” reflects how AI demand is reshaping the entire semiconductor ecosystem. HBM is essential because modern AI accelerators require massive memory bandwidth to feed increasingly large models efficiently. Without sufficient HBM availability, even advanced GPUs cannot reach their full performance potential.
Compute Becomes the Strategic Asset
The presentation also demonstrates how leading AI laboratories are securing unprecedented amounts of compute capacity. OpenAI’s contracted compute supply is expected to grow dramatically over the next several years, measured in gigawatts of datacenter power. Similarly, major cloud providers and AI labs are signing multi-billion-dollar agreements for GPU capacity. SemiAnalysis notes that AI cloud contracts commonly generate annual revenues of $10–13 per watt for providers over multi-year periods, creating attractive returns for infrastructure investors.
These agreements show that access to compute is becoming as strategically important as access to talent or intellectual property. Companies such as Microsoft, OpenAI, Anthropic, Meta, and xAI are effectively competing for future compute resources years in advance. The result is a new economic model where hyperscalers, AI labs, and semiconductor suppliers are increasingly intertwined.
Why It Matters
The significance of this transformation extends beyond the semiconductor industry. AI is creating a demand cycle that spans silicon design, foundries, advanced packaging, memory manufacturing, networking equipment, power generation, and datacenter construction. Every improvement in model capability drives additional demand for compute infrastructure, which in turn drives investment throughout the supply chain.
For investors, this means that value creation is no longer concentrated solely in AI software companies. Semiconductor manufacturers, memory suppliers, networking vendors, and datacenter operators may capture significant portions of AI-driven growth. For governments, the report highlights the strategic importance of energy availability and semiconductor manufacturing capacity. For enterprises, understanding token economics becomes essential for evaluating AI deployment costs and long-term return on investment.
Bottom line: Patel’s thesis is that AI should be viewed as a vertically integrated industrial ecosystem. Success will depend not only on better algorithms but also on securing memory, power, compute, and infrastructure at scale. The companies that master this full stack—from silicon to tokens—will define the next decade of technology leadership.
Also Read:
A tower-like heterogeneous packaging architecture for the AI era
Agentic AI and the Future of Chip Design: From Productivity Tool to Engineering Partner
Disaggregating AI Compute to Break the Tokens Barrier
Share this post via:
Intel 18A vs Intel 18A-P: What Is the Difference and Why Does It Matter?