CEO Interview: Patrick T. Bowen of Neurophos

Patrick is an entrepreneur with a background in physics and metamaterials. Patrick sets the vision for the future of the Neurophos architecture and directs his team in research and development, particularly in metamaterials design. He has a Master’s degree in Micro-Nano systems from ETH Zurich and PhD in Electrical Engineering from Duke University, under Prof. David Smith. After graduation, Patrick cofounded Metacept with Prof. Smith; Metacept is the world’s foremost metamaterials commercialization center and consulting firm.

Tell us about Neurophos. What problems are you solving?
We say we exist to bring the computational power of the human brain to artificial intelligence. Back in 2009 it was discovered that GPUs are much better at recognizing cats on the internet than CPUs are, but GPUs are not the answer to the future of AI workloads. Just as GPUs were better than CPUs for neural networks, there could be architectures that are better than GPUs by orders of magnitude. Neurophos is what comes next for AI after GPUs.

AI large language models in general have been limited because we haven’t had enough compute power to fully realize their potential. People have focused primarily on the training side of it, just because you had to train something useful before you could even think about deploying it. Those efforts have highlighted the incredible power of large AI models, and with that proof people are starting to focus on how to deploy AI at scale. The power of those AI models means we have millions of users who will use them every day. How much energy does it cost per user? How much does the compute cost per inference? If it’s not cheap enough per inference, that can be a very limiting thing for businesses that want to deploy AI.

Energy efficiency is also a big problem to solve. If you have a server that burns say 6 kiloWatts, and you want to go 100 times faster but do nothing about the fundamental energy efficiency, then that 6 kiloWatt server suddenly becomes a 600 kiloWatt server. At some point you hit a wall; you’re simply burning too much power and you can’t suck the heat out of the chips fast enough. And of course there are climate-change issues layered on top of that. How much energy is being consumed by AI? How much additional energy are we wasting just trying to keep data centers cool? So, someone needs to first solve the energy efficiency problem, and then you can go fast enough for the demands of the applications.

People have proposed using optical compute for AI for nearly as long as AI has existed. There are a lot of ideas that we work on today that are also old ideas from the 80s. For example, the original equations for the famous “metamaterials invisibility cloak”, and other things like the negative index of refraction, can be traced back to Russian physicists in the 60s and 80s. Even though it was sort of thought of, it was really reinvented by David Smith and Sir John Pendry.

Similarly, systolic arrays, which are typically what people mean when they say “tensor processor”, are an old idea from the late 70s. Quantum computing is an old idea from the 80s that we resurrected today. Optical processing is also an old idea from the 80s, but at that time we didn’t have the technology to implement it. So with Neurophos, we went back to reinventing the optical transistor, creating from the ground up the underlying hardware that’s necessary to implement the fancy optical computing ideas from long ago.

What will make customers switch from using a GPU from Nvidia, to using your technology?
So, the number one thing that I think most customers care about really is that dollars per inference metric, because that’s the thing that really makes or breaks their business model. We are addressing that metric with a solution that truly can increase the speed of compute by 100x relative to a state of the art GPU, all within the same power envelope.

The environmental concern is also something that people care about, and we are providing a very real solution to significantly mitigate energy consumption directly at one of its most significant sources: datacenters.

If you sit back and think about how this scales… someone has to deliver a solution here, whether it’s us or someone else. Bandwidth in chip packaging is roughly proportional to the square root of the area and power consumption in chip packaging is generally proportional to the area. This has led to all sorts of contorted ways in which we’re trying to create and package systems.

Packaging is one of the things that’s really been revolutionary for AI in general. Initially it was about cost and being able to mix chiplets from different technology nodes, and most of all, about memory access speed and bandwidth because you could integrate with DRAM chips. But now you’re just putting more and more chips in there!

Using the analog compute approach restores power consumption for compute down to the square root of area instead of proportional to area. So now the way in which your compute and power consumption scales goes the same way; you are bringing them into balance.

We believe we’ve developed the only approach to date for analog in-memory compute that can actually scale to high enough compute densities to bring these scaling laws into play.

How can customers engage with Neurophos today?
We are creating a development partner program and providing a software model of our hardware that allows people to directly load PyTorch code and compile that. That provides throughput and latency metrics and how many instances per second etc. to the customer. It also provides data back to us on any bottlenecks for throughput in the system, so we can make sure we’re architecting the overall system in a way that really matters for the workloads of customers.

What new features/technology are you working on?
Academics have for a long-time sort of dreamt about what they might do if they had a metasurface like we’re building at Neurophos, and there are lot of theoretical papers out there… but no one’s ever actually built one. We’re the first ones to do it. In my mind most of the interesting applications are really for dynamic surfaces, not for static, and there is other work going on at Metacept, Duke, and at sister companies like Lumotive that I, and I think the world, will be pretty excited about.

Why have you joined the SC Incubator and what are the Neurophos’ goals in working with their organization over the next 24 months?

Silicon Catalyst has become a prestigious accelerator for semiconductor startups, with a high bar for admission. We are excited to have them as a partner. Hardware startups have a big disadvantage relative to software startups because of their higher demo/prototype cost and engineering cycle time, and this is even more true in semiconductor startups where the EDA tools and mask costs and the sheer scale of the engineering teams can be prohibitively expensive for a seed stage company. Silicon Catalyst has formed a pretty incredible ecosystem of partners that provide significant help in reducing their development cost and accelerating their time to market.