The Biggest Chip Ever Built — Why OpenAI Runs On It | Cerebras CEO Andrew Feldman

Yeah, what happened, I think, was for a long time AI was sort of a novelty, right? It was like a parlor trick. It was cool, but not useful. And what happened somewhere around the middle of 2025 was the AI got smart enough such that people began to use it. And remember, we make AI with training, but we use it with inference. And suddenly people wanted to use it. And the minute you want to use it, the minute it's productive, right?

Speed matters, right? Fast tokens are more productive. And so the conversation moved from everything else to how do we make our inference faster? How do we deliver tokens more quickly? Because those are more productive tokens. We get more done in less time, and therefore it's more valuable.

The right metric is tokens per second per user. That's how fast you get the first token all the way through the last token in your response. And it's true for queries from chat, but it's also true for agentic flows, right? If they're sort of multi-cycle turns, waiting is amplified. And so what you want is blisteringly fast responses so that the AI feels like it's in real time. You can engage with it.

I think that's right. And I think that's a very good analogy. I think if you think of something like Netflix, right? When the internet was slow, right, Netflix delivered DVDs in envelopes. You would get a DVD in an envelope. And when the internet became fast, they didn't get more efficient at delivering DVDs in envelopes. They became a movie studio, right? The speed enabled them to become something completely different. And that's what speed does in general and in particular for AI.

It opens up a whole new domain. It allows you to use the AI differently. You will stay longer, you will come more often, and you'll work on harder problems.

Yeah, that's right. I mean, how big is the market for slow search? How big is the market for dial-up? It's zero. How long will you wait for a website to resolve? Will you wait eight seconds? Nobody waits. And so it's the exact same with AI.

That's right. While it's running and running and running, I think that is not what people want.

Well, I think there used to be one chip to do it all. It was called a CPU. Yeah, right. And there emerged a coprocessor to do discrete graphics. And as the AI workload became interesting, we focused chips more and more on that particular workload. And today, several companies make traditional GPUs, so NVIDIA and AMD. They make very standard GPUs. There are a group of companies—the hyperscalers—that make some of their own parts. So, the TPU.

TPU is Google, and Trainium is AWS. And then there were a group like Cerebras, and we were among the pioneers to build a part from scratch optimized for AI and nothing else. And that we weren't inside of a hyperscaler, we weren't optimized for a hyperscaler's problem or for one lab's problem. We were building a chip for a collection of AI problems, and all our thinking was around how to accelerate AI. And that's sort of the landscape to this day.

An ASIC is an application-specific integrated circuit. And it's a word that now has a wide range of meaning. It means you have made a series of choices away from general towards a narrower class of problem solving, and that you've made some decisions in your architecture that make it much better at some things and much, much worse at others. Right? And that's choices that are made across the spectrum. So the TPU has made some choices like that. It can't do graphics.

It's very good at matrix multiply. It's not very good at a collection of other things that we do in mathematics. Same for the GPU. We've all made different choices. Right now in production, there are NVIDIA and AMD. There are a collection—there's the TPU for Google, Trainium. Just coming up is a Maia part, is a part from Microsoft, Cerebras, and there was one other: Groq, that got acquired by NVIDIA. Yeah.

Yeah, I think one of NVIDIA's sort of most durable moats was the perception that the GPU could do everything and it was all you needed for AI. And the acquisition of Groq for $20 billion and the structure and the speed with which they chose to do it made clear to everyone that that wasn't true, that the GPU architecture could not do fast inference, and that this market was large and growing quickly. And we were the fastest at it and the largest.

And our sales were more than 10 times Groq's, and they paid $20 billion for the number two player. So that was a good day. That was a good day.

Right. Groq.

That's right. Jalapeño is a part that has been a long time coming. It was announced with Broadcom before OpenAI did the deal with us. Remember, we did a huge deal with OpenAI. This is probably one of the largest deals in Silicon Valley history, north of $20 billion. I think they have a yawning need for silicon. And one of the things that OpenAI has been, I think, the best in the industry at is looking at an exponential curve adoption rate of growth and not being afraid of what it says.

Others have had to go out and strike really bad deals to get capacity, where OpenAI saw this coming. They struck big deals for memory, for compute with us, with others. They've really been sort of visionary in understanding what it means to extrapolate from an exponential curve. And that curve is AI usage, right? It is growing so unbelievably fast. The whole industry is chasing demand.

We usually—well, often it's the other way. Often people are building it, hoping it will come. And in our case, all of us are chasing what people already want to do, let alone what they might do in the future. Hey, Ed.

Yeah. Multi-silicon environments are a healthy ecosystem, right? I don't think anybody would say that the x86 environment was a healthy place. For 20 years, there were two players. And if you notice, when a new workload came around, the cell phone workload, that was very similar but required low power and battery, both lost, right? Intel got zero share. AMD got zero share. And a company nobody had previously heard of became the largest seller of compute in the world, and that's ARM.

And that healthy ecosystems have lots of different ways to solve problems.

I don't know that that divide is real. I think they are an industrial adversary. I think they have made really interesting investments. They've made investments in power, in their grid, which is a real weakness in the U.S. I'm sure in France you have nuclear, which sort of turns out to be pretty cheap power and pretty clean. But in China, they've made huge investments in their grid. They are behind in chips, but their approach at the next level is open-source models, where they're producing some extraordinary models.

Not as good as GPT or Anthropic or Google's Gemini, but very good.

And I think they don't have the same chip strengths that the U.S. does, but they've got leadership in other domains, particularly in power, which is what we need for data centers.

We don't sell.

For regulatory reasons and for some geopolitical reasons. Okay.

Yeah, I think if you look at the way the ecosystem for apps and cloud emerged, everything you can do, you should do on your phone or your laptop. But the ability to get real processing power to a phone or to a laptop is constrained because they're generally working off a battery. And so you want to do as much as you can as close to the data as you can. But the truth is, in most situations, for real compute, you have to go up to the cloud, to the data center.

And that's exactly the way it's going to be with AI. We're going to do a lot of work on the cell phones, on the laptop, but for the big work, you're going to go to the data center. And that's where our focus is. Our focus is data center compute for AI.

Okay. I think that if you spend a lot of time staring at the public markets and watching their ups and downs every day, you're watching the wrong thing. The wisdom received from the great thinkers of public market investing is that, in the short term, the public markets are voting mechanisms: who's most popular? But in the long term, they're weighing mechanisms: who's created the most value, the most weight? And so what we can do as an early public company is, we cannot look at the day-to-day fluctuations, and we focus on just building extraordinary technology, winning new customers, making our existing customers happy, ensuring they buy more, being better every week.

And so I think the day-to-day fluctuations are out of our hands, and that's about the voting.

That's not exactly our experience. I mean, obviously we have enormous demand from OpenAI, but we have huge, dozens of other customers who are trying to place very big orders on us. And historically, bubbles were when supply got out ahead of demand, right? In the '90s, we built out telco infrastructure, right? We built out fiber years before it was going to be used, and it took six or eight years, and it all got used. But it was a sort of, if you build it, they will come mentality.

Whereas what's different about AI right now is we're all trying to catch up. We're trying to build data centers faster. We're trying to increase our supply chains for demand that's already here today. And only a very small portion of the world are using AI anywhere close to its potential. And we're already sort of overwhelmed with compute. We're overwhelmed with the demand for memory, which is a real weakness in the GPUs. It's not a problem we face.

And the ecosystem, there are constraints left and right. And that doesn't feel like a bubble.

So there are three major bottlenecks right now. The first is all GPUs and all ASICs except us use a type of memory called DRAM and a particular flavor of that memory called HBM. And that memory is made by three companies in the world: SK Hynix, Samsung, and Micron. Three companies. And they're sold out. That's the number one problem. We don't use it.

Build more factories. And so you have this huge problem for GPUs and even now for CPUs, in which you can't get this memory because of our architectural choices. We don't use it. The second constraint was a process at TSMC. The process was called CoWoS, and this used silicon as a motherboard. And this is what NVIDIA and AMD and others use as a motherboard. So they put their chips and the memory chips on it, and it's more efficient than a traditional motherboard.

That process is sold out, is highly constrained. We don't use it. Third limitation in the space is 3-nanometer fab space at TSMC. So this is the most aggressive factory, with the most advanced technology. Our chips are at 5 nanometers. So we don't use the 3-nanometer technology.

That's right. They build factories, and the factory etches transistors that are a certain distance apart. And the smaller the distance apart, the more transistors per square millimeter, and the more you can do per square millimeter. And so the history of the computer industry making chips has been, we've gotten better and better at putting transistors closer and closer.

And so right now, the bulk of the GPU world is all at 3 nanometers, and there's tremendous congestion at the factory.

Tooling at the factory. And we use the 5-nanometer factory, and this is our chip here.

This is the largest chip built in the history of the computer industry. It's 58 times larger than a GPU. It has 2,500 or 3,000 times more memory bandwidth.

And for AI, bigger chips process information more quickly, and therefore you get answers in less time. And obviously, you don't want a bigger chip for a laptop or a cell phone or for lots of other things, but for AI work, big chips are undoubtedly the best way to go.

So agentic AI is a world in which AI doesn't just provide answers, it initiates action. So that action might be: go to a website, learn, gather some data from a website, bring it back, take another action. Those actions are done by CPUs. And so, as AI gets better and better at doing things, at making instructions, calls for things to get done, we're using more and more CPUs, right? And that is driving up the consumption of CPUs and therefore the demand for CPUs.

And so this sort of huge push for more CPUs is being driven by AI on machines like ours and GPUs doing agentic work and asking the CPUs to take an action: to go to a website, to order a burrito, to find a piece of information, to pull it from storage. All that work is being done by the CPUs.

Including in a system like ours.

Yeah.

It means that AI, when we do AI in agentic, the AI processor is like the brains and the CPUs are like the body. They're taking action, they're doing things in the digital world under the direction of the AI, which is running on the accelerator, on the Cerebras system. And so, as we do more and more AI work and more and more agentic work, we're making more and more calls to CPUs, and therefore the demand for the CPU is just through the roof.

Same memory shortage. It's the exact same memory. Oh, that's right.

The way you have perfect timing is to have horrible timing for 10 years, so you have perfect timing.

2016.

It was all vision.

It was all vision. And that's why we don't believe the right thing to do is to embed the latest and coolest model into your circuitry. That's a mistake. We're the fastest in the world at transformers, and our architecture was set before transformers existed.

We're the fastest in the world at diffusion, and our architecture was set before diffusion. What you want to do is get the underpinnings of those so that, when the market moves, you can be good at that as well. Otherwise, because of the delay, short life.

Some have, some have. And historically, that's been a structural mistake.

You want to think about what is the underlying calculation. The underlying calculation of all this work is sparse linear algebra. And if you can accelerate that, whatever the model builders invent, you can make faster. And that was our approach.

I think the lessons are large and many. I think experience is another name for having made mistakes and learning from them, right? I think we, as a team, have built dozens of chips over the past 25 years, and the returns to experience in chipmaking are enormous. And we built a different type of computer at SeaMicro, a type of computer optimized for low power and optimized for a workload that was very different than AI, for something like web browsing.

But the fundamental underpinnings, the questions you ask as a computer architect, are always the same. What can I do to make this work faster? And is there enough of it to make it worthwhile? These are the two questions we ask. Should we build a part for it? What could we do to build a chip optimized for AI? And will there be enough AI so that you can build a business around it? Those are the questions we asked in 2016.

And the flip side of that was: wouldn't it be a surprise if the GPU, which had been optimized for graphics for 20 years, had been pushing pixels to a monitor, was suddenly good at a new world?

Wouldn't that be serendipitous? And we came to believe that it wasn't the right architecture for it. It was just better than the CPU, and that we could build an architecture that would be vastly faster, that would use less power, and could drive down the cost of it.

Then we wandered. Then we wandered in the desert.

Yeah.

So first, we, at the beginning, were honest with our VCs and told them we were going to attack a really hard problem. We weren't going to build something that was a little bit better than a GPU. And our idea, our strategy, was that you will never beat a great company like NVIDIA by doing something a little bit better than they do. They're going to buy everything for less. They're going to have pricing pressure. They're going to be able to bundle.

And the right strategy would be to do something incredibly hard in engineering that was way better—10, 15, 20, 30, 50 times better. But to do that, the ordinary and obvious paths are all closed. Everybody else has taken them already. And so what we observed was that speed in inference was going to be a function of memory, and that there were two types of memory. There's this DRAM, or HBM, and they can store a lot, but they're slow. There's another type of memory called SRAM that is unbelievably fast, but per unit area can't store very much.

And that for graphics, everybody had always used DRAM. They used HBM, and the AI workload was fundamentally different. In graphics, you move data to the GPU and then you work on it for a long time, and then you send the result. So the time spent in total of movement plus work is dominated by work. In inference in AI, it's the exact opposite. You move a huge amount of data, all the weights from memory to compute, and you do one calculation to generate the next word.

And then you have to do it again.

So all the time is dominated by the movement of data. So that's why GPUs have so much trouble being fast.

So we observed that if we chose a strategy using SRAM, we could be faster, but then we have to overcome the trade-off with SRAM that it can't store very much. That led us to the solution that if we could build a part vastly larger than any part in history—the size of a dinner plate—we could stuff it to the gills with SRAM and thereby get the benefit of SRAM, that it was fast, and overcome the weakness that it can't store very much.

And that led us to a strategy called wafer-scale. This chip is made from a single wafer. It comes out of TSMC.

A wafer—all chips are cut from a wafer. A wafer is a circular piece of silicon that's 300 millimeters across, and the process of chipmaking stamps out, like your mother does with a cookie cutter, stamps out chips. The biggest chip that had ever been built before us was 800 square millimeters, 840 to be exact, and this is 46,000. So we had to invent all sorts of new technology to build a chip this big. And once you build it, you have to invent ways to power it, to cool it. There are no vendors waiting for you.

Right. Because you look like nothing else ever made. And that took years. And it was a deep-tech problem. It had never been solved before. And we had an 18-month period where we were spending $8 million a month and we couldn't build them. So if you're deep-tech founders, you have $8 million a month.

Because what everybody thought was hard, we solved quickly. And what nobody else knew about, because they'd never actually done it, turned out to be really hard. Imagine, I tell people, imagine the first group that was going to climb Everest, and they're at base camp and they're having tea with a group that just failed. And the group that just failed says, halfway up, there's this part. It's unbelievably hard. We couldn't do it. Okay. Your team climbs up, makes it all the way to the top, comes back.

They're having tea again. And the team that made it leans over to the team that hadn't made it and says, that part in the middle, that wasn't the hard part, because nobody had gotten past it. Nobody had gotten past certain things, so they didn't even know what to be afraid of.

We now know, and it was something—a step called packaging—and that's how you affix a wafer to a motherboard, how you deliver power to it, and how you cool it. And nobody had done it before. And over that 18-month period, we became the best in the world at it from approximately zero.

And we did that by failing again and again and using good engineering methodology and doing a failure analysis of every single failure. So we failed differently again and again and again and again. And we told our board—we met with our board every six weeks—this is the strategy. Nobody's ever done this before. This is what we're going to do. And we had to invent new materials. We had to invent new techniques. We ended up building things that everybody else had partners who could do.

But it all gets to 2019, when Alex Wood solved it. And my co-founders and I, the first time it worked, were sitting in a tiny little lab, and we couldn't believe it. We were the first people in history to make one work. And we just stared at it. Watching a server run is about as exciting as watching paint dry.

And there we were, the five of us just staring at this machine, not believing it might work. We'd made it work.

Emotionally, it was a completely different thing. It was that we had made our idea work. And I think for deep tech founders in particular, there's always this little thing in the back of your mind that says, maybe it's shit.

That's right. Maybe it's going to fail, and maybe it's not going to work. And maybe I don't have time. Maybe we're going to run out of money. Maybe, maybe, maybe. And the flip side of that is the joy that this was my co-founders' ideas. I mean, these are their ideas manifest in the world, and that is an extraordinary thing. That is when someone's ideas take physical shape. And then the next step is when you watch other people's work sit on top of your idea. Then you know you love making infrastructure.

And so when we rang the bell and we went public on May 14th this year in the largest semiconductor IPO in history, we did something unusual. We invited all our engineers who'd been with us since the start, everybody who'd been with us more than nine years, and their families, and we all rang the bell together. We all got up on stage, and that was sort of a moment of a different pride that we had done this together and that we had managed not to die.

Sometimes with a startup, people aren't honest. They don't tell you that, but we'd avoided some. We'd made plenty of mistakes, but we'd avoided the fatal ones, and we'd made it through to a level of success that gave us the opportunity to pursue a new level of success. That's what an IPO is. It's not the end. You've gotten to a plateau that you can chase a new level of success in the public market. And that felt pretty good, I'll admit.

Sure.

So you have to think very carefully about failure mode, and we invented a technique that has about a million identical tiles, and if one fails, we can shut it down and use a redundant one. And they keep going. So if you're going to go big, you have to think about, in the very architecture of the computer, how you're going to manage failure. I mean, the GPUs have a huge failure rate, so I'm sure you guys have spoken about this.

Infant mortality is enormous, and they fail all the time. There's good data. Facebook put out a paper on the number of failures they get in a big cluster. Now, we can do some other things because we have all this compute in one spot. We can invest more to cool it. So we pioneered water cooling in AI systems, and we run these much colder than GPUs. And the failure mode in electronics is temperature. And so, by running them colder, we are more reliable.

And so that was an advantage. But again, we had to invent the technique to cool off a chip this big. And so I think we have a systems mentality, right? If you're going to do something big, you've made trade-offs, and you have to think about an architecture that allows for redundancy and repair. You have to think about the pros and cons of every aspect of the architecture. And that's how you do something different and new. Great.

How tokens are generated in inference is why it's faster. In inference, to generate a word—and our answers are a whole stream of words; they can be code, they can be pixels, but call them words—the weights of the model are moved from memory to compute, a calculation occurs, and that generates the word.

That is both steps.

Okay. There are two steps in the computation necessary to do inference. When you type into ChatGPT, "Explain to me the history of this village prior to World War II," and it can't see you, two things have happened. The first thing is your prompt has been processed. That's step one. And step two: your answer has been generated.

And the way your answer is generated is called decode.

And decode is sequential. Processing your prompt, which we'd call prefill, can be parallelized. So you can process many of them simultaneously, but the speed with which you get an answer is a function of the decode, and it is step by step by step in sequence, and that can't be changed. And how you do that step is you move weights, which are the intelligence from the model, to compute to generate a word. You do a calculation and you get a word, and then that word is used to generate the next word, where weights are moved from memory to compute.

So the process is one of moving weights from memory to compute. How big are the weights? In a little model, like a 70 billion parameter model, the weights are about the size of 100 HD movies. So to generate a single word, you move 100 HD movies from memory to compute, and then you have to move them again for the next word.

And you want to do this 1,000 times to get a good answer, a 1,000-word answer. This is where HBM is slow. This exact step is where HBM is slow. And that exact step is where, by having all the SRAM here, we're blisteringly fast. And so the speed of moving weights to compute is about 2,500 times faster here than on an NVIDIA GPU. And so that's the essence of what we've been able to do here and why it's so much faster.

No, exactly. Because remember, two things are happening. One, we want the models to be smarter. Two, one of the ways models are getting smarter is with RL, and RL uses inference inside of training. And so the faster you can do the inference inside of training, the faster you're training.

Yes, that's a market for us as well.

We do RL and we do traditional training too. Not for the largest models, for the largest labs, but for the next tier.

Pre-training, fine-tuning, our full set.

GPUs in training have some challenges that have been solved by a very narrow selection of the community. A GPU is a very small chip, and the calculations that we need to do in training are very large. And one of the most complicated parts of training is breaking up the calculations and spreading them apart on multiple GPUs. And that's called distributed compute. That has historically been the domain of the supercomputer world and is very difficult, not just because cracking a problem and having lots of others work on it is hard, but they have to constantly share information.

And that sharing information is why NVIDIA needed to buy Mellanox, right? They needed to control a fabric over which all this sharing, in order to get an answer, would happen. Breaking up a big matrix multiply, a big calculation, is called running tensor model parallel. You are breaking up the tensor and spreading it apart. And the best labs in the world are good at that, but nobody else is.

When we run training, we don't have to run that way. We run what's called model data parallel. And data parallel is very simple. And so it allows teams who are good, and very good, to quickly test ideas in training. And so we are easier to use and faster because we allow them to use a technique which is much simpler.

We're using it as a technique, a little bit like when you were in eighth grade and you wrote different drafts of the paper. Single-shot inferencing: you write a query, you get an answer. The easiest way to think about reasoning is you're going to do several graphs. It's going to break the problem up, it's going to solve it in parts, it's going to bring them together, it's going to review the results, it's going to improve the results, and then it's going to give you an answer.

And so that is going to take more compute. And if your compute is slow, that's going to be more than an irritant. That could be crippling speed. And so, as the best models—all of them, whether they're domestic or Chinese, whether you are OpenAI or Anthropic or Gemini, whether you're any of the Chinese models—they all moved to a reasoning approach. But that meant more compute was being used during inference. That was a huge advantage for us because we were faster.

And it made the GPU slowness stack up, and it made our speed have an even bigger advantage. And so this was a huge boon for us. We think this is going to stay as a fundamental essence of the way these models are run right now.

Sure. I think the verification problem is a little bit like the guardrail problem. And what you'd like to do is, after you've written two or three drafts of your answer, you would like to be sure that it wasn't wrong. And that's your verification step. And you can do that maybe with a different model. You can do that by asking your existing model a similar question in a different way, right? All of these are ways you can pressure-test your result.

Guardrails can work the same way, right? You want to review, either with a model or with another technique through a scoring mechanism, that this question isn't out of line, that this answer isn't about how to make biological weapons or calling upon information that you are directed to the FBI. And all of that takes compute time. And so whether you're trying to improve through reasoning or whether you're running guardrails, by being faster, you can get results in less time.

Well, I think those work together. I think your big model produces an answer, then you want to double-check your data. I mean, in the journalism industry, people would write papers that have data checkers, right? Somebody would go and make sure. Back in the world, journalists checked data, right? Being a data checker, right? That was the job. Each claim was checked independently. That's a different model, right? The main model wrote the piece, and then a little model checks some of the answers.

And I think that's a very good way to go about it.

Well, we just announced that we were fastest in the world on one of Google's multimodal models. I think the truth is that there's very little text that doesn't have charts and graphs, right? You must, to understand text, be able to understand illustrations, graphs, charts. And so that's the first and easiest part. And then you ought to be able to create both, right? And then you want to understand images. And I think the new models are very, very good at that.

Obviously, what follows that is video, because a video is just a collection of images, but that takes an enormous amount of compute right now. And that's one of the reasons it's been set aside by the leading labs. They're so unbelievably computationally intensive, right?

We make compute, and that compute is optimized for AI. It is the fastest at AI in the world. If you have a data center, we will sell you hardware for deployment in your data center. If you don't have a data center and you'd like to rent it by the month or the year, we have data centers in the U.S., so you can rent our equipment through our data centers and through our cloud. And so that allows us to get AI natives, as well as large enterprises and governments.

It was 50/50 last year, and I think this last quarter it was maybe 75/25 in favor of hardware sales. I think this year it might be 50/50. And as our OpenAI deal continues to unfold, it'll probably be 30/70, with 30 on-premise deployments of hardware and 70 cloud.

It's a shorthand. I mean, it turns out right now—and we didn't talk about this because they're sort of in the adjacent supply chain—we went through the shortage of memory. We went through the shortage of a process called CoWoS, 3-nanometer capacity. The other limitation in our industry right now is data center availability. And I mean, that is a limiting factor for everybody. And that's why Anthropic did a huge and sort of very expensive deal with Elon for data center capacity.

Our deal with OpenAI was because data center capacity is a limiting constraint, measured the way data centers are measured: in megawatts. The deal is 750 megawatts: 250 megawatts in '26 on a multi-year lease, an additional 250 megawatts in '27 on multi-year leases, additional in '28 on multi-year leases.

Data center for them. We're delivering a full cloud solution, so they connect to us via an API.

I am looking for data centers. My next meeting is, in fact, with a data center provider. We're doing a lot in Europe right now, a lot in the Nordics.

Yes. It is because there is low-cost power, clean, low-cost power, and low-cost cooling.

Yeah, there is an additional latency called transport latency, and that's the speed of light through fiber to get from Helsinki to New York, right? And you have to account for that if your customer's in New York and your data center's in Helsinki. Yeah, I mean, it's usually about two-thirds the speed of light, in case how long it takes. You would like data centers on the same continent, both of ours.

That's right. It's the disaggregated solution you mentioned before, where their Trainium part is doing the prefill and is doing the parallelizable step.

That's right. Trainium is doing the prefill step, and our chip is doing the decode. And so you get a lot more blisteringly fast tokens. Sweet. It's a good deal for us. It uses their data centers. So these are deployments in the AWS data centers. Yeah.

I think that's one of the reasons why we went from being a traditional chip and system provider to also offering data centers, is that what our customers want are fast tokens.

And anything we can do to make the delivery of fast tokens easier—for some of them, that's in their data center; for some of them, it's with an API. Just point your traffic to us, and we'll point the fire hose of fast tokens back.

We do.

Yeah, I think we have an on-demand service where you can come to our site and book a month. I think you can even buy buckets of tokens for Kimi or GLM or some of these models. Many of our customers come there, get excited about it, and then move to a dedicated offering where they take hundreds of machines for a year or two or three or four, once they've proven out the benefit for them in their work.

Often they do A/B tests. Not surprising, people like faster. Okay.

It's a full environment.

You can go and use it.

Play around.

So it's going to get interesting.

Well, I don't think CUDA is a moat.

We should talk about that. I think two years ago, every state-of-the-art model was trained in a CUDA flow. And right now, Gemini is trained without CUDA. Anthropic Claude is trained without CUDA. OpenAI is trained with CUDA. So in a one- or two-year period, they lost 70% share of training models that are state of the art, because Gemini is trained on TPUs. It's very good. Anthropic is trained on Trainium, not H100s. And so I think the story of the moat is still present, where the data show the moat is clearly shrinking.

There's no moat in inference. It takes you eight keystrokes to move from a GPU to us in the cloud.

That's it, to move your traffic from GPU API to us. And so, obviously, CUDA was enormously important in the creation of our industry, and it allowed the graphics processing unit to be more general than graphics processing. But since 2023, 2024, I think its ability to serve as a durable moat shrunk substantially.

Yeah. We built an ecosystem. I think our moat comes from the fact that, by virtue of our architecture, we are doing things no one else can do.

And it's not that they can spend more money or they can't pay more for GPUs. If you want fast, you can't have it. Well, let me shape it. It doesn't work. And so that's where we're building sort of our strength, and that's how we're sort of delivering value to customers.

We are TSMC. We have very close collaboration. They were investors in us. They've been exceptional partners. I'll tell you an unusual story. In 2017, we showed up, and we were about 30 guys total, and we showed up in August. Horrible time in Taiwan, right? Don't go to Taiwan in August. The heat is brutal. Not that it's so nice here today. It's only 90 here.

Yeah. But also the humidity, and you've got to put on a suit. And we met with the leadership of TSMC. We said, "We're like your little pipsqueak company. We believe we can solve a problem that nobody solved in history, and here's how we would modify the way you make chips to make this possible." They thought about it, and they said, "We agree. Let's do it." In the meeting?

In the meeting. It wasn't, "Go away for a month." In the meeting.

First, the salesperson had gathered the decision-makers. Second, our proposal was sort of really good at allowing them to use what they were good at, and it didn't require them to change a huge amount, but it did require them to make real changes. And I think they saw this as sufficiently bold that they would learn as they did it. And they also knew that AI was better on big chips. And so, the combination of prepared mind, willingness to take some risk, bold thinking from a very large company.

It is fascinating. I mean, that's how big companies win, right? And how rare is that? It was extraordinary.

Two years. Chipmaking is a long, hard process, and most of the time your first chip isn't a winner. And there are lots of startups now, some of them with really smart guys.

But their first chip will not be a winner. The TPU—Google had some of the best guys in the industry. First chip wasn't a winner, nor the second, nor the third. Fourth was really good. Now they're on their eighth, and it's a really good chip. The Annapurna team at AWS, first chip wasn't great. Second, third chip really good. It takes time. And so we built a chip, we delivered it, and this gets to an earlier question you asked: we solved a problem that nobody in the history of compute had solved, and we delivered it in 2020, and nobody cared.

Nobody bought any, and nobody cared, which—we were like, "Oh my God." Everybody said we were crazy. It would never work. And now it works, and nobody wants to sell it. It had its hardest. So then we built the next one. I mean, the first one we probably sold 20 or 50.

Nobody wanted it because AI was a hobby. It was, right? And who cares if your hobby's really fast? You care about fast when it's in production. You care about fast when you use it every day. And so we built another one, and that one we sold 300 or 500. Yeah. And we built the third one, and we sold tens of thousands.

Amazing.

Isn't that interesting?

So it's very hard to diversify away from TSMC. Chips are so hard. And you actually, when you design a chip, part of the design is for the rules of that factory, right? So you can't take your design from TSMC and go to somebody else because a huge amount of the work has been to be sure your design is within their rules. And so even in, I think, only one or two exceptions in history, each chip generation goes to one fab.

So that's—we're going to be with TSMC for our next generation as well. I think some of them, we have a supply chain that is built in many parts, but we bring the chips back from TSMC to the U.S. We package in the U.S. and we assemble in the U.S. We do our manufacturing in the U.S., and then we ship from the U.S. I think when you're growing as fast as we are, right, there are a whole range of garden-variety supply chain challenges.

A vendor screws up a batch. It gets stuck in customs. The number of ways that things can go wrong in the supply chain is unbelievable. But we manage these every day, and we're increasing our manufacturing throughput exponentially. And so it's really that part of the business in clover. Wow.

GPT-6 will be the worst model you ever use. And whatever you think is cool about it right now is going to be boring and backwards in six months. And that is so exciting. And I think I watch the way our young engineers use it, and it's very different than the way I'm using it. And it's sort of a fun time where you can learn from your young team members that they're using AI very differently. I think the business of dashboarding and the business of IT, just the damage AI is doing to SaaS is, I think, unrepairable.

For all these, you can ask your AI, "Build me a tool like Salesforce." Thirty seconds later, you have a working tool that is just unbelievable. And all the things that were difficult because they cut across your internal organizational silos, right? One of the things that's really hard is if you want to know, for your top-performing people, when was the last time they got a stock option refresh and how much holding power is left. So, how much unvested stock they have left at today's price.

It was like five systems. You're in Workday, you're in your stock.

You're in your comp, you're in Carta. None of those can handle, and that's what a CEO wants.

How much holding power is for my top guys?

And I used to have little tools I wrote for this and, oh God, I've got a little app that I had. And Spark, right? Swarm.

Thank you for having me on your show. I really appreciate it.

Bye.