Intelligence Isn’t Enough: Why Energy & Compute Decide the AGI Race – Eiso Kant

Good to be here, Matt.

I was actually listening back to our podcast, and I hadn't realized that it was the very first podcast we had done as Poolside. And yeah, two years have flown by.

So it comes back to why we started the company. If you go back to early 2023, when we started Poolside, we started it because we had our own point of view on the research. So if you remember earlier in that year, GPT-4 had just come out, and the narrative in the world was all we have to do to reach AGI is scale up language models, do more next-token prediction, a larger number of parameters, and more data. And we agree with the importance of scale, and to this day still do.

But our point of view was that reinforcement learning was going to become the most important scaling axis for model capabilities. Extremely contrarian opinion at that point, still when we were doing our podcast, and probably for the following 12 months as well. Today, that's of course very different. Today, I think that's become consensus, and I think the world has understood that we now have an ability to continue to scale models toward more and more capable directions and really close the gap between human intelligence and AI.

And that mission was the original mission of Poolside, and is still very much the mission of Poolside. But along the way, I think what we had predicted would happen played out. We were going to continue to scale up, more compute was going to be required. But at the same time, the first paradigm of pre-training, of predicting the next token on the web, was becoming sigmoidal and was slowing down in terms of the gains that it had. By now, we've caught up on the pre-training side and really on what we refer to as the table stakes of foundation model building and language modeling, while building some pretty serious advantages on the reinforcement learning side.

And now we're at a moment where those two things have come together. All of us, as frontier AI companies, are developing our big model runs still at the same scale. So there are a series of advantages that we've built in our research. There's advantages that we've built in our engineering, and those are probably some of the most important sustained ones. But now we're getting to a point where scaling up our models to frontier sizes is also necessary for us.

And hence, we've recently had some announcements about the sheer scale of compute that's coming online. And so we're competing. We're in this race. I think I listened back to our podcast and I said, we're going after OpenAI and Anthropic. That's still very much true. But our starting point was a little bit different, right? We said, to get to highly capable intelligence that can reason, that can do planning, that can understand the world, you're almost better off putting a set of blinders and focusing on one proxy for that.

And for us, that proxy was software development. And as we've gotten more and more capable at doing longer-horizon software development tasks, we've also gotten more and more capable at, frankly, all types of knowledge work tasks. The world from a model perspective, I think we are all over time converging to a similar point. I would actually go as far as saying that intelligence is going to become a commodity. In that world, who we want to be is we want to be trusted by enterprises.

We want to be trusted by businesses to power the knowledge workforce that started with coding agents and is now going beyond that as well. And I think in this space, so much is going to transition. We're like a pre- and post-electricity moment. So I don't think it's a winner-takes-all market. But it does seem to be that it's a small number of companies who are able to get there.

So Project Horizon is us building out one of the largest data center complexes in the United States. And this comes back to something I said earlier. We talked about intelligence becoming a commodity, right? My thesis for five years has been that there are three layers of the stack that fundamentally are going to matter. It's energy, it's compute, and it's the intelligence built on top. And within this world, if you think that intelligence is going to become less distinguishable between the companies building it and becomes a commodity, a commodity probably more like oil or cloud compute than like bread at the bakery, there's two things that matter: your ability to scale it and the cost at which you deliver it to your end user.

Now, the ability to scale it was frankly the primary motivating driver for Project Horizon, but cost as well. And it's important to think about what you could do 12 months ago versus what you could do today. At the scale of compute that we were talking about two years ago in our industry, you could call up a data center colo and say, "Hey, I want this much compute in six months," and there would be space, like physical space where you could deploy it, or there'd be someone who has the capacity available to you.

Today, we're talking about a scale of compute that gets counted in the hundreds of megawatts and soon gigawatts, where there's no one you can call.

Right? These are built-to-suit data centers that are so large that no one is building them before having a tenant. And so as you're approaching the frontier and your models are getting more capable and you want to serve that intelligence to the world and you want to scale up your training as well over time, now your lead time from deciding to do that to being able to do that is no longer calling up a hyperscaler or calling up another partner and having it in months.

Now we're talking 12 months, 14 months, 18 months with huge capital numbers attached to it. My co-founders say this thing to each other, and probably not really for podcast material, but if you're a foundation model company and you're not building physical infrastructure, you're cosplaying your business. And this is kind of the really fundamental nature. It's not that we went out and said, it'll be cool to build infrastructure. It was a necessity for the mission to be able to achieve it.

Yeah, I think as a foundation model company, you can go two paths, right? You can choose to deeply partner with a hyperscaler and kind of have them become an owner in you and really go all the way. And I think that's one direction. But at the same time, the world is getting to a point where I don't think anyone has any doubts anymore that we're now on track to reach human-level capabilities and intelligence. And in that world, $29 trillion of knowledge work rewrites itself, right?

Scientific progress starts pushing beyond levels that we've ever seen, and all of it is intelligence on compute. And so a foundation model company is far more in a physical infrastructure business than most people realize, because the ability to scale that compute and bring it to end users, and frankly, do so cost-effectively, right? The cost of your tokens are going to matter more and more as our intelligences all become closer to each other. The ability to scale this up and do so to end users where the cents per token are kind of a determinant if someone's going to buy it is critical.

So we already started several years ago asking ourselves the question, what would it take to move towards this? And then over time, we learned more, we observed more, and then we started acting towards it. And the acting towards it is Project Horizon. So Project Horizon, in a nutshell, is today announcing a 2-gigawatt campus. We can actually go far beyond 2 gigawatts on the site that we're in. It's a partnership with an incredible family out of Texas called the Mitchells. The Mitchells have one of the single largest parcels of land in the United States.

It's half a million acres. And it always blows my mind because, for context, LA is 300,000 acres. And on that land, there's renewables being built, there's grid connection, there's water, but there's also a 20-inch main gas line, and that offers us a possibility to start scaling up energy that powers data centers incredibly fast and at an incredibly large scale. And so what you're seeing from us is that before the end of this year, we're starting construction on a 250-megawatt data center. It's natural-gas-powered with grid connection as a backup.

And it will house an incredible amount of compute, but it will be the first phase of many that we're building out. Great.

I'm not yet convinced that gross margins in our industry will look like a SaaS company, like 80% plus. I think when we're talking about a commodity as intelligence, that we build value-added services on top as a foundation model company. On one hand, you sell your tokens, your barrels of oil, your raw material, and on the other hand, you're building up value-added services, your products that you bring to end users and customers that unlock value for their businesses or for consumers in the case of others.

In that world, I think it looks a lot more like a cloud company. So I think when we're going to look at the scale of this, it's going to look like cloud companies with a zero behind them. But from a margin perspective, probably pretty similar, because if you think about what is an AWS or a GCP or an Azure, it's effectively virtualization of hardware with services on top. I don't think foundation models are that different. So I think from a margin profile, we'll probably sit far closer towards that 40% mark that you see in cloud companies than you do the 80% on the software side.

And now, when you start thinking about what sits in the stack of cost of intelligence, you effectively—you've got the land, you've got the energy, you have the data center, you have the chips inside the data center. And just for context, if we, for instance, break this down with illustrative numbers that are roughly in the right ballpark, to energize 250 megawatts is about half a billion dollars' worth of physical infrastructure. So in the case of gas turbines, it's gas turbines, but there's different paths to it.

$5 billion. $5 billion. So kind of all in all, when we're, like, adding this together, you're talking—

Being the chips and the networking and everything that kind of is what we'd refer to as the fit-out of a data center. And so you got about an $8 billion cost of 250 megawatts. So when you hear in the industry, you might say a gigawatt is $40 or $50 billion, that's what people are referring to, right? It's the breakdown of those costs. Now, the life of a chip has a certain amount of years to it. And in our industry, there's lots of argumentation about how long they last, and it's a demand and supply question.

A data center has kind of traditionally a 15-year life, but it needs retrofitting as new generations of chips come online. And your power infrastructure can last a very long time, but requires maintenance.

Now, in that $8 billion project of 250 megawatts, you're annually spending somewhere north of $300 to $350 million of OpEx. And so you can split that about 50/50 between the cost of financing and the cost of energy and operations, energy being the most. And this is where there's already an interesting insight. All of a sudden, you realize that the energy part today is not the biggest part of the stack.

You're talking $160 million a year in energy. And of course, I'm taking numbers that are West Texas numbers, so energy can be double that in other places in the country. But still, compared to the total cost, TCO, of compute, it's relatively minor. Numbers are still very big. Now, the reason I mention this is that what is going to happen over time, like what's always happened in capitalism, is that margins compress everywhere. Now, margins have already compressed for energy. We already try to make energy as cheaply as possible in the world.

Data centers effectively have already been increasingly more commoditized over the last 20, 30 years. GPU compute today is still a very high-margin business, and over time we'll find a similar level of compression. And as the chips become more compressed, the other parts of the stack are more important as well. And when you end-to-end own all of this and build all of this, everywhere where previously margins sat in between falls away. If that's the person who you were paying for the cost of land, if you're talking about the cost of the energy, the cost of building the data center, kind of all the layers in the stack, this adds up to a very large amount because the foundation model itself is—it has CapEx for building it.

It's your R&D cost, and you amortize it over time. But the physical infrastructure is really, frankly, where the majority of your cost sits. And so I know I went super detailed straight away, but I think it's useful for people to kind of get this sense of where it is. So when you take all those margins out, all of a sudden you can start seeing that you can serve your tokens 20%, 30%, 40% cheaper than someone else. And this is going to matter because we're shifting from human labor that leads to economic output to intelligence on compute combined with human labor.

And we've barely started this in the world. And so we're going to be in a world where this is a commodity that will look far larger than we've seen cloud compute be. Frankly, it's often the largest bills of companies, what they're paying. In that world, yeah, controlling that cost has a big impact, but cost is really only part of the answer here. You have to be able to scale it. You have to be able to say, if I next quarter need more compute, can I bring it online?

Because there's no shortage of chips. There's a clip yesterday from Satya on a podcast that I saw where he said, look, it's energy and data centers that are my bottleneck. It's not chips. And it's true, right? NVIDIA does an incredible job at supplying the market with chips, and TSMC and everything in the stack can scale up to huge levels. But physical space, bringing power online, and building data centers, well, that's when we kind of in tech will always start realizing the real world hits us.

Exactly. So if you think about traditional data center business, and this is not unlike it, you have an amount of equity that goes into that business, you have a loan-to-cost ratio, and the loan-to-cost ratio really depends on the lease, the tenant that you have on that data center. You've got a 15-year lease contract traditionally, and then it's traditional project financing that is highly effective. The world has been doing this for a long time. It is relatively low-rate financing. And this is not new.

This is not new to tech. It's been around for a long time.

We are not yet at a place where we can be signing 15-year leases on massive multibillion-dollar buildouts. But we are ramping up rapidly, where we see a path to being able to scale up compute at those levels. And so we found a really great partnership here with CoreWeave that was announced. CoreWeave, frankly, is second to none in terms of operating NVIDIA's compute. First to have GB200s online at scale, now coming with the GB300s, and have been this great partner for us in the space.

And so we found a really great hybrid solution with them. So they are the anchor tenant on our data center. Jointly, we can scale up our compute inside that data center, and any capacity that we choose not to take or are not able to take in our space, they can bring to other customers. And this was really an important thing because we're not a hyperscaler who can make multibillion-dollar bets for two or three years out. So we needed to find our way of having great partnerships that were kind of win-win situations where we could scale up our compute but didn't need to make a lead-time decision years ahead of it.

And this has been a fantastic setup because it kind of brings the best of both worlds to the site.

We followed an algorithm, I would probably say, most of my career by now. At least, I remember going back as far as nine years, is you have to, by all accounts, avoid the Dunning-Kruger effect as a founder, right? Because when you get into something new, like it has been on the physical infrastructure side, you start with learning, you start with reading, you start by meeting the experts, learning and learning. And as you learn more about a topic, you fall on this point where you start thinking you really know it, but you don't really know it because you haven't hit the real world or you haven't done it before.

And what I've found to be one of the best pieces of advice I got a very long time ago: whenever you find yourself in that situation, this is when you want to start finding the experts to join you. But how do you find the experts? Because you yourself aren't. And here's when you just start interviewing 100-plus people for every role, and you build a distribution. You start understanding who sits on the right-tail-end outlier of that distribution. And it turns out, I wouldn't say our knowledge at this point was shallow, because we'd spent years understanding the space and getting close to it.

But definitely not at the level of an expert. We got to the point where we started being able to identify who were the outliers. And those are the people you hire and bring on, right? And empower, give the right autonomy to. And that, in combination with the values that we care about. So at Poolside, since day zero, we've always cared about low ego, kindhearted, extremely hardworking. In our space, you work way too many hours, a lot of intellectual curiosity and horsepower, and deeply care about the work that you deliver.

And we recently added a sixth because we found that's really been true: people are very explicit and direct in their communications, right? Or can be very clear.

And so we started with that, and you start from the top, right? We found an incredible hire in Lance, who's our VP of data centers, and from there we've been building out the team. What was particularly interesting to learn about the infrastructure space over time is that it's an incredibly small world. Everybody knows each other. It makes tech feel like a big world, right? The construction of data centers at scale, it's really only been done by quite a small number of players.

And so everybody knows each other across the entire end-to-end supply chain. So you also have an ability to very quickly understand who's considered a good actor and who's considered a bad actor, who has built up a reputation. Small industries are very reputation-driven. And so it all starts with people, empowering them, learning from them continuously, and knowing where you trust, and knowing where you go deep and you ask questions and try to challenge people. And that has actually not just led us to building an infrastructure company with an incredible team, but also to find the people in the industry who were able to start rethinking some of the ways data centers have been built.

So we haven't just gone and said, we're doing the exact traditional thing everyone else is doing. We've actually taken a quite new approach to data center construction as well.

So if you think about data centers, there's kind of been a traditional way of building them, and I wouldn't say a new way because it's been around for over a decade. But on one end, you have kind of stick-built buildings. Everything is done on-site. So all the materials are brought there, thousands of tradesmen are assembling the data center. And then on the other end, kind of like Vertiv here in the West, or players like DayOne and their parent company out of China, who've gone full modular.

So think of your data center units rolling out of a factory. And what was interesting is that both of those approaches have their limitations, right? The moment you're bringing thousands of people to site and you're assembling everything, and especially in what is effectively the Permian Basin, a desert in West Texas, comes with its challenges, both on mobilizing workforces, but also the weather, the dust, the logistics, the supply chain. But then the factory side has not yet scaled up at a level in the West.

I would say this is different in China, where you can actually bring out gigawatts over the course of a couple of years. And so we kind of decided to meet in the middle and took an approach where we said, we're building the actual building stick-built. So that's being built on-site. Think of it as an extremely long corridor with leaves attached to it. But let's take a modular approach, one that fits on the back of a flatbed truck: five-megawatt compute-ready skids for electrical, mechanical and cooling, and compute.

Those are kind of the three elements that sit inside a data center, and do it with manufacturing partners here in the United States. And what you start finding here is, once you start breaking the problem down, you realize that the electrical modular components can be done by lots of great manufacturers. This is not unique. So you actually have great players, including very large ones, that can help you there. When you then get to the cooling and mechanical part, it's a smaller number of companies, but again, they have been doing this for a long time, are highly reputable, and you can work with them.

And then when it gets to the compute side, there's recently been an unlock. Because on the compute and cooling side, what you're finding is that now we have direct-to-chip liquid cooling, the designs are changing. There's a lot more that you can do in a far more contained manner. And so we decided we'd rather have a little bit more redundancy on every individual room that we build so that we can bring them incrementally online. So where most of the data centers being built for AI factories today, you'll see them bringing them online at kind of 40-megawatt chunks.

Five megawatts of compute online as you're doing construction? Think of it again, on this hallway, as you're bringing on units of kind of 1,000 GPUs, or depending on the GPU type, can be less in the future, you're bringing this online. And the combination of those things that we refer to as incrementally delivered hybrid modular data centers is kind of bringing the best of both worlds. It usually de-risks your ability to deliver. It de-risks your ability to bring compute online: five megawatts.

Like, I just want a little bit more. On the other hand, you're taking advantage of the fact that for the kind of bigger construction, the building and stuff, you take the traditional approach. And on the manufacturing side, you can partner with multiple manufacturers, and you can do so actually very geographically advantageously as well. Most of the things that we are bringing to our site are actually manufactured in Texas or very nearby. And so now you're actually reducing your supply chain risk and your time-to-site risk as well.

The biggest advantage, though, of all of this is not just speed, but it's also your workforce that you can mobilize. Because we're in a quite remote area, and that remote area offers all of these advantages, right? Pretty much infinite land, so we can build a single level horizontally, an incredible amount of source energy, but also being remote, meaning that mobilizing workforces there is more challenging. So all of a sudden, when we're building 250 megawatts, we're doing it with a less than 450-person onsite workforce, with very skilled tradesmen and construction workers that we're bringing over there and we're housing over there, versus a world where you might need traditionally 2,000-plus people to build the exact same thing.

So it's all about de-risking and speed, and you kind of see a common theme. We think about lead, and we do the same, by the way, on our model-building side. We think, how quickly can you go from decision to having something online? And how can you make sure the capital that you have to deploy does not have to sit for 12 to 18 months? Because that's when you, as a smaller company, can all of a sudden scale up. Because the big numbers really have more to do with the fact that these decisions need to be made years ahead of time.

We can make them months ahead of time.

So at the beginning of Q4, end of Q3. So next year we've got the first compute coming online, and then we're finalizing in Q1 2027, the first 250 megawatts. But already next summer, the next 250 megawatts starts construction. So it's a staggered build. And this means that we will always have a horizon on essentially three times a year, an additional 250 megawatts coming online.

Let's first be honest about this, right? We're using natural gas for the generation of power. And in the world that we are right now, at the scale that data centers are being built out, the sun doesn't shine all day, the wind doesn't blow all the time. And so renewables only, in combination with the battery capacity that you would need to power a data center if you want renewable only, it's just not economically viable yet. But the combination of natural gas with renewables, with batteries—we've got a big BESS being built out on the site, so a big battery system—allows for kind of an optimal point in the middle.

Now, when you talk about natural gas generation, you want to make sure that you have your emissions in check. So what you're adding is SCRs. And so you're—

An SCR is essentially selective catalytic reduction. So what you're doing is you are filtering out some of the more harmful particles that come out of natural gas generation. And you do this all within federal standards. You file for air permits that are approved at the federal level. And so this is a critical thing to do as a business. And so you're combining those things. So that's on the energy side. The second side of environmental concerns around data centers is usually water, right?

Water is, depending on where you are, a scarce resource. In other places, it's an abundant resource. It can impact a local community. And so we're very privileged because of where we sit and the sheer scale of land that our partners have. Water is both a resource that is there at the levels that's required for us without actually taking away from the local community. And this is not the case everywhere else in the United States or in the world. So picking this site was not just about how large it can scale, because it can scale into an incredibly large size, but also, can you do so in a way that's responsible?

And community, what goes beyond the environmental topic, is incredibly important. You're building something out. Even though we are building out in a relatively remote area, at the end of the day, this would not be possible without the support that we get from the local community. So we're in a place called Pecos County, which is just some of the best people that I've ever met. Honestly, if you want to go and just walk into a bar somewhere and spend time with some of the greatest people in the United States, go to Pecos County, because we've been amazed with how welcoming the community has been.

And so this goes hand in hand with how do we invest in job programs to bring people— I don't mean to start sounding like a politician, Matt, because I think this is almost—when you talk about community, it feels like you are. But these are projects that are not one-off, right? When we talk about this big multi-gigawatt project, we've got about eight years' worth of construction that we are planning out there. And potentially, as nuclear becomes a viable option in the coming years and we can build out more power, potentially in nuclear, over there, this can be construction that happens out there for as long as we can imagine.

The space is close to infinite as you're going to get. And so bringing kind of everyone along has been kind of a critical point for us. We just try to be good actors and try to be transparent about the positives and also the negative externalities and how to mitigate them.

So when we did this podcast two years ago, we spoke about our approach of reinforcement learning from code execution feedback. I think at the time I called out we had tens of thousands of these environments where the models would go in, they would be given synthetic tasks, explore solutions, and then be rewarded on the solutions that we provide.

Exactly, yeah. So what we have done is, at the time it was about 10,000, or I believe 20,000, codebases, like real-world repositories that have complex software in them, where we define tasks and we would look for often single-shot solutions. So here's a task, the model can write some code, it gets tested, it might have additional signal from reward models and others in there, and it will be able to provide feedback. Was the code correct? Was it wrong? Was it rightly styled, et cetera?

Over time, that has grown a lot across two axes. One is right now we have over a million of those environments and tens of millions of revisions, putting us in the 30 million range. So just two orders of magnitude larger over the last two years. And that's because, as you increase the diversity of the type of problems that the models need to learn in, you can increase the capabilities of the models. The second axis is that the models got so capable in their reasoning and longer time horizon capabilities, multi-step, like complex planning and execution, that RL moved from kind of single-shot reinforcement learning to agentic RL.

So meaning that now we give a much higher-level task, we send in an agent, that agent goes and tries to solve for the problem by using its tools. So editing code, executing commands, updating packages, like doing everything that a software developer would do. And when you see it today in coding agents, it looks even quite natural to how we do things, right? Increasingly more like how we do it. And then it gets rewarded along the way as well. So what we're seeing is that we're entering a world where the reasoning capabilities of models and their longer-term horizon planning and execution is now getting so capable that the tasks we provide them are no longer simple tasks, like the questions at the end of a textbook, but now it's like the projects that you're made to do in school, where you have to do a lot more steps.

And so we've continued to scale on those axes. Today, I believe everyone is scaling amongst those axes. And it's true, where two years ago this was a contrarian opinion that no one was doing, today it is absolutely consensus. We have again found ourselves in a point where we now have again a contrarian opinion on the future. And it's really not by trying to be contrarian. It's just that we, I think, have had a long time thinking about this, right?

I started thinking about this first in 2016 when I was building source{d} and did the first RL with LSTMs, with language models, and now the last two and a half years with these larger LLMs. And our view now has become that the world is going in a direction again that we actually might not fully agree with on RL. So what you're hearing a lot about at the moment is the scaling up of environments with human expert rubrics. So it's not just going into coding where we've got verified rewards.

Now we're entering into areas, if that's how to operate on a spreadsheet or if that's how to do a complicated task as a chemist or marketing professional, where you use kind of a rubric created by an expert to make it part of what a model can use to judge the answers. And it's highly effective. And by the way, we do it as well. It is a highly effective way to get models singular capabilities on singular tasks to push it up. And I think what we'll see in the next couple of years is a lot of model companies, ourselves as well, we go into these enterprises, we find high-value use cases, we use our agents to get to 60% or 70% of quality, and then use additional reinforcement learning with domain experts' knowledge to get it to 80% or 90%, or potentially even 100%.

But the narrative that that's going to scale us to AGI, yeah, we don't agree with. We think this is the right thing to do in the awkward teenage years ahead of AGI. It's because it allows you to create really economically valuable outcomes for your customers. At the end of the day, we're businesses, right? We need to be able to fund our economic engine towards our mission to Mars, towards reaching human-level intelligence and beyond. But the singularly collecting of skills this way by creating environments and experts, we don't think will scale to artificial general intelligence.

Our view is that reinforcement learning will have a moment similar to what we've seen in language. So what makes language modeling so beautiful? It's the fact that by predicting the next token on the web, you have this general self-supervised objective. You just can throw the entire internet at it and, of course, continue to filter it into more quality and improve it with synthetic, all the things we spoke about last time, but it forces the model to learn. But the reason that never got us to AGI is because, well, the internet never actually included the dataset of the thoughts and actions that created it. It's the final piece of code, it's the final article you write, but not the thoughts that you had and actions that led to it.

Exactly. And so our view is that there is a generalized objective for reinforcement learning that doesn't require human judges or environments or reward models or anything external, but can generalize over language. The same way that we have seen next-token prediction generalize. A way of thinking about it is, if you have traditionally taken the web and used synthetic techniques to reformulate it, to improve it, we've moved forward. We've since taken the web and generated forward. What if you could generate backwards?

So what I mean by that is, what if you could reverse engineer the web or decompress the web into the thoughts and actions that created it? Is there such a general objective for RL that can be found? This is where we've done a lot of our research in the last year and a half, and we have now started seeing a lot of promise towards that being possible. Not yet ready to talk more openly about it because we're still at a stage of the research where we've got a lot of things to prove out.

Yeah. So we call it reinforcement learning to learn, RL2L. Okay. This is the first time I'm probably publicly saying this.

Yes.

It's a perfect way of framing it. And we don't think they're mutually exclusive, right? We think that you want to have a model learn how to think and reason as early as possible in its training, and then you want to actually have it learn in the environments to sharpen its skill sets. And it's not unlike us. We'll have learned a lot coming out of university, but then we're put in the job and we're actually doing it, and we're learning from experiences.

So the learning from experiences, the reinforcement learning from environments and from experience, is effectively like renewable energy. The density of information in those tokens is not as high as in a physics book, where there's a huge amount of density within a small amount of tokens. In experience, it's less, but it's highly valued. So our view is that pre-training and predicting the next token on the web is an incredible bootstrap of understanding language and helping us get to a level of intelligence.

Reinforcement learning to learn, RL2L, internally we also refer to as the Bondi techniques—it's kind of our code name for it—we think will push models to a level of reasoning and thought that will happen far earlier in their training than it does today. And then you have reinforcement learning from code execution feedback and from other verified environments that help learn really what it is to sharpen those skill sets in simulated environments. And then the fourth stage of training, over time, increasingly will become learning, continuous learning from real-world experiences of these agents.

And so those are kind of the four stages that we think training will go to.

Today, there's one thing about foundation models that we have yet to really optimize almost at all, like, over all of this time, which is the ability to learn from a single sample of data. Foundation models today can't do it, and we don't yet have a path to them successfully doing so in a way that really improves it. Internally, we call it the hot stove problem. If you're a kid and you touch a hot stove once, you're never touching it again. Single sample and you're good.

Foundation models, because of the underlying technology, gradient descent, it's such a data-hungry algorithm. It requires so many samples to be able to navigate that higher-dimensionality space to a place where it does something more optimal. And so we've got this big hole towards AGI of, like, how can we get a model to learn continuously from a smaller number of experiences like you and I can? Now, there's a question if that's required to reach AGI. I would actually tend to say that if we take the definition of AGI to be that foundation models are as capable as you and I to do the vast majority of economically valuable tasks, maybe first behind a laptop and over time embodied in robotics, I would tend to say that it might not be necessary, but it, of course, is an incredible thing to add to intelligence because it will massively make it more compute efficient and will also make it more valuable over time.

And what we do have already a path towards is when we've got a large number of users or the real world giving feedback on what an agent is doing, what a model is doing. Models and agents, to me, are effectively the same thing. We can incorporate that feedback in improving the models. And this is really the learning from real-world experiences. This is when you've got maybe hundreds of thousands or millions of agent trajectories, tasks that were done where some feedback was provided.

That can be by the person who instigated it, or can also be by some form of environment or system. And so it's kind of taking it from in our clusters as foundation model companies to where it actually touches the users, and that feedback will come back. And so, is that necessary to reach AGI? Not so sure yet. It's a very honest answer. But will it be valuable in the journey towards it to make our models more capable and more directed to what they want them to be?

Absolutely. In all of this, I have a bit of an issue. I started to use the term more because it's just more commonly used, AGI, but it treats intelligence—we often treat it as this singular spot that once we're there, it encompasses everything. But intelligence is so multidimensional, right? You've got people who are incredible creative writers who can create incredible works of art, a Dostoevsky. On the other hand, you've got incredible mathematicians who can prove theorems. On the other hand, you have incredible people who are amazing at designing a factory.

And it is not yet obvious that that is all a singular thing, right? And we see this very clearly with modalities, right? Someone who is not able to see can still be an incredible software developer, but is unlikely to be able to be an incredible race car driver, right? Or unlikely—let's call it impossible—to be an incredible race car driver. And so intelligence is so multidimensional that we can get to a point where it's going to be as capable of doing all the knowledge work that we do behind a laptop, and we're still not going to agree that it's AGI, and we still won't have solved every technical challenge.

And that's also okay. I think we've got a long road ahead of us to truly get to the incredible, versatile features that we have as humans. But will we get to a point in the next couple of years where economically valuable knowledge work can be done by AI? I think absolutely.

I haven't yet listened to the Karpathy interview, and I really want to because I don't think you noticed, but I got into this space because of Karpathy. So in 2015, Andrej Karpathy wrote an article called "The Unreasonable Effectiveness of Recurrent Neural Networks," a blog post about language models. And that singular blog post let me start my company, Sourced, and got me down this obsession for now a decade of what language models can do. And so I have an immense amount of respect for his opinion.

And so while I haven't listened yet to the interview, I can't comment specifically on what he said. But what I can say is this: What is today the definition that we use of an agent? It's a model running in a loop inside an environment with access to a set of tools, right? And it's doing longer-trajectory work. And how are we training agents, right? Agent capabilities as foundation model companies is that we take that agent, the binary, the piece of code that runs this in a loop and has access to—it could be a container or it could be access to a bunch of tools—and we train it together with the model.

And so this is why you see the most capable coding agents really coming out of people who are training with the models. Because, and this, by the way, will likely hold true for lots of domains, when you already have a model that has all of the intelligence and all of the capabilities to do an agentic task, so it doesn't require more intelligence, it doesn't need to be pushed further in reinforcement learning, well, then the question is, what's the differentiation in the agent? Well, differentiation in the agent is whoever's building it needs to have access to some form of proprietary data, some form of proprietary environment, something that allows that loop that the model is running in to be more advantageous than another competitor.

And there's lots of opportunity there. But what I am careful about, and we see this in coding, we see this ourselves, is if you decide to start a company to build a coding agent where you are not able to improve the model, the agent is not able to train together with the model, and foundation model companies like ourselves are deeply focused on doing so, you don't have that edge. And so the cat-and-mouse game that you're playing becomes a lot more difficult.

It's not impossible. I've seen incredible agents being built for specific things because no one can singularly focus on everything, not any foundation model company, including ourselves. But we have a phrase at Poolside, which is, over time, everything collapses into the models. And I think increasingly, that's what we're seeing. Frameworks or agents or products that were around two years ago have either gone or sometimes their UIs have collapsed. So you used to see these products with lots of bells and whistles and lots of things behind the scenes, and you would see lots of UI options.

And today it's just like a screen that says "agent," and you talk to it, and it goes off and does the work. And so I think you have to ask yourself the question: Where do you sit? What's your unfair advantage that you have? And if a foundation model company decides to focus on it, will that advantage still sustain, or will it fall away because they're able to train the models to be further and more capable in combination with their agent than with yours?

I think we're all ignoring something, right? And maybe it's the wrong way of putting it. I think we have a hard time holding the point of view that models will reach the same level of intelligence and capabilities that the world's most capable people in every field have. And when you take a step back and don't take the next 12-month view, but just take the next—I think it's in the next 36 months for knowledge work. Maybe it's the next—someone else might think it's the next five years.

I think few people today would argue that it's not going to happen in the next decade. And when you're building a business, you're not building it for the next five years. You're building it for the next 20 or 30. You're building a company that can sustain and be at the level of success, at least particularly when it's venture-backed business. In that world, what does your business look like if AI is as capable as the most capable person in the field? And that means certain things still exist.

Like, my Uber Eats app still exists because I want to look at my food, et cetera. But a whole bunch of vertical software or vertical agents are probably gone. And finding out where you sit in that world, I think, is critical. I think we have a tendency, all of us, and myself included, to get so caught up in what's right in front of us that we don't take a step back and come back to the big things. And I think the big thing really here is that we are going to reach those levels of capabilities, and it rewrites the world really like the pre- and post-electricity moment, far more than we, I think, have even factored in today.

I think not even in the financial markets or even in the way that we operate our businesses. It's truly living on an exponential to a point that I don't think we have ever seen before. Everything we've created till now was because of intelligence, but now intelligence itself is something we will be able to create and scale. And I think it becomes a very different world afterwards.

Frankly, the same thing. I've had this question for two and a half years now, since starting Poolside. This is not the first podcast: are we hitting a wall? We are, I think, at a point where we are continuing to see, with every new generation of chips, an ability to make the model larger. And it's important that this links to every new generation of chips because it is not a world where you can throw infinite dollars at scaling. That's a false narrative.

And you can think of it very easily yourself, because if I take the size of a model, which is still effectively the biggest determinant of how much compute it takes to train it, because of the limitations on the networking and because of the limitations on the FLOPs on those chips, it is not that I can linearly add more GPUs and train an increasingly larger model. If I do so, the time it takes becomes exponentially longer. So if tomorrow someone said, "I wanted to train a 300 trillion parameter model," completely out of the realm of anything that anyone's doing today.

So it just wouldn't be possible, no matter how much money you have, no matter how many chips that you put up. It would become exponentially more expensive and longer to train that. And this is because of the current hardware limitations. But every two years, and now the chip cycle is even getting shorter, we have an incredible new chip and an incredible new networking stack that is improving, that all of a sudden makes the next generation of model size possible. Now, I don't think this will scale infinitely.

I look at intelligence as compression, and at some point you can't compress further, or at least not to a point where it's worth it. It's diminishing returns, but we still have an ability to do that. So on one hand, we have a free lunch. As new generations of chips come out, we can build and train larger models. And it continues to show that it improves the model capabilities. On the other hand, and I think far more interesting, as we're now getting into the world of scaling reinforcement learning, we're able to train models for longer duration with more data, right?

Because on one hand, we have the size axis, and the other thing, we have the duration axis: how much data we're showing the model. And every single year, we've also become multiples more efficient with the data we had to make models become more capable in a lesser compute budget. It's kind of mind-blowing to see where, even if I think about our pre-training at the beginning of the year, or we have a new run that's going on at the moment versus now, you can really see incredible improvements because you're refining the data better, you're creating it cleaner, you're creating a better curriculum.

I know it's a long-winded answer to your question, but I think it's important to understand that it is not infinite dollars you can just keep throwing at intelligence. It's kind of bounded by the physics of every chip generation. As reinforcement learning is becoming increasingly more of a scaling axis, we are able to improve models within those generations far more, but we have not yet found a generalized, infinitely scalable dollar amount we can throw at reinforcement learning either. And so, both on increasing model size and on increasing data from RL, there are still limitations.

And if those limitations didn't exist, Poolside wouldn't have been able to catch up. Right. I think this is a really important part because we're now at a point where we should definitely spend some time with the models after this. They've gotten really darn good. We're now scaling up compute to make it to the frontier. That wouldn't have been possible if it was just a world of dollars. But it is definitely a place where, yeah, I think right now I don't see any limitations.

I see more than limitations. I see opportunities in some of the research we're doing in reinforcement learning, to learn, and others that might completely open up new scaling axes that can go even further. And then every generation of chip that comes out, we'll see a stepwise function that goes up in our industry.

If you go back to our first podcast, or we put a post on our website on day zero of the company, we always said the path for AGI runs through software. It is not software. And we laid out this three-step master plan, which was step one: assist people in coding, just in the very early days. Step two: allow people, anyone, to build software. I think the world is clearly there right now. And then step three: generalize to all domains. And we're now really in that step-three moment.

And so already our models today have become generally capable across the board. But because we had kept our blinders on for those first couple of years on really software development capabilities, abilities as a proxy for intelligence, it allowed us to go really far, allowed us to push the things that mattered. In our view, what was really missing was pushing reasoning. Improving knowledge, you can improve knowledge outside the model. You can give it access to the right sources of information, but improving this kind of complex reasoning is what was missing.

So we're kind of converged on the same point. So today I use our models—the other day, I had it write a sci-fi book while I was reading it, which was a lot of fun, right? And my brother uses it in his growth marketing job and prefers it over using other models. And so we're already in that domain now. But software development is still, with our deep focus on enterprise and government, often the highest driver of cost and also impact of knowledge work in a lot of businesses.

It's not the only one, but it's a big one. So it's been an amazing place to enter into organizations with. And now we're starting to increasingly go beyond that. So we had a big announcement this week with a company called Redpanda we're big fans of, and we just integrated their 300-plus enterprise data connectors inside it. And the demo that we were showing at the New York Stock Exchange was one of an AML process in finance that our agents are end-to-end, together with these data connectors, doing.

And so we're starting to open up much more broadly outside of software development and come back to what we've always said: we want to be able to power all of knowledge work in the world.

Yeah. Five years ago, one of the first things was Point, which was like the code completion models. Today, they're table stakes. We have them, but it's not where the intelligence sits, right? So Malibu was our first big family of models. Within the Malibu models, they were originally oriented to be very capable coding assistants. Now they've become incredibly capable coding agents, and now they've also become incredible knowledge work agents. And so within that, those models are right now, for their size and weight class, best in class.

They have become incredibly capable coding agents, but they're not yet at what I would define as the frontier. So frontier today is OpenAI, Anthropic, Google. And that's why we have all of this compute coming online to scale up those model sizes. That's our upcoming family of models, Laguna, where we'll have a small, a medium, and a large. Small is finishing training in a couple of weeks. Medium actually starts training this week, and large starts training as our 41,000 GB300s come online.

We do have benchmarks.

So if you think about our benchmarks today, we primarily share them privately with the organizations that we work with. We beat GPT-5 Pro when it came out. And so it's a much smaller model, but it's really pushed those capabilities because of our work in reinforcement learning. And so we should do a demo after. I think we're actually doing a demo in New York publicly, because we've only done it privately with enterprises, in a couple of weeks at the AI Engineer Summit.

And so if anybody's interested, they can come and see it there. And I think that will be recorded as well.

So our view has been quite simple, actually, is that we will make models publicly available when we are clearly best on a very valuable axis. And I think that's important. It's not just about having hype on Twitter for a couple of weeks and having something out. You need to bring something in that can scale and that is valuable to the world. So before—and this gets to the commercial part of your question—we weren't at the frontier. Now we are getting increasingly closer and closer, and that's also opened up our market.

So before we were at the frontier, we picked a market where no one else could go, which was the defense industrial base and government. Because one of the things that we're willing to do, kind of from the enterprise DNA we have, is take our entire model weights and the full stack, all the way with the agents, anywhere the customer needs it to be. So we today operate on workstations that go into SCIFs. We operate on servers that are on-prem or even as far as air-gapped.

We operate in commercial cloud, like AWS, in private VPC environments. We also operate in more commercial, just regular cloud environments, and then on GovCloud, Top Secret cloud, kind of the places where the weights have to travel to the customer. And we did this because it wasn't just about the model. This is because we knew that as the world was moving towards agents doing increasingly more work in the enterprise, it was going to become incredibly important to build out everything around the model. So today we can rattle off a very long list of enterprise features that we've built over time so that these agents and models can actually operate in complex regulated environments.

And so that goes everything from a data access layer towards the managing of assigning agents across role-based access control, deep integrations with all of the kind of Active Directory-like systems that people have, all of the monitoring, audit logging that associates with observability, go on and on. And we battle-tested that in the defense industry because they're extremely large organizations, they're highly complex, they're highly regulated, and they often need segregated deployments for different missions that they're on. And so we've been scaling it up in that industry.

Now, as the models have gotten to a point where we say, okay, now we feel that we can compete, we're going to wider enterprise. And so you'll see us increasingly more showing up in kind of the large enterprise names amongst financial services, industrials, technology companies. And we treat our business as kind of two business models. On one hand, we want to make our models publicly available and allow anyone to use them. But we want to do so when we say, hey, we have something here that sets us apart, and the scaling up of compute is necessary for that.

Yeah, the Laguna family will be public. And then we have the part of the business where we go very value-add. And here we started with product. So we started with a coding assistant that sits in VS Code and IntelliJ, now also sits outside of it. Then we started with kind of the web interfaces that you expect to be able to chat and use the models, connected with data sources. But we found something over and over again, which was that a lot of these enterprises and organizations, they have incredibly high-value problems that can be solved with AI today.

Hell, in many cases with AI capabilities of two years ago or a year ago, but they're not successfully doing so. And a lot of that has to do with the gap between what the intent of the project is versus being able to bring it all together: the right data sources, the right context engineering, often additional reinforcement learning that's needed. And here we started building up a very strong forward-deployed motion. At Poolside, we have former Palantirians who came over and really kind of instilled some of that DNA, the DNA of what it means to find a high-value problem and come in with high agency and help a customer solve that.

Right.

Yeah. And look, what we found is that there is a gap between the skill set of doing traditional forward-deployed engineering, where you're focused on a high-value problem and using kind of piping data sources together and building interfaces on top, which is, by the way, an incredible thing. I have a lot of respect for everyone who has done this. The research engineering side is interesting because what you're looking for is the people in that who also have the experience and the natural tendency to work very well with models and with agents and can work on additional reinforcement learning if it's necessary.

And so we're taking these highly capable research engineers and we're putting them inside our customers. And that's something you're going to see us scale up increasingly more. It's why we're going to be opening an office in the UK, actually, next week. We're going to be opening an office here in New York and really kind of scaling up that motion of forward-deployed research engineering.

Yeah, so we've always been an American company. We've been incorporated, headquartered out of the US since day zero. And I think at any given moment, the balance of people will have always sat somewhere between 40-60 or 60-40, one way or another, throughout the life of the company. So building it as a global company was critical for us. But the decision we made early on, and I think we spoke about it on the podcast two years ago, is to hire researchers outside of the Bay Area, and particularly originally very centered on Europe.

And still today, if we look at where researchers for Poolside sit, they sit predominantly, 95%, outside of Silicon Valley. And this offered an incredible opportunity for us. It offered the opportunity to find highly capable, highly motivated people who were not in the echo chamber that is the Bay Area. It's one of the most valuable echo chambers in the world. I personally love it. I spend a fair amount of time there. But you've seen and probably realized that there's a bunch of things we've done that were quite contrarian to the belief of that echo chamber.

And I think that was partially unlocked by the fact that we sat outside of it and continued to do so. So we hire all across the United States, we hire all across Europe, we're hiring increasingly more in Asia as well as we're scaling up. But we have and continue to try to avoid as much of an echo chamber as possible because I think the world—we're very early in all of this still, and the path towards AGI is one that deserves to be built by opinions and people that aren't all the same.

And we have just found that there is no shortage of incredible talent in the world. It just might not be as obvious on a CV or LinkedIn as it is in the Bay Area.

Models reach the frontier. That's right now. I think we see a straight line towards this. You'll see the scaling up of physical infrastructure to both empower training even larger and more capable models, but also serving them to the world. You'll see us have forward-deployed research engineers in an increasingly larger number of enterprises globally. You'll see us just continue to work towards the mission. For us, we never want to lose sight of the fact that we are building this company and the economic engine associated with it because we think that the world that lives after this is one where a lot of abundance can be created—abundance through scientific progress, that's going to happen, but also abundance through the fact that the cost of goods and services ultimately depends on the cost of what it takes to create them.

And as we shift intelligence to compute, we can find ourselves in a world where we can drive that cost down. And if you look back at the last 100 years, there's no moment that I think any of us would rather live than today, right? And so that's its core mission. It always has been, and you'll see more of that out. And yeah, looking forward to probably being here in 12 months to tell you about what happened.

Appreciate it. Thank you, Matt.