Who Feeds the GPUs? Inside AI's Hidden $30B Layer | Renen Hallak, VAST Data

Thank you.

Right in the middle. That software infrastructure layer is the one that we do, in collaboration, of course, with the hardware vendors underneath us and with the model builders on top of us.

Yeah. So everything you would imagine from the old stack, we need for this new stack. We need to manage the compute, we need to manage the storage, we need to manage the networking. And then, as you go up the stack, we need to make sense of all of it, make sense of the data through a database, make sense of the compute through training and inference functions. And as you go even higher up the stack, and we shift from training predominantly to inference and agents and reinforcement learning, we need more and more tools to enable this new world, make it simple, make it safe, make it fast and scalable and resilient.

All of that sits in that middle layer. Sometimes I like to call it software infrastructure, and other times I like to call it the operating system for this new era.

I think even below the GPUs, the data centers are looking very different today than they did five years ago. The old stack, you had 10-kilowatt racks. Today you need 500-kilowatt racks. And so you need more power, you need more power density, you need GPUs, of course, versus CPUs. You need extremely fast networks and very large SSDs rather than 10-gig Ethernet and old hard drives. And so every layer of this cake needs to be redone. And from our perspective, you need much more data and much faster access to that data because we're no longer analyzing numbers and columns of a database the way you did with big data in the old stack.

We're doing pictures and video and sound and natural language, and that takes a lot more space. And that's before new generative AI data that gets generated. And of course, all of this compute, all of these GPUs need very fast access to this information, whether they're training on it or whether they're running low-latency inference functions. And so the entirety of the stack is new. And I think the culmination of this stack is what we call the AI factory.

I think it'll be for everyone eventually. The big labs are starting, they're ahead of the pack, and they're leading the way for everybody else. But anybody who stays with the old stack, with old business information systems, will get left behind. And so people need to start leveraging these new AI abilities. Some start by leveraging them out of the clouds, out of the hyperscalers, the neoclouds. Others build within their environments. But in all of these cases, enterprises will need AI factories in the medium term.

I think once you get to a certain level of scale, definitely it's more cost-effective to buy than to rent. And so, from that perspective, yes, you should build your own AI factories. But beyond that, over time, I think all of these organizations' IP will be distilled in models. It will be weights, and they don't necessarily need to build the base model. I like to give analogies of the human brain. We were all born with a base model, but then over time, as we travel the world, we learn new things.

We infer during the day, we fine-tune at night, and tomorrow we'll be a little bit smarter than we were today. And my model of the universe is slightly different than your model of the universe because we had different experiences. And so I think, in the same way, all of these agents will fine-tune models through reinforcement learning over time, and every organization's IP will be able to be distilled into the models that they own and the weights that they've accumulated.

I think, again, I like analogies. The analogy I would give here is, if you go back to the 1970s, you needed a PhD in computer science to operate a computer. Today, everybody can do it. And the reason everybody can do it is not because we learned; it's because operating a computer became much simpler. The operating system made it intuitive, made it easy for everybody to use. It also made it safe for everybody to use. These enterprises are under a lot of regulation that they need to comply with.

And so it is our job, as the ones building that software infrastructure layer, to make these new technologies accessible to the enterprises such that they don't need to think about it too much and that they can start adopting them at pace rather than get left behind and have new companies with AI experts take their place.

I think so. I think the physical location of this AI factory can be on-prem, can be in a colo, can be in an AI cloud where you rent it from somebody rather than manage it yourself. Eventually, my guess is that the hyperscalers will also enable these AI factories to be built within their premises. But so long as it's within your control and so long as you're the one that owns it in terms of, really, I think, owns the information, owns the models, owns the weights, owns the agents, I think that's the distinction that I'm trying to make here.

It doesn't necessarily need to be physical equipment within a building that belongs to you.

I, again, like to think of agents as artificial people. And over time, I think we're all going to have agents that work for us, whether we're people or organizations, for different tasks. And we'll have teams of agents that can help us develop software or personalized medicine or whatever it is that we want done, we will have them do on our behalf. And I think over time we will get attached to our agents and we will expect them to know things that happened yesterday or that happened a year ago.

And that will allow us familiarity with our agents, but that requires them to fine-tune models based on experiences that they had with us. And of course, that also makes them smarter and able to do more for us. And within teams, they can learn from each other. And I think that generates the next level of artificial intelligence versus where we are today, where everybody's based on—

Some base model from one big lab or another, and every few weeks we get an upgrade to that model based on what they did in their environment. In this new way, each one of us, each organization owns the IP generated through these fine-tuning mechanisms rather than letting the big model builders basically own everything over time.

You don't want to because you don't want to expose your proprietary information, your processes, your secret sauce out to somebody else such that they can take it and run with it.

Not necessarily. I think we need to build a way for people to monetize these abilities that they worked so hard and paid so much to get to. And so we need a way where, on the one hand, an enterprise does not expose their data. On the other hand, a model builder does not expose their weights. And again, the software infrastructure layer is the right place to enforce that type of trust. And over time, as the enterprise starts to generate their own models or fine-tune on top of base models, they should also be able to monetize the skill sets that they develop.

And so if I run a carpentry shop and my carpenter robots are the best in the business, I should be able to lease to you that brain module such that you can build a chair or whatever it is that you're trying to do. But I'm only giving it to you for a week. And at the end of that week, you don't have access to it anymore. Or maybe I'm selling it to you, but I still don't want you to see those weights.

And so there should be ways for us to build an intelligence marketplace.

Sure. So basically, what we're trying to do is fill out that software infrastructure layer. And what we realized was that the five-layer cake that we discussed is almost complete. The models don't really live on top of the software infrastructure. Models, over time, have become a resource, just like a GPU is a resource or an SSD is a resource, and resources need to be managed. And so this announcement is all about model management. How do we, as the operating system, now get an understanding that this model is really good at coding, and that model is really good at generating images, and this one is expensive, and that one is inexpensive?

And that way, we can understand which tasks belong under which models. And that's today, when we have four or five of them. As we start to reinforce learning and fine-tune, as we start to have hundreds and thousands of agents, and over time generate new derivatives of those models, that's going to be a much bigger task. One specific aspect of model management, which we're announcing—and we have a lot of the model builders as partners joining the announcement—is around confidential computing and enabling the ability for enterprises to run inference on-prem without exposing the model builder's weights.

And so both sides can rest assured that my data is not exposed and your weights are not exposed to me. And so with help from NVIDIA and encrypted memory underneath, we're able to make sure that no one sees what they're not allowed to see and to hopefully increase the total addressable market of these model builders into regulated industries, into large enterprises that are careful with their sensitive information.

Affinity for math, for sure. Whiz kid, definitely not. It's funny, when I came up here, the Math Museum is downstairs, and we had some big news this week about math. When I was in school 20 years ago, I spent six months of my life trying to figure out if P equals NP, which I hope your audience knows. But for those who don't, it's the question of: is it easier to verify a problem than it is to solve the problem? Which intuitively it is.

It's much easier for us to review a document and say, “Yeah, that works,” than to write that document, or to look at a picture and say, “That's beautiful,” than to paint that picture. But mathematically, it's unknown if that is the case or not. And interestingly, this is one of seven Millennium Problems that the Clay Institute, back in the year 2000, put a million-dollar bounty on. And I remember spending those six months trying to solve it because I thought, if I prove that P does equal NP, then I also solve the other six and I'll get $7 million.

And if I prove that it's not equal, then at least I get one. And after six months, what I realized was that I'm not nearly smart enough to do it.

Yeah, another one of those problems, Navier-Stokes, was solved by OpenAI, or at least they claim that they've solved it. And the rumors are, hopefully by the time that this is aired, some of those rumors will come true, that there's at least two more of the seven problems that OpenAI and Anthropic are trying to solve and are close to solving using AI. And these are problems that have been open for nearly 100 years. And for the last 26 years, there was a million-dollar prize associated with them, and still people were not able to solve them.

And so for the first time, I think we're at a point today where AI has proven that it can do things that we don't know how to do. But going back 20 years, it was clear to me that if we could get computers to help with these hard problems, that's the way to solve it, because people are slow and the evolution of our knowledge is slow. And if we are to see any of these things get solved within our lifetime, we need computers to speed us up.

We didn't know how to do it back in 2006. And so I went along my way and joined a few technology companies. The last one I was with was a company called EMC, which built large storage systems.

XtremIO, yes.

That's right.

That's correct.

Yes.

And that acquisition happened in 2012. And by 2015, it became clear that there was a shot at computers helping us think and helping us solve problems. Because neural nets, which again, when I was in school 10 years earlier or 15 years earlier, were a curiosity. They weren't really able to do anything. Now they were, for the first time, starting to show that they can do something. They were able to recognize which videos had cats in them and which videos did not have cats in them, which was, again, astonishing to me because suddenly we can try to mimic the human brain without really understanding the human brain.

And so immediately I tried to figure out how this happened, and it was very clear that it wasn't necessarily new algorithms. It was giving those neural nets much faster access to a lot more data that enabled them to perform these very simple tasks.

Correct.

2015, beginning of 2016, we started. Yeah.

There was no generative AI at the time. Google acquired DeepMind a year previous to that. But it was clear that none of the systems, none of the infrastructure that we had available to us, would be scalable enough, would be performant enough to enable this next revolution. The architectures were built on an assumption that things needed to be fast or big, and here you needed something that was both fast and very, very large. And so we needed to build a new underlying layer to enable the success of this next revolution, if it were to come to fruition.

In the early days, of course, our customers were not doing generative AI. They were doing large-scale analytics. They were doing autonomous driving projects. They were doing medical imaging analysis, genomics analysis, hedge funds.

Yeah, hedge funds were trying to get signal from news feeds and natural language. And so it was the beginning of AI, but before ChatGPT.

Disaggregated and Shared Everything.

So the old way to build scalable systems is based on a concept called sharding, what's known as shared-nothing systems, where you have a lot of nodes. Each node is responsible for a piece of the pie. They collaborate with each other in order to serve up application requests. And as you add more nodes, you get more performance, you get more capacity. It works well up to a certain limit. And so once you have too many nodes, the communication between them starts to show diminishing returns.

And so if you're analyzing text, it may work. If you're analyzing numbers, it works well. If you're analyzing columns of a database, that's fine. Once you grow beyond that, and today the amounts of data are four or five orders of magnitude beyond where they were back then, soon they will be seven, eight orders of magnitude bigger. It doesn't work. It breaks down because the communication grows quadratically within the cluster. And then, from a resilience perspective also, when one part fails, everybody needs to recover.

That takes time. You can't really have more than 100 nodes in one of those systems. To build AI, we need systems with many millions of nodes. We have one system at one of our customer sites that is today multiple exabytes within a single cluster, delivering tens of terabytes per second. This is tens of thousands of nodes even today, before the agentic revolution. And so to do that, we needed to build an architecture that ended up being the opposite of the way shared-nothing systems work.

Instead of having the SSDs, the media, directly attached to the CPU, it's on the other side of the network. And through a very fast network, and through new protocols like NVMe over Fabrics, and through new media types that allow us to save not just data on the other side of the network, but also metadata, we were able to have all of the nodes see all of the information as if it was directly attached. And that's what led us to being able to do what we call shared everything.

All of the nodes now don't need to communicate with each other because they share access to all of the information. That's the basic idea behind our architecture. And it proved really, really good for a new type of storage system for AI, and then for a new type of database for AI, and then for a new way to orchestrate compute for AI. And we realized over the years, our customers forced us to realize that it's not just storage that needs to be redone, but the entire part of the stack that needs to be redone.

It was terrible.

I owe you guys a lot of our success because, as hard as it was to pitch storage to VCs, now we're reaping the benefits of having a lot less competition than we otherwise would have had. And so, thank you.

The ones that we're competing with started three years before us or 30 years before us. And when we built this new architecture, we were basing it on forward-looking technologies. When we started in 2016, the underlying parts weren't there yet. They became available throughout 2017, 2018, 2019. And so anybody—

And so it wasn't just difficult to raise money from VCs. In those early board meetings, when they kept asking me, "What do we do if this part doesn't come to life?" I kept telling them, "We don't have a Plan B," which they didn't like. And so I was not their favorite person in those days. Hopefully that changed over time. They made some money. But yes, there was no way for a company that started before us to make those forward bets.

And so that's our advantage today. That's our moat: the fact that we're the only ones with this architecture. And it's true that there aren't many other companies in our layer, I think mainly because it's not sexy. It's a lot more appealing to build a model company or an application company than it is to build the plumbing.

That's right. We have one product; it's this operating system. But all of the parts that we're adding are based on data, and that's part of the reason why we can't take open source and bolt it on top. We have to write most of it ourselves.

Same as with storage, we realized that there was a trade-off between price and performance and scale and resilience and ease of use in the world of databases, in the same way that there was in the world of storage. We realized that because our customers told us that. We have a big customer, for example, in Singapore, and they have a lot of camera feeds coming in, and all of those feeds need to be vectorized as the images come in. And that results in trillions of vectors that need to be placed in a database.

There isn't a database that was built for trillions of rows. There isn't a database that was built for thousands of agents analyzing different aspects of what's happening in the system at the same time. Old-stack databases were built for big amounts of data, which today seem very, very small. They were built for people running queries and doing the analysis. But again, that old architecture, the shared-nothing architecture, is not scalable enough to the levels that we need today.

I think both of those companies that you mentioned started, again, about five years before us. And so they too are built on that older architecture. And they started top-down. They are based on underlying cloud services like S3, which were not built for this era of AI. And we're seeing that today. That's a big part of the reason these AI clouds are starting to pop up, because the traditional hyperscalers that built S3 built the old stack, and these new AI clouds are building the new stack.

And now the older clouds need to adapt in order to stay relevant.

They don't have to start by moving all the data to VAST. Usually, we start by just serving the new applications, just serving the new AI applications. Usually, what happens after that is that the customer, the enterprise, the organization realizes that this is actually useful, and I want it to access the entirety of my dataset rather than just this subset. The good news is that we support all of the old-stack interfaces. And so, if you're talking storage, S3, we support file systems, we support block devices. If you're talking databases, we support SQL.

If you're talking streaming, we support Kafka. And so it's very easy to plug in those old-stack applications onto our platform in parallel to the new agentic workloads and AI applications. And in that sense, we become a bridge between the old world and the new world as companies make that journey.

So because the interfaces are all standard, it's just as easy for them to move off of us as it is to move on to us. It's our job to make sure that they're happy. The one chart that I look at every morning is our customer happiness chart. That's the only thing we care about. And if—

It's three colors. It has green, it has yellow, it has red. And anybody in the company is allowed to move a customer from green to yellow or from yellow to red. But very few people are allowed to move them back. And so—

The only metric that really matters in order for us to move a customer back is that they say so, that they say, "We're happy now. Everything is good. Everything is solved." And yeah, that's what we work so hard every day to accomplish. I think the numbers show that it's working. The numbers that people like you usually care about are net dollar retention, gross dollar retention. We don't have any churn in the company. Our gross dollar retention, it's not 100% because a few of our customers have gone out of business over the years, but no one's actively decided, "I'm going to stop using VAST."

which is something I'm very proud of. And on average, our customers double and triple the amount of data that they place on our systems every year.

Training is super simple. Training, you just need a lot of GPUs, and you need to feed them with a lot of training data. Once in a while, they put down a checkpoint so that if something crashes, they don't have to go all the way back to the beginning. But other than that, there's not much to it. You need large scale, you need fast access. That's it. Inference is becoming a whole complex world of its own. Inference started by requiring low-latency access because people are on the other end prompting, and they want fast responses.

That meant that you need a distributed system close to where those people are, at the edge. That also means that you need a system that's always up. Somebody once told me, one of the big model builders, that if training is down, nobody notices. If inference is down, there is no service. And so it needs to be up 100% of the time. Resilience, distributed nature, low latency, very secure environments. All of that is important in inference versus training. But that's just the ground level.

Once we start talking about multiple models, once we start talking about agents, once we start talking about reinforcement learning, definitely once we expand outside of the world of data centers into physical AI and actual devices, each one of those adds another layer that means that we need to develop a new set of tools. For example, now that we have multiple models, each model is good at different things. And so I want to make sure that this prompt makes its way to that model.

Because it's good at software development. I want to make sure that this prompt makes its way to that model because it's inexpensive and this isn't a high-value task. I want to make sure that no sensitive information is exposed to this model because they're going to use that information for things that I don't want them to. Whereas here, I have a contract that protects me. And so just that aspect of it is starting to get complex once we start thinking in that direction.

There's all kinds of optimizations that we need to do around things like KV caching and saving context windows and making sure that we route the prompt not just to the right model, but to the right GPU that has that model loaded and that has my context in cache. Context is just the last few minutes of conversation. We need short-term memory, the last few weeks, or long-term memory, what happened two years ago, to also be accessible through things like RAG into these environments.

And so all of that requires us to manage data: the raw data, the generated data, the input data, the weights of the model, the context. And we need to make sure that nobody sees information that they're not allowed to. We need to keep track, as we do reinforcement learning, of what information went into fine-tuning each model and make sure that that information isn't leaked through interacting with somebody who's using that model, whether it's a human or an agent. It becomes very, very interesting as you layer one, two, three, four of these things one on top of the other.

So we need policies to be set. And today we just connect into your organization's authentication and authorization mechanisms. If you use Active Directory, if you use access control lists, we then know who is allowed to see what from a human perspective. We then extend that to agents as they inherit properties from the people that deploy them or from other agents that spawn them. And based on that, we know what's allowed, what are the rules of the game. Then we need to enforce those rules, which becomes very difficult when you're talking about multiple organizations.

Most of these model builders won't let you infer within your environment, which means that you just need to send your data to them and hope for the best. That, I think, has been slowing down the adoption of AI in a significant way. And so we need to make sure that it's secure. We need to make sure that it's safe. We need to make sure that there aren't any obstacles for AI to achieve its true potential.

Very similar, yes. So access control lists that apply to a user ID. Now this user ID is assigned to an agent rather than a person. We have access controls based on roles. We have access controls based on attributes. And so it's very fine granularity, and you can, in a dynamic way, set your policies.

Correct. And who they're allowed to talk to, because now you have multiple agents talking to multiple people. And so that conversation happens over a persistent streaming service that we provide that allows us then observability into it because we can query through the database and ask questions about those conversations. And so the entirety of this—remember, it's not in one data center anymore, it's across geographies—and so the entirety of this new world gets managed by this operating system from an observability and from a control and from a policy-setting perspective.

Especially when it starts to expand outside the data center into cars and humanoid robots and satellites, then the stakes get higher because physical elements can create physical damage. And so rather than just data safety, we need to think of things like actual safety. That robot should not pick up that tool, or that car should never accidentally hit a person. And so you need us to be in the data path to not just be able to report on these things, but also to intervene and enforce the policy.

That's exactly right. And I think for a long time, we've been hoping that interaction between AIs and interaction between agents will give us that next step function in level of intelligence, in the same way that when we talk, we make each other smarter and we give each other ideas. I think this solving of the Navier-Stokes problem earlier this week proves that that is actually happening. They set 10,000 agents loose at an age-old math problem, and within a few days, they came back with the result.

And so that's one aspect that was missing, I think, which is good for theoretical problems. The other aspect that's still missing is access to the natural world, which is good for physical problems. And so hopefully that brings us to a level where the AI can help us solve some of the big problems that we've been struggling with over the years.

Enclave, yes.

I'm not sure I like the name, but that's what we chose.

It's super simple. In fact, it all goes back to encrypted memory. And so if today model builders want to keep their weights very close because that's their IP and they're afraid of giving it to a million customers and having one of them take it and do something that they're not supposed to. On the other hand, a specific class of customers, specifically enterprises, cannot just give their data to the model builders in the way that they're expected to today. What we're doing is we're letting that inference process, rather than running within the premises of the model builder or within one of their clouds, run within the premises of the enterprise, whether it's a physical enterprise premises or a VPC, a virtual private cloud that they own.

And the way that we make sure the model builders are happy is by encrypting those weights, and we encrypt them end-to-end. We leverage hardware abilities underneath us from companies like NVIDIA, which is a big part of this launch, who allow us to bring encrypted data through the network into memory, from memory up to the GPU, and only the model builder's inference application is allowed to access it. And so we have full custody, chain of control, end-to-end, such that those weights are safe and the enterprise knows that nobody can ever see their information.

I don't think there's anything new in the underlying technology. It had to be plumbed because this was, as you say, available for a long time for CPUs. Now it's available for GPUs. And so that piece of it needed to be done underneath us. Our layer requires us to manage all of this and make sure we understand which model is which, who does it belong to, who is allowed to see it, where should we block it, those types of things.

I don't think there's a technological breakthrough that was required for this. I think it's just an idea whose time has come. And it also requires many companies, many parts of the ecosystem, to work together in order to provide this end-to-end solution. And so we're working with the AI clouds to do it. We're working with the OEMs to build physical boxes that can sit in the enterprise's data center. We're working with the model builders.

Exactly. Yeah. Companies like Cisco or Supermicro are now collaborating with us on this. And of course, the chipmakers themselves, like NVIDIA.

I think that's where the trust should be. That's the layer that is, on the one hand, malleable because it's software. On the other hand, it's low enough in the stack such that it can benefit all of the different applications. Historically, that's where safety was done in the operating system. And so we need it to be done in our layer. I hope that we are trustworthy. We're working very hard to build that trust with everybody else in the ecosystem. And yes, we're very, very humbled to be in the middle and in that position.

I think the demand is there today. I'm not sure it gives me comfort, and sometimes it scares me as to how much demand is there and the acceleration in demand, because every quarter our customers are coming back to us and telling us they need a lot more than they thought they did. We had a customer, one of these AI clouds, one of the smaller ones, come to us a quarter ago, and we told them, "We need to plan ahead for the next three years because there's supply chain implications and you need to be aware of this."

And they said, "We're probably going to need about 500 petabytes over the next three years." And last week they came back to us and said, "We're going to need an extra two exabytes on top of that 500 petabytes." And my expectation is that, from what I'm seeing in other parts of the landscape, they'll come back again in a quarter or two and say, "We need a double-digit number of exabytes. We undershot." And we're seeing that across the board, both in the smaller and medium-sized environments as well as in the big environments, where they thought they would need tens of exabytes and now they're talking in triple digits.

Well, the big labs are building because they have end customers and end users that are using it. And the AI clouds are building because they have both big labs and medium labs that are consuming it. And I don't think you'll find infrastructure that's just lying around and not being utilized at the moment. In fact, the opposite is true. There is a lot more that people want to do that they cannot do. Some of these AI clouds have stopped selling more capacity just because they're sold out for the next year and a half.

And so I think the limiting factor here is physical. It's land, it's power, it's chips. Building fabs takes a long time. And so, as fast as we feel this is moving—and it is moving very, very fast—the buildouts are still happening much more slowly than the demand requires from them. Will it sustain forever? I don't know. I'm sure over time there will be hiccups, but as far as I can tell, we're just at the very, very beginning of most of the world running on AI.

And if AI will really be able to do all of these things that we're starting to see it do, then it should continue at least for the next five to 10 years at this pace. So we have a lot of work to do.

So I think, going back to the beginning, the old stack and the new stack, I think there's a new ecosystem getting built around the new stack. And some of the older companies are joining this ecosystem, and some have not yet joined this ecosystem. And so it's lacking in liquidity at some points in this supply chain. We see the buildouts that need to happen in some points need money in order to finance them. And sometimes you need that money ahead of when the results appear and ahead of when you actually get returns on that investment.

And I think the companies that are in the ecosystem are a natural spot to get that liquidity because they see what's happening, they believe in it, they understand how big this is going to be, versus someone who's on the outside looking in that maybe feels this is riskier for them to finance. And that's, I think, what's happening.

We are.

I guess it's probably a combination, but I think it goes towards our business model. We sell software, and so our gross margins are high. And as we grow on the trajectory of AI growth, which has pretty consistently been about 10x every two years, we get more and more efficient, and we generate more cash and more profitability. The companies that are higher up the stack have a lot of compute costs that they need to do. And the companies that are underneath us in the stack are building hardware.

And so they too have a very different business model than the one that we have. But yeah, I think the combination of fast growth and efficient growth is something that we've built into the way we work from the early days. And it goes back to VCs not necessarily wanting to invest in the space in the early days. So we needed to be self-sufficient. And it also goes back to VCs not wanting to invest in the space in the early days.

So we have less competition than maybe exists in some of these other layers.

So they need access to power, they need access to hardware, they need access to money. And I think the ones that will succeed are the ones that know how to build. I find the end users come, what NVIDIA calls off-takers, and they want immediate access to compute. And if you have it, then they'll come to you and they're willing to pay. And if you don't have it and you tell them, "Let's build it together, we'll have it ready for you next year," then they'll go to somebody else.

And so I've seen the ones that are most successful be the ones that build in anticipation of demand rather than behind the demand.

And you don't have to own all of the layers of the stack. There are multiple layers of the stack underneath us in the neoclouds. But yeah, and I find the ones that have that combination of good relationship with the hardware vendors, access to power and land, and that are savvy in the way that they finance their operations, to be the ones that are most successful.

I think that the reality has proven to be the opposite of what you just said. The fact that we have so many of these new AI clouds and the fact that the hyperscalers have not beaten them yet, I think it goes back to the innovator's dilemma. When you have something new to build and you have this old cash cow to focus on, it's a lot harder than when you have something new to build and you don't have that legacy. But yes, regardless of where it started, I think these AI clouds have built up a skill set and a specialty.

And it's not the same to build this new stack in these AI factories versus to build the old stack and the way that the hyperscalers built clouds five and 10 years ago. And every day that passes, these AI clouds that are actually in the trenches doing the work, they're learning. And the hyperscalers that are sitting on very cheap financing but are not yet doing this work are not learning that.

And so the gap in skills just keeps increasing. Now, I think the hyperscalers have realized that. And if two years ago they were saying, yeah, we have this, we have storage, we have compute, we have networking, we know what we're doing, we don't need to worry about this new thing, today they're singing a very, very different tune. And I think they're aware of the fact that their lunch is being eaten by someone else. And I think they're reacting to it.

So it'll be very interesting to see how that dynamic evolves over the next couple of years.

I think this becomes a basic need. Over time, in the same way that electricity is a basic need or that water is a basic need, AI intelligence will be a basic need. We will not be able to live in the way that we would like to live without it. And so, in the world that we are in today, countries are adversarial one to the other, and they don't trust each other. And in the business community, as much as we have competition, I think we collaborate.

Very, very nicely. They don't want to be in a situation where the U.S. can shut down its AI, or that somebody else can shut down its AI. And so they're trying to build sovereign AI solutions. Obviously, not every nation can build the software. Not every nation can build the hardware. Not every nation can build all the different layers of the stack. But I think they get comfort from knowing that it's within their borders, at least, and within their jurisdiction from a legal perspective.

I think the best way for me to think about this is to look to history. And historically, hardware tends to commoditize relatively quickly because it's standard—not because it's easy, but maybe it's easier to copy. I don't know. The software layer tends to accrue a lot of value. If you look at the most valuable companies in the world today, if you look at Amazon, if you look at Microsoft, if you look at Apple, if you look at Google, those are the companies that built those software layers of the cloud, of the PC era, of the mobile era, of the internet.

And they are worth trillions of dollars as a consequence. The application layer—there tend to be a lot of applications, some of which will have a lot of value and others will not have that much value. But I think consumer loyalty is something that is also valuable in those previous revolutions. And so, the software infrastructure layer tends to accrue a lot of value, and then, on a more selective basis, at the application layer. Where the models fit into that is very, very difficult to say because we never had a model layer before.

So that would be interesting to see.

So I'm not sure those two layers will stay separate over time. Perhaps they merge back into one. In the old stack, we had only four layers. In the new stack, perhaps that merges as well.

So interestingly, other than as an investor, there's no legal document between us and NVIDIA. They don't resell our solution. We don't actually do anything together from a legal perspective. Having said that, they are our best partner by far. And at any given point in time, we have a high-teens number of projects that we're collaborating with them on. We have a triple-digit number of developers working with them. They have the same on their side with us. Everything ranging from new networking gear to new inference microservices to this agentic confidential computing project to the next generation of GPUs and DPUs and how we take advantage of them.

All of that we love because as we build this together, the solution ends up being better for our joint customers. And so we really enjoy working with NVIDIA, also from a cultural perspective. They move fast. They believe that anything that is physically possible is possible. They believe that it can be done when others believe that it cannot. And so it's been a lot of fun working with them, in parallel to it being also very lucrative because they are the ones that are creating these new environments and putting names on all of these new concepts.

And so they're leading the charge, and we're trying to, as much as we can, ride their coattails. Having said that, there is no type of exclusivity. We work with AMD, we work with other companies in parallel to working with NVIDIA. In practice, they do have the majority of the market, and so the majority of the deployments that we have are with them.

Well, you can't move faster than Elon. I was fortunate to see some of the ways in which he pushes his team to move faster. And it's super simple: you find the limiting factor and you get rid of it, and then you find the new limiting factor and you get rid of it. And that cycle keeps accelerating you faster and faster and faster. And that's what we like. The reason we keep innovating and keep accelerating and keep demanding more from ourselves is, A, because we're paranoid and we're afraid that somebody from somewhere will pop up and try to catch up to us.

But B, because we love it, we enjoy it. It's the joy of building new things and exploring and figuring stuff out that you didn't know how to do last week or last month. And our customers expect that from us. Our customers are the most demanding in the world, and they're all in a race, and they're racing each other in this quest for better AI. And they need us to never slow them down. We can never be the limiting factor in their progress and in achieving their success.

You have to talk to people and ask them that question. They know. As CEO, you don't know, but the people that are on the ground know. And so you need to build an organization that's as flat as you can possibly build it, such that you have direct access to everybody. And you need to build an organization where people are not afraid of raising those problems. There cannot be a chain of command because that slows things down. I think I heard this from Elon.

Bad things should be stated loudly and often, and good things once and softly. So we can't get into a mode where we're congratulating ourselves too much. We always have to be in this mode of, somebody's going to kill us. We don't know who it is, and we have to keep fixing everything so that they don't have a chink in the armor to get in through.

I think 10 years from now, again, assuming AI—and I think we're starting to see proof points—will not plateau at the level of human intelligence, but will continue beyond the level of human intelligence, and assuming it has physical access to all of the things that we have physical access to on this planet and in space, I think everything is different. Nothing is the same 10 years from now. We will not be needed for all of these tasks. And I think the way government is structured will be different.

And I think the way money is transferred and ownership of things will be different. And the way we think about how we live our lives will be very different. And so it's impossible for me to imagine that world, except to say that it's probably going to be a lot more different to what we have today versus where we are now and where we were 1,000 years ago. So I think in the next 10 years, we'll see more difference than we did in the last 1,000 years if this plays out in the way that it seems to be playing out.

We want to enable this so that it actually can happen and that we don't see obstacles in the form of, we don't have the required infrastructure for it to happen because we can only build this fast, and we want to build 1,000 times faster or 1,000 times bigger. We need to make sure that it doesn't kill us and that it's safe, and that along the path, we put safeguards in place and enable us as people to understand what's happening, at the very least, if not guide what's happening.

I tell my team all the time, if all the data is managed by us, we can't ask for anything more than that. We're still a ways away from that, but that's the ideal that we're marching towards.

Thank you.