State of AI 2024: Frontier Models, AI Geopolitics and The Robotics Renaissance | Nathan Benaich, Founder, Air Street Capital

Good to be here.

Yeah, so my day job is I run a fund called Air Street Capital. I invest, I guess, like you, in early-stage companies with a focus on AI. I set it up by myself about four years ago, still run it as a solo GP, invest in North America and Europe, and have kind of diverse interests that are oftentimes reflected in the report in an editorial capacity. I think the main reason why I enjoy making the report, and the kind of original reason why we did it, is because I think making contributions to the ecosystem that you work in and work with is just, like, a nice way to be a value-added contributor.

And I think in our space, in AI, everything is so deeply technical, and a lot of the breakthroughs do arise from universities or industry labs, that I think to be as smart as you can be around where progress is going, you have to understand where it comes from and how it works. And then the second thing is that there's so many different participants to the industry, whether it's now politics, big companies of all different sizes and sort of sophistication of AI, and universities, et cetera, that all these different players are experts in their field, but they don't sort of have the time to really pop their head over their parapet and perceive what's happening in their space.

And the third one was just, I was, like, a fan of Mary Meeker's Internet Trends report a couple of years ago, and I was kind of sad when she stopped making it. And I just felt like there should probably be something like this for AI. I don't think anybody enjoys the sort of consulting, buy-my-product thing. And so we wanted to make something kind of rough around the edges where it literally looks like someone's behind the desk typing things out and messing up the formatting and stuff.

And that was reflective of progress and took an opinionated view that was hopefully sufficiently good that if it landed on the desk of the CEO of an AI company, they would think that their work was represented properly. But then if it hit the desk of somebody in policy, that they could actually digest what was going on.

Yeah, there's no correlation between that. It looks good post facto, but it's not a vision of VC plus content, plus media. A lot of things just kind of come across because I just feel like it's a good idea and I'm like, no one's doing it. And so we should just try.

Yeah, well, I guess the prior is I like producing kind of long-form analytical content. I did undergrad, did a thesis, I did a master's degree, I wrote a thesis, I did a PhD, I wrote a thesis, and I've just been kind of trained to do my homework on time. I think if I had to cram this, it would be really painful. And so on a monthly basis, I'd already, since 2015, started compiling papers, industry trends, companies that were getting started, and just kind of knocked together an analysis of why I thought they were interesting and what people should know about it.

And so we've kind of labeled this, like, Guide to AI, which we publish on a monthly basis. So at least that provides kind of a yardstick every month for what was interesting in that space of time. And then when it comes to doing the report, obviously we have, like, kind of 10 months of history that then we can look back at this monthly yardstick and be like, the stuff that was included actually had faded away or changed, or it's not cool anymore, or this thing is actually way bigger than what it was when we wrote about it.

Yeah, I think it's accurate. I mean, there's been vibes in different kind of flavors. One is, like, when you're trying a model, oftentimes builders will talk about the vibe, like what it feels like when you're talking to it. That's like one category of vibes. But then the other major one, which we highlight particularly in the politics section, is just, like, the sort of pendulum switch from existential risk: if we get this wrong and if we scale it too fast, we can potentially generate the extinction of the human race, basically, to switching to the complete other side of, we need to scale these products.

And these models are very useful. We probably have a technical overhang right now of just more products that we could build with the tech that we have. And there's a big race of trying to make money on this. And so what I found pretty surprising is just how powerful this vibe shift can be, where you're like, the same individuals who were in front of government leaders 12 months ago kind of warning about this are the same ones that are buying billboards in Paris or in New York saying, like, please use my app.

So yeah, it's tough for me to understand how that can occur and how that same individual can maintain these two modes at the same time. And maybe it's just smart people change their minds, and that's how it goes. And I'm happy the shift has occurred that way because I felt like the x-risk kind of debate was kind of overblown.

Yeah, I guess it could be a few things. I mean, there were statements like, "We should be bombing data centers." And this was, like, a major article in Time. These things are not a joke. It's wild. And there were individuals on CNN with Christiane Amanpour telling millions of people that AI systems are like living things, that we grow them, we don't code them. This is scary to the normal person. And so what's happened since then? I think there's been enough research now that the general population doesn't really fall for deepfakes, whether it's calls or synthetic media. It doesn't really change consumer opinion that much.

People are not dumb. I think with models, people have gotten used to them quite fast. I think it's still probably hard to determine, hey, this text is written by ChatGPT or Claude, unless it says, "Certainly," or, "Ah."

I just noticed it the other day, actually. It was like somebody wrote this text and edited by ChatGPT, and it said, "Certainly." I'm like, this person definitely doesn't say that. And then Claude is well known for, "Ah." Yeah. So unless there are these artifacts, it's very hard to tell. And then I think the other part is, I think tech people in SF obviously live in a bubble, and it's quite techno-utopist. And I think they underestimate how much human inertia there is to the diffusion and adoption of technology in the real world.

Because in our job, the number of times where you meet a company that describes to you this problem in an industry, and it's not like the product—it's not like that industry has the inability to source the technology to solve their problem. It's like people don't want to use it, or there's human inertia against adoption. So much stuff in industry or manufacturing is the equivalent of pen and paper. You could use a SaaS, but it just doesn't happen. So I'm not necessarily sure that, should AGI come, then from one moment to the next, all these companies are screwed.

And I don't know, the world sort of, like, tectonic plates shift that fast because human inertia is real.

Yes, this feels like Twitter comedy.

The major shift is: first, we started with single modality, just language input, text tokens, output text tokens. Then we started to do two modalities, maybe text plus image, termed vision-language models. And that's been really powerful for enabling this kind of renaissance of robotics, otherwise termed embodied AI. And then now we're maybe adding a third dimension, which is audio, because oftentimes images and video come with joint audio, which probably describes words. And so all these modalities fit well together, and models trained with all these modalities can be more powerful.

One shift is this growth to multimodality. Llama has it, so Meta's Llama has it. Anthropic has it, not all the modalities. OpenAI as well. On that topic, I think what's been interesting is China was not in this fight 12 months ago and now is very much in it. There are models like Alibaba's Qwen, and then the spinout from a quantitative hedge fund, DeepSeek, which publishes code models and others. And they've been actually a very lively contributor to open source.

And on some of these vision-language model benchmarks, they perform incredibly well. And that's interesting because, A, China hasn't been, or is certainly not portrayed in the West to be, the most open ecosystem, number one. And number two, they've been victims of sanctions from the U.S. government and other countries that are trying to rate-limit access to either GPUs or EUV, these lithography machines from the Netherlands that are used to make these chips. So despite all this, they're still coming out with good stuff.

If he doesn't know, I probably don't know. He's the king of open source. But yeah, it's interesting. And Western companies are using them. I think the data as of a week or two ago from his platform, Hugging Face, is there's probably over half a billion Llama derivative models that have been downloaded. I think Qwen is growing rapidly as well. So that's one part of this debate. The other one is, of course, this constant fight between OpenAI, Anthropic, Google DeepMind, GDM, as being the main contenders.

I think on the one side, you could say the gaps seem to have narrowed in capabilities. When you're looking at these various benchmarks, they sort of look increasingly esoteric to the non-expert perceiver, but the gaps seem to be pretty small. This is outside of vibes and what they feel like. And I think there are consumer preferences for one or the other. But while the gaps have gotten small and Meta has these ginormous systems they're giving away for free, which are interesting in their own right, a sort of satirical take on all this is you could have basically looked at the state of the landscape—who's number one, who's number two, who's number three—12 months ago, and then just deleted Twitter, not read any machine learning news for 12 months, and then basically turned it on again.

And basically seen the exact same thing. In fact, today I think there was some report from Ramp Data, which looks at credit card spend from their corporate customers and breaks down which model their customers are using. And they made the claim, "Oh, look, there's a sort of fragmentation and more models are being used." And so, look, there's happy competition, but really, when you sum the makers of all these models that are getting used, it's still like 80% OpenAI. That hasn't changed.

So it doesn't look like fragmentation to me. It looks like domination. So, yeah, that's, I think, one of the main things. And that probably also goes to vibes. It's sort of like Pepsi versus Coke.

Yeah, it kind of tastes the same, but it's very hard to convert a Coke drinker to a Pepsi drinker. It just doesn't happen.

Yeah, it's probably one of the best ROI trades of a public company in a long time. And basically the chart shows that from the launch of metaverse, companies saying, "We're going to invest, I don't know how many bajillion dollars in metaverse," to basically, "We're going to stop doing metaverse," when the stock price was just bleeding. And I think various shareholders had written letters to tell them to stop, et cetera. That was like negative $600 billion or something like this in market cap for Meta.

Two trillion—sorry, three trillion—market cap appreciation, which you could argue, who cares if anybody's using this stuff? That in itself is amazing shareholder value creation. Yeah, there's been a lot of downloads. A lot of companies are using the Meta backbone, or at least the pretrained model as a base for downstream tasks. And I think that's been very useful and will probably continue to be useful. I think there's some implementation details about whether it's dense or sparse or whatever, but I think the question I ask myself is, okay, if there are half a billion downloads on Hugging Face of this open-source model and they're pouring a bajillion resources into this, why is it that OpenAI and Anthropic's revenue keeps ripping?

Why is it that it appears that developers and companies still vote for closed models that have more convenience, or are just reliable and fast? You don't have to worry about it. It'd probably be a bad analogy, but why is it that individuals who live in the West who are sort of part of a middle-class-or-up demographic buy iPhones? I don't remember adding somebody's phone number recently that's not an iPhone.

And so that just provides you with a full-stack, nice experience. It just works. It always gets updated. And that seems to me very much like OpenAI. And then the Android is like, you can fork it if you want. You can do some funky stuff with it if you want, but people don't really do it that much anymore. And then the challenge with open source too is, who's going to run it? How are you going to get big distribution?

Who's going to run inference for this? Who's going to host it? And then when you look at Google, they have their own effort for their own models. They're probably going to prioritize that. Amazon also has Amazon AGI. They pseudo-acquired Adept for this and a few others. So I'm not sure if they're going to host it. And then they have the Anthropic relationship. And then Microsoft is in this constant, I don't know, frenemy situationship with OpenAI. So they're probably not going to host it.

We're not going to invest very much in it. So who's left? Probably Databricks. And Databricks and Snowflake, probably more Databricks, will be this enabler of open source. But yeah, it's tricky.

Workplace, I think.

Yeah, I don't know, it's hard to give him strategy advice, but it seems like it's a shame they don't have a cloud business that they could at least serve this stuff to customers if they wanted to. But I do think ultimately that just comes back to what's Meta's core business? Ads. And I think in his Meta Connect presentation not that long ago, he gave some stats around how consumers click through at a higher rate. And it was material.

It was like somewhere in the range of 7% to 10% click-through improvement on generative ads, and that they've served like tens of billions of ads in the last month. So that's the ROI right there.

Yeah, that's a major one. I mean, I'm probably included. I think anybody who tells you, "I'm going to launch a product, it's going to make billions of revenue in one year," I mean, it's hard to believe, right? Because it never happens. But it did this time. So that's one. And then the second thing is that these models would be so expensive to run, like, there's no margin in it. And that one seems to be changing a little bit, whether it's just the sheer cost drop from the most expensive models, like a year ago, to now.

0.06 per million tokens. Not the same level of intelligence, but not that far off. So I think most people I talk to who are close to the coalface working on inference improvement say that you will likely have similar quality of intelligence for much lower price and much smaller models as we kind of improve things like knowing what pre-training data to feed it, sort of the idea of curriculum learning, like very much the analogy to going to school. You don't get taught PhD-level physics when you're like 10 years old.

You get taught the simpler stuff first, and then things like how to refine post-training, like what kind of examples should you give it? And just given that, I think we probably forget to include this in general discourse, but until ChatGPT and AI became super hypey, there were not that many contributors who actually worked in this field. I think there was some data from various firms we included in prior reports, but it was like, I don't know, 100,000 machine learners in the world, a million machine learners in the world, like really small numbers.

And now I think you have everybody who spent years doing ad optimization, infrastructure optimization, DevOps, all this stuff of making software run really fast, who are now making AI run really fast. And these people are really smart, and they will undoubtedly find issues that AI architecture people will not have been experts in. And so I think that's also what's driving some of the cost reduction in addition to, of course, good old-fashioned price wars.

Yeah, it was something like, just directionally, retention after one year in the 2022 cohort was like 43%, I think. And then the cohort from 2023, after 12 months, was like 65%. So that was pretty material. And then we also showed the quarterly billing. So, how much individual Ramp customers were spending on AI products every quarter. And that's roughly doubled, or at least grown by 50%, which is like an argument against, like, hey, this is all demo where people are not really using it for real.

And then in the next slide, or very close to it, we have data from Stripe, which I was looking for last year: examples of this. And now I think it's pretty clear so far. And it basically looks at the 100 most promising SaaS companies on Stripe and the 100 most promising AI companies on Stripe, founded before 2020 or after 2020. And then it charts how long those companies took from their first sale on Stripe to reach certain revenue targets. And it's something like, I think 2020 is obviously not GenAI, but pre-2020 it took like 12 months or something like this.

Maybe you have the data.

That's crazy. It's like a year and a half versus five years.

I mean, this is insane. So you could argue, okay, whatever goes up that fast comes down that fast. But, I mean, it's pretty amazing we got there in the first place, and I'm not sure we're going to come down that fast. So I think that is really striking. And they sell internationally way faster. That's the other part that we didn't include in there. I think the interest was so big that I think expectations were, like, we would immediately see the result of this everywhere.

But the reality is tech just takes time to diffuse. Patterns, like user behavior patterns, take time to change. I think when you look at certain demographics, like age demographics, people adopt things quite differently. Older generations, when you show them ChatGPT voice mode, it's magical. And then younger kids using voice notes and text-to-speech, this is normal. So I think for us, we're still used to Google, and we train our minds so much to Google, it's a bit harder to move. But if you're in this, like, I need to learn things really fast on the fly, which is like what we do every day, it's pretty hard to do that when you're in front of Google.

But if you're in front of a generative search solution, it's way faster. And then I think we can get into the avatar stuff, video and voice.

Yeah, I think a good litmus test is if you have a meeting with somebody who's knowledgeable, smart, but they're not in AI, and they say, "Hey, Matt, why should I be excited about this? What's cool?" What is the demo you give them?

And for me, the demo I give them is voice generation, because I think it's basically solved and it's magical. And this was not possible before. I mean, Deep Speech 2 was written 10 years ago, which was the paper at Baidu, which has all these authors on it that have now gone on to found Anthropic, like Adept. They lead teams at NVIDIA, et cetera. And that was the first paper that really showed scaling laws for deep learning in speech: scaling models to large high-performance computing clusters and showing more data, bigger models, more parameters leads to lower loss.

And so 10 years on, now that you can clone your voice with probably a few minutes, if you want it perfect, like 30 minutes or something, reading arbitrary stuff, and then you can just sit there in a meeting and say, "What do you want me to say?" You just type it and it's there. It's amazing. So that, and then doing your avatar and that plus your voice, whether it's video or through a call, is, I don't know, just magic.

And I think this is the main thing for me. It's that classic Wait But Why chart of how people can perceive technology progress, where you're sort of on this, I don't know, 1% or 2% compounded annual growth rate curve, and you're like, "Oh, it's kind of going," and then you have this crazy wall in front of you, but you can't see it until you're really there.

That's what it feels like here. Which is why earlier in January, we did a five-year retrospective of the report, which is to say, what is it that we have available today that basically we take for granted? And if you showed this to somebody back when I started investing, or when you started investing, you would literally say it's magic. And I think that's hard to argue with. And so that kind of progress is truly insane, which is the thing that's nice to remind yourself when you get jaded with this.

Yeah, I'd say I'm trying to find the classic: what is not consensus now? What are people not believing? What has been tried, and they're like, "That's not going to work," but for some reason you think it will in two, three years. And obviously, if you're wrong after three, four years, you're basically dead, and wrong timing, et cetera. So, taking that lens, I was excited about AI and biotech five, six years ago, made a few investments in there. Defense was interesting in Europe.

It played out a little bit more in the US a few years before, but pre-Ukraine, there was really not that much. And now there's more and more.

Yeah, it's less contrarian. Everybody's talking about it. We can debate whether investors are actually wiring money to companies, which I think is not really happening that much outside of the big kind of proto-winners in the US. And then only one or two years ago, robotics was still like an iron touch in the market. And now, with these vision-language systems and other things, it's now like a renaissance. And I'm trying to ask myself now, what is the next thing there?

I don't have a super good answer to it, but I do think that the surface of products that you could build in the consumer/prosumer space is vastly bigger than what you see in enterprise. In enterprise, it's like search, retrieval, summarization, some kind of reasoning agent for blah, blah, blah. I don't like the word agentic.

Well, so I just think it's too techie. And actually, I had this conversation with somebody who was like a head of design for a consumer GenAI product. And he was saying, a big test for me is if I can show it to a normal person and they look at the interface and they know what to do with it. And you kids that are saying agentic workflow clone, normal people think that's scary. That's not a product somebody wants to use. So kind of get out of your bubble with real people and try to understand how to relate to them.

So it's too buzzword-y a moniker for me.

Yeah. Now that voice cloning is super easy, what can you do with that? I think there's all sorts of consumer, social, productivity—I don't know—just little micro apps you could build now that you can code basically for free. Perhaps you could expose some of that capability to an end user so that they're not stuck in this, like, I'm an average user basically just stomaching the monthly release cycles from a big tech company. Like, why can't I customize my thing?

Why is my camera so dumb? So I'm trying to explore that and try not to be too opinionated on what the starting product is, but find people who are opinionated, tasteful product engineers and product designers, more so than super deep AI people. Because I think we have the primitives now that you can benefit from all the good stuff that SaaS people had for the last 15 years, which is like, hey, I have a hypothesis. Cool. I can just spin up this, spin up that, spin up this, and have my service and test the idea without burning through my whole seed runway.

I think back when I started, and certainly probably when you started, it's like you'd probably burn your whole seed runway trying to figure out that your thing even works to go test the hypothesis with somebody who may care about it. And then it's like game over if you lost.

Pretty much, yeah. In the starting days, yeah. I think if I were doing something, I'd really look at what system can I use out of the box, use some sexy prompting, sticky tape, whatever, and then use that to really test the product hypothesis as much as you can. And then if you can start scaling your product with some third-party system, go for it. And then maybe when you get to the point where you need to assess unit economics, maybe you learn enough about what the user is doing so that you can see, hey, look, there's a distribution of tasks that my model has to answer and solve for my customers.

Maybe only 10% of that distribution needs this sexy, expensive model, and then the rest I could probably do with something else. Then, until we get to the point where intelligence is too cheap to meter, et cetera, you might as well just peel the onion and say, hey, for this, I'm going to use this model; for this, I'm going to use that model, and then just optimize it. And maybe at that point you need somebody who's a bit closer to the weeds and can optimize your stack for you and really juice out your economics and stuff.

But I think until that happens, eat the free lunch.

Yeah, well, I think the moniker would probably fall under embodied AI, which I first started hearing really through Alex Kendall at Wayve, who's been professing this for his whole journey at Wayve, which is like seven years or something. This idea of fully end-to-end learned robotic systems that don't operate only in the digital world, but have some physical embodiment and can navigate this world. The most hypey instantiation of that today is humanoids. I don't think it's a mass VC product yet, but certainly everybody's talking about it.

And then a derivative of that is like a robotic foundation model, like some general-purpose system that can control different robotic hardware, whether it's like a four-legged dog or a bipedal robot or a drone. There's some companies purely working on models there that have raised a ton of money at very high prices also. And then there's probably the more prosaic but useful application, which is warehouse robotics, where I think we've had 10 to 15 years of companies try to train pick-and-place robots.

And generally meet a customer that didn't really want them or hadn't invested in the sort of infrastructure layer, which for them would be warehouse automation systems, which just move parcels around. And those have become way more popular post-COVID. Labor issues have really been a thing. And now you no longer need to train a single model for every customer where you say, hey, give me your data, I'm going to annotate all these different products and train a system to work specifically in your environment.

Now it actually looks increasingly like you can take one of these vision-language systems and train a general-purpose system that can work across all your customers. And that's a pretty big unlock because the go-live time and therefore ROI is way faster. So I'm excited about that and to see where that goes.

I think this is so hard to time. Even a couple of years ago, some people were thinking it would be great if AirPods had solutions for the hearing impaired or could do noise cancellation and stuff. And it's like, now we have it.

So it's hard to—it's like, the AirPods are so great. And then to compete against that as a startup, you're going to have to basically build an AirPod plus everything else and compete against a trillion-dollar company. Again, like Coca-Cola, Pepsi, this is hard to compete against. So I don't know. I think consumers are very fickle with hardware. In that sense, it's been pretty amazing to see the positive reception for Meta's glasses and also for the recent Orion product.

I mean, I can't remember a time where tech analysts and commentators were as positive on a Meta product release. And if it's true that Apple is pulling back the Vision Pro, I mean, it's pretty astounding. And maybe that's an interesting insight. People like the glasses because the music is great. The AI sucks, basically, as far as I can tell from people, but the music is great. So I think you have to crack an everyday behavior and get to the point where it's good enough to compete against Apple and other companies.

And then maybe you have a shot at winning more of the experience around the physical product. And yeah, like the pins, there were solutions looking for a problem.

Yeah. If you want to go back in time, take a flight.

I mean, I think until May, Claude was not available in Europe. Apparently, OpenAI doesn't have plans to release Advanced Voice Mode. I remember requesting not to have my Facebook data be used in GenAI products, which I had to do before the end of June as a European resident. And then xAI had a bunch of run-ins with, I think, the Irish administration too, around using training data from citizens there in their Grok models. And so they just gave up. And so I'm not using it at all.

So, yeah, it's been tough. I think I even saw on Twitter trending a picture of Apple Intelligence US, and it's like, oh, we do all this sexy stuff. And then in Europe, it's like, we'll tell you when you have a phone call or something like this. So, yeah, it's pretty stark. And at the same time, it seems like there's some hints, at least from what we read online, of buyer's remorse among some individuals involved in the EU AI Act. They might have overstretched. And then the Draghi report around, like, hey, we're basically back on economic progress because we have too much legislation and we didn't invest enough.

It's sort of like the classic facepalm.

I think the inertia is too big. I mean, Macron has had this narrative for a while, and France was one of the bigger defenders of open source, et cetera. But I don't think it's going to change much. I think the UK had a cool opportunity because, I mean, I think Brexit was a terrible idea, for the record. But the value of that, at least, is they can define their own policies, which they did well during COVID with approving their vaccine really fast.

It was not a great vaccine, but the approval process was good. And so, in theory, they could use that to be competitive in AI. But again, I think the country has all sorts of other problems that they're working through, which are probably higher priority than solving for AGI and things like this. And then other countries just seem like they're in complete disarray and have basically no growth. So it looks depressing to me, to be honest. And then it doesn't help that on the Twitter feed, every other week or something, there's somebody that's like, I'm so happy to announce I'm leaving London for New York or SF or something.

I don't see that many people going the other way.

Yeah, I mean, look, I think there's fantastic contributors there. It's this classic narrative that hasn't really changed. So many of the top contributors in AI are either European or spent most of their educational career in Europe, including Nobel laureates. Many amazing companies are based there and would open offices there because of that talent. I mean, Europe has produced pretty impactful deep tech companies as well over the years, whether it's ASML or Novo Nordisk or Spotify or other things, right?

DeepMind. So it's like we have the capability to do it, but it's become more difficult, I think, given the federation of interests. It's not like a unified place. And there's something to say about unified authority and a single country, single market, more risk-taking mentality, and culture is very hard to change.

Yeah, I was born in Europe and educated there as well. I mean, other than US undergrad. But I sort of see myself floating between the US and Europe for work.

And trying to do whatever we can to change it, whether it's trying to get the UK government to push for better policies around forming companies out of university research. If we want to have sovereignty in energy, AI, defense, et cetera, we need to create a positive engine for forming these companies. And that's not really been the situation in the UK and Europe for many years. Now, our latest thing is trying to push for better procurement in the Ministry of Defence and other defense organizations across Europe to try and get more startups and tech companies in there.

So, yeah, we'll try and put pressure where we believe we have insight and some valuable opinion to share and some context. And I'm not partisan. I'm not nationalistic either. I'll go where the opportunity is. So I hope it always comes across as, like, this is just a genuine opinion from somebody who's lived through the system and wants to see it improve. And if it doesn't improve, then it's just sad to see.

Yeah, for sure. But I think there's a lot of entrenched interests for the financial services industry, for technology, for progress, for people creating a better life. That's like the whole purpose of, I think, why people came here in the first place. And then the second thing is, I think to me it's like, for so long as good people still want to come here, no matter what policy comes into play in this country, that's pretty hard to fight. So that magnetism, I think, would still make me bullish about the US.

If that starts to change and people are like, "I don't want to come here anymore," then I think you have a problem.

Predictions?

So we do predictions mostly to keep ourselves accountable, to sort of show that this is not a marketing exercise. I'm not selling anything. It's really our view on where things are going. So, yes, we make about a dozen predictions every year, and we grade them. We try to be impartial about it and provide evidence where we got it right or got it wrong. And on average, it seems like we get roughly 50/50. So there you go.

And so, yeah, one of the major predictions I think we got right a few years ago was that when Arm and NVIDIA announced that they were going to merge, we were like, "This is not going to happen on competition grounds." And that eventually came true about a year or so later. We made a similar prediction again last year, I believe, where we said that the CMA or the FTC or some other regulatory body would start investigating acquisitions and these massive pseudo-majority investments between Microsoft and OpenAI or Anthropic and Amazon.

That seems to be happening also. We've been pretty bearish on alternative chip companies for a long time, saying that there would either be consolidation or failures or acquisitions of some form. That's happened to some degree, but not to the extent that we thought. I thought it'd be much more of a bloodbath.

Yeah, sort of a last-minute slide I made, just because the best way of describing my feeling about alternative chip companies is: okay, how many billion dollars have been invested in an NVIDIA competitor over the last few years? Call it seven private companies and $6 billion invested in those businesses. And so we took the amount of money that was announced in that funding round at that time and then divided it by the NVIDIA share price on that day.

Calculate the number of shares, and then sum the number of shares that you would have alternatively bought with NVIDIA with that $6 billion over the last six years. What is that worth today versus what is your investment in this basket of alternative NVIDIA competitors? And it's basically like $6 billion turned into $30 billion, roughly, of which half is Cambricon, which is a Chinese listed company. And then NVIDIA would be worth $120 billion. It's like 20x in a liquid market. So I don't really see how this is going anywhere. I think NVIDIA for sure is real, and we have some really interesting data about just how much longevity that company has in its products.

So we chart the use of specific chipsets in published open-source AI research every year. And so there's, I don't know, tens of thousands of papers or something like this. And if you sum all the papers that use NVIDIA chips versus all the papers that use TPUs, FPGAs, ASICs, Apple, Huawei, the sum of all NVIDIA papers versus the sum of everybody else, the delta is like 11 times. So it was 19 times last year. So it's dropping a little bit, mostly driven by the growth of Google's TPU usage, which is up like 5x year on year.

But the chasm is massive. And the other thing that's interesting is the types of chips that researchers are using shows you just how long they are relevant. So, for example, last year, the most popular chip was the V100, which was released almost six or seven years ago. So we can chart basically the growth of papers with the V100 over time. It peaks last year. And then if you sort of unscientifically complete the Gaussian, that length of time is like 10 years.

So this company is shipping new products twice a year on the chip side, not even including the interconnects and NVLink and other things. And each generation is useful for 10 years. How do you beat that? It's truly astounding. So, yeah, I thought there'd be a lot more carnage in the chip space than what we've seen. So sometimes we do repeat predictions just to enforce our conviction.

Yeah. So this year we predicted that there would be some kind of blockbuster app, call it Apple Store top 100 or top 50, that was fully written in a generative fashion, so that some guy or girl who doesn't know how to code wrote this thing and it just went viral. Sort of like Flappy Bird or something, but generative. And just given how much hype there is around, "Hey, I spent 15 minutes and I wrote this Mac app."

I have friends who do this. It's pretty wild. It's making me want to try things as well. And we also predicted a year ago or so that you'd get the first generative music track that hit a top Billboard chart. And I think that happened in Germany, actually, on Spotify. So, just some favorite snippets.

Yeah, the TL;DR is I feel like it's going to be self-driving. The progress so far looks exciting, the demos look good, but we obviously don't see everything that's happening in the background. And I think it's going to be a long slog.

Yeah, just delay after delay and overpromise and overpromise.

But now you fly to SF and Waymo is epic. I have friends who put their three-year-old in there, and that kid calls it a robo-car and is excited to ride it on the weekend.

Yeah, it just gets boring.

And I just think about these kids that grew up in that era, and then when they become teenagers, like, "Why is somebody driving my car? Like, get out of my car." Yeah, just the same way that, why did you have to take—

The driving test? Yeah, exactly. Kids use Uber now. They don't drive. But next generation: I don't want somebody driving me at all.

Yeah, this is my space. Also, I think it helps the cars are really plush. Yeah, exactly.

So, yeah, that's been a really cool turnaround. And perhaps another contrast of where we were in 2018 versus now is just the general capability of visual common sense. I show you a scene, what's happening in it? I mean, this basically is solved right now. Machines can describe this in intricate detail and get the nuances pretty well. But it was not too long ago, in 2008, visual QA was really hard. You showed the computer system a picture of a baby holding a toothbrush, and it would say, like, "A young man holds a baseball bat," or something.

And there were memes of this on Twitter, and it was so easy to spoof and brittle as hell. And it was, I think, with iGPT that OpenAI started showing that, hey, you could treat an image as a sequence of pixels. And so you can process pixel by pixel as a sequence instead of doing these image patches. And the model could somehow learn to do completions that way. So I think that was, like, when we put this in the report a few years ago, I think this was, like, the first sort of sign that—I think it's Andrej Karpathy who edited the report at that point—said this is, like, a general-purpose architecture that scales way better than convolution.

So anything you can model as a sequence, turns out this thing can learn.

I'm going to shamelessly say Twitter or X. I think that's amazing. Most of my Twitter feed is machine learners, so this stuff just pops up. But I think if you're interested in semiconductors, Dylan Patel at SemiAnalysis does a pretty epic job of diving into exactly what the hell is happening here and the kind of big industry dynamics. Those are the two main ones that I really enjoy, and then occasionally some essays from key people.

I think the Dario essay recently was very thoughtful, well-written, and I try to stay away from the sort of hype-y stuff to, like, we're gonna die stuff. But otherwise, I think you've got to follow the researchers on Twitter, and they have good hot takes. That's like the town hall for machine learning, basically.

I think it'd be hard to do a job without it.

Thank you. Thank you. Good to have you.