When AI Improves Itself | Richard Socher (Recursive)

Great to be back. Thanks for having me.

Yeah, it's a somewhat surprising fact. And you may argue, clearly not. We're making so much little progress on so many different things. But when you think about how much progress have we made on antibiotics, bacterial infections went from a death sentence and the plague to a nuisance. We now have antibiotics for almost all the different bacteria, and we truly solved that. And we have clearly not solved viruses or cancer the same way we solved bacterial infections. When you think about foundational novel things like E=mc² and general relativity, we clearly have not made progress on many theories in physics either when it comes to such foundational things that then literally led to nuclear energy.

And fusion and fission and other kinds of research that could be conceptually done. And so a lot of the fields have kind of gone through from we understood some foundational pieces to we can now do a lot of engineering, but they've also split up into thousands of different subfields. It is almost impossible nowadays to be this sort of generalist genius that can dabble in all of these different fields because each field takes years and years and years to get really deep into. And so what we found is that as there are more and more subfields and niches, it's actually hard to have enough people in each of these subfields.

And Stanisław Lem and others have talked about that and predicted that that will be a big part of why we're slowing down.

That's right. Yeah, it's so hard to even understand all the lingo. And I've sort of gone through this myself, first when I started studying linguistics and then computer science. But now that I'm sort of trying to study, and have studied now over the last few years sort of on the side, biology, it's like, man, every time you have a conversation with biologists, like 10 sentences in, they're just telling you so many abbreviations and terms that you're not familiar with that most people, after a while, just space out.

And so it's really hard to describe and explain very complex, deep fields to someone who hasn't been in them.

Yeah, 100%. People often—the way careers work is, you want to be kind of novel, but if you're too novel, if you're too far out there, then your papers will get rejected. And that certainly happened to me a lot in the early days, like 2010, of neural networks for natural language processing, where the majority of my first couple of papers got rejected from NLP conferences when they got accepted in some small sub-niches and subgroups. I still remember the first sort of deep learning workshop at NIPS back in the day, now NeurIPS, that was basically like 30, 40 people, all the now super famous folks.

But it's just like a couple of us renegades who thought that this would clearly be the right way of going about it. And we came from different directions. Like, feature engineering seemed like not the right path to doing things. Feature learning was a big part of what got us started. Some were neuroscience-inspired. And it was, again, that sort of combination of different fields that was at that intersection where it's interesting, but also often hard to publish well in the beginning.

That's exactly right. Even, like, a good example again in biology is that you study not all of biology anymore. You study either biology at the cell level, or the tissue sort of medical level, or the biochemistry level, or the protein level. And if you ask a PhD in biology about, like, a deep question in one of the other layers, they often don't know either.

The high-level idea is that science has gotten really good at understanding smaller and smaller pieces, but to bring them back up and bring them together, we actually have to use AI. And AI is kind of what calculus did for physics. AI will do for biology in the sense that it'll help us weave back together lots of very complex pieces that build these complex systems that then have certain properties. So concretely, for instance, your microbiome or the brain, they have so many different pieces.

And we actually understand many of the individual neurons. So, like, hey, this neuron has these synapses, and this is how it fires, and so on. And this is the chemistry, which people still often ignore in a lot of neural models and so on. But then as we put more and more of them together, we stop understanding why the whole brain can now have a thought or do certain things. And I think AI is the perfect language, the perfect way of thinking about some of these complex systems, and has the ability to also exhibit sometimes patterns that are almost hard for us to then understand.

But of course, it's always easier to understand and study a neural network than it is to study the original biological system. And so I think when you put all these together, and I'll talk about sort of the details of the Eureka machine and its four columns and so on, and you look at the data, you look at sort of different fields, everything from physics, chemistry, biology, neuroscience, medicine, economics, astrophysics, and so on. You look at all of these levels and you see so many small improvements in all of them that can help you extrapolate that AI will usher in this new age.

I think it's quite counterintuitive, so I'm glad you asked, because if you ask a biologist if we'll have a model that can predict something as complex as aging or general different cancers and so on, they'll say, no, this is decades out. And in a similar fashion, 10, 20 years ago, natural language processing researchers would have told you that it is impossible to build one neural network that could answer any and all kinds of questions. In fact, you can find this online on OpenReview, my paper where I described prompt engineering, like one neural network that you can just prompt with any kind of questions, called DecaNLP.

That paper was wildly rejected by basically all the reviewers and the area chair and so on as just, like, completely useless, too crowded, made no sense. Not even humans have one system to answer all these different kinds of questions. And so it was something that's so obvious now that people are like, you can't even invent prompt engineering because it's so obvious. It was such a non-obvious thing to the field. And so we've seen that time and time again throughout different fields.

But now I think once you showed that we didn't have to actually truly understand and have perfect rules for every single aspect of translation or question answering, we will see similar things in, and have seen already in, for instance, the language of proteins. So you have just sequences of amino acids. No human has been sort of evolutionarily trained and has learned to speak the language of proteins. But just like the language of natural language, of English and so on, AI doesn't really care if it's English or a sequence of amino acids.

And so you can now generate completely new kinds of proteins, like you can generate new kinds of sentences that have never been in that combination in the training data. And I think when you now apply similar ideas to chemistry and even lower levels, such as molecules, you'll see that that idea that you have very complex systems that, with a lot of data, can then make very useful predictions that explore the overall space better and better than any human could have manually. That helps you then say, okay, this is much more.

This next-token prediction is such a beautiful yet simple idea that incorporates knowledge about almost any domain.

Yeah. So next-token prediction, like, why is it so powerful? Imagine you could try to have a model where you understand the world's geography and where every city is. And here we're in New York, so maybe I'll use a New York example.

Yeah, to Princeton or somewhere, or north to Boston. And so you basically, just by trying to predict the next token, will stumble upon sentences where someone somewhere wrote, like, "Oh, I was in New York and I was driving north to..." And now it might be Yale, but maybe more likely it's Boston, right? And the more likely it is, probably the bigger the city is. And so you now incorporate, by trying to predict the next word in that one sentence, "I was in New York driving north to," that next token being Boston.

Now you predicted something about geography and locations of cities. And so by virtue of doing that billions and billions of times, in fact trillions of times nowadays, basically as much data as you can, you actually incorporate knowledge about geography in that. And what we're seeing, in very similar fashions, a lot of people are familiar with protein folding. When proteins fold, some amino acids are actually closer to each other in 3D space than they would be in a sequence of these folded proteins.

And it turns out that you can analyze the neural network that was trained to do next-token prediction, and you see that indeed, the ones that are closer in physical space after being folded, the neural network, just by doing next-token prediction, also has those correlations. And so we basically know that large neural nets trained on very large specific domains will incorporate the knowledge of that domain just by trying to predict the next token, quote unquote, in a sequence. And token, for the non-technical folks, means it can mean an English word, can mean a subphrase of an English word, just like a sequence of characters that together make up a word.

But a token can also mean a protein. It can also mean a piece of a few pixels in an image or in a video or in a piece of sound. So everything can be kind of tokenized, basically discretized into a set of vocabulary tokens that are then able to be predicted.

So there are clearly some places where it feels weird, like it feels oversimplified. And you could clearly say all models are wrong, but some are useful. And that's also true. We know proteins aren't just sequences. They have a 3D structure, for example. But it's surprising how far this analogy is being able to be pushed. Like, even in chemistry, you have various loops within molecules, but somehow if there's just a standard way of describing molecules as a sequence, as a string, using that is very helpful for a variety of different aspects of chemistry too.

So yes, it's oversimplified. Yes, all models are wrong, but this particular set of models is quite useful for all the different sciences. And then, of course, you can go into arguing, but how truly novel can ideas be from some of these models? And the truth is that in almost all of science, we stand on the shoulders of giants. And just exploring the recombination, the clever combination of all the different ideas that are already out there, will lead to incredible progress. We have a lot of the foundational pieces figured out in small pieces, like the smaller sort of subsets, but to weave them back together will lead to incredible progress, especially in biology.

And then you can also explore how well these models can create novel ideas. And there we also have real examples of researchers having had an AI ideate, and then a few months later, people publishing a paper with essentially the same idea. Jeff Clune, one of our co-founders at Recursive, has tweeted about several such things happening where he had novel evolutionary algorithms create ideas that later have then been published by people. So clearly the novelty threshold has been met in several cases.

Now, will it derive, like, a completely novel way of looking at the universe, maybe be able to disprove or prove any of the many string theories that are out there and things like that? There's still some research that has to be done, but I'd argue that with existing technology and giving it more data and more compute, we can already cure a lot of diseases. We can already cure many of the aspects of aging. We can already build better fusion reactors and systems.

We can already create new materials. We can already help with economic questions of how much to tax or subsidize certain populations if you have a certain objective or reward that you're looking to achieve inside your economy. All of these things are already within our grasp.

I think we can basically predict where AI will certainly have superhuman capabilities. And those are all scenarios and all domains where we can either have a simulation and/or a verification tool. Any kind of domain that we can simulate will basically result in a world where the AI can essentially infinitely many times experiment inside that simulation. And assuming this simulation doesn't take years to run every time you want something useful from it, you then know that you can solve the problems in that domain.

And so all the games, I was never that surprised that AI will eventually, fairly soon, be better than us in games, especially the games that are perfectly visible. They don't have hidden variables. Like, you don't know other people's cards. Like in chess and Go, you see everything, and it's all on the board. Now, there were too many combinations on the board to just do brute-force kind of winning of those games. But if you have enough training data, the AI can learn the intuitions also behind it by playing many, many games.

And in this case in particular, it's even easier because the AI can play against itself. And that idea to play against yourself is also a big part of open-endedness and recursive self-improvement that we've applied. And I can talk about rainbow teaming and security and safety research, for example, with LLMs also. But just to go back, simulations: anything you can simulate, AI will solve. Now, what else can be simulated that's interesting and can be verified that's useful? Math. Math is going to change massively within the next few years.

Terence Tao and the most famous and most sort of frontier mathematicians are already fully aware of it. The whole field will change, just like the field of AI has changed. And a lot of sort of skills that used to be useful, where you manually feature engineer, and then you manually architecture engineer, and you manually do these things, are not that useful anymore. That will be true for a lot of mathematics also. And so I think that's a great sort of situation if you cared about solving as many things, proving as many theorems as possible in math.

It's not a great thing if you just love the sort of pursuit of math for intellectual sake and for fun. And we'll actually see that play out in many different ways. Chess is now more popular despite being dominated by AI, if you wanted to. I think most sports, intellectual and physical, in the future will probably benefit. I don't know if you saw the robot running really funny in the Chinese Olympics recently.

My hunch is humans will try to see if they can emulate that funky run to then run faster too, because AI in simulations, like robotic simulations, tried many different ways of running and found this weird new way that somehow humans, despite having run all our existence, haven't thought of yet. Right. And so there will be, just like the best chess players and best Go players are better now because they can compete against an almost perfect AI, I think even that will be true for athletes in the future.

So I think it will push the field forward. And what's the most powerful thing we can simulate and verify and build verifiers for is programming. Software is eating the world, famously said, I think, Marc Andreessen. And so AI is eating software. So now you can basically—and one example, a simple example is you can show the picture of a website and you say, make it program such that it looks exactly like that, right? And then you can create infinitely many examples like that and then have a perfectly capable AI building front ends for websites.

And the truth is you can create all kinds of verifiers like that. And so all of programming will change, and that will be a major impact on the entire digital economy, which is the knowledge economy and so on. And then the question is, where does it stop? Well, what things are hard to simulate right now? And that's where it becomes interesting to look at the natural sciences, where we cannot yet perfectly simulate a complex cell, let alone tissues or organs and full humans.

And we do need to collect a lot more data in various robotic forms. And those are essentially the four columns that I talk about in the Eureka Machine too. You start with human knowledge and LLMs, then the second pillar are all the measurements we can take, and more and more of those that we already have, and we should incorporate that into the model. The third thing is a simulation. And the fourth thing, fourth pillar, is essentially robotic process automation to collect even more data and verify whether the inventions really made sense.

And on top of those four, you have an agent swarm and a community of scientists.

So at Recursive, we are fairly sure that we have to start with AI for AI and then make it really good at doing research on creating better AI so that it has the equivalent of 50,000 PhDs in terms of knowledge and its own capabilities, and only then go after the physical natural sciences like physics, chemistry, and biology. Especially biology, I think, will be most interesting. I do believe that in the next two or three years, while we're focused on recursive self-improvement, we will also have more and more data collection.

Tahoe Therapeutics is a great example. Parallel Bio is another one. I think I mentioned both of them in the book that basically help create much, much more training data. And then, in the case of biology, for instance, you can do these perturbation studies. Like, you take a cell, you try to knock out one gene, and you see what happens when I knock out this one gene, or I add this one molecule to it and I see what happens. So if you do many, many perturbation studies, eventually maybe the AI will learn the underlying patterns behind it, just like it learned the underlying patterns of, like, oh, I'm in New York and I'm driving north, and then predicting Boston.

It might be like, oh, I add this molecule to this kind of cell, and then I get the output of— and then it's just like it can start to eventually generalize. But we're just nowhere near having enough training data for biology. And so we need organoids, we need eventually all these perturbation studies to come together so that we can then try to build a virtual cell. And then in that virtual cell, the AI can then go and experiment many times.

Yeah. So a lot of folks struggled with hallucinations in models for a long time, especially in the earlier versions of these models. The book, I started thinking and writing the first sort of ideas down, like, three years ago, and had to change a lot of chapters from someone should do to someone has done, and let me talk to them and talk about their startups and whatnot. But I do think hallucinations can also be very helpful for AI when you want it to explore novel kinds of proteins.

Yes, every AI can memorize things. Every computer can easily memorize things, right? But where it's interesting is, how well can you hallucinate? How reasonable, or just outside of the distribution in some interesting way, are your predictions, right? And we also know that we can, just like with humans, right, you give them a certain kind of molecule and their visual cortex goes off into a really different world. You can also increase the temperature, as we call it, a technical term, and the AI, the large language model, will then generate tokens that are more and more different from things it has seen before.

And so I think hallucinations are, in some cases, a feature and not a bug. Of course, when you ask a factual question online to a search engine or an LLM, then you want to have it be correct. And when we know that that's the kind of question you're asking, it's easy to prime the model and say, well, here are search results. You.com does, of course, basically take the facts from a real search engine, built for agents, plug them into the prompt, and then the AI will kind of summarize that.

And so I think initially people thought, oh, we need neuro-symbolic reasoning, blah, blah, to do all this. We just needed more examples of, don't hallucinate now, take real facts from a search engine and then mostly summarize those. And then those hallucination problems were, to a large degree, resolved. And then if you want to write a poem for your wife, you don't want it to just look and sound like the other poems that are out there. You want to create a new one.

You can also do that.

That's right. Yeah. I mean, Heisenberg and other physicists, and there's all kinds of interesting stories about absinthe in some cases, also just actual mental states that were eventually quite unhealthy, and just psychosis and so on, have in some cases pushed the field forward.

Yeah, I think when I started studying, the first time I studied biology was in high school. And I never, to be honest, loved it back in high school because it's just like, memorize these processes with all of these different pieces. You write them out, you get an A, and then, like, six months later, you mostly forgot about that process. And so that wasn't that interesting to me. But what's changed in the last few years is that biology is becoming a programmable science.

It's becoming an engineering science. And that's often the case, I think, in sort of the transition of different sciences. Once you've understood most of the basic pieces, you now want to learn how to put them together in novel ways such that they are useful for you. And there's, like, low-hanging fruit when a field transitions into that becoming sort of an engineering science. And I think biology is in that state right now, where we know, okay, this protein does this, but if we change that protein a little bit, maybe it can do something else.

And you can package. And sometimes you can connect different things that one piece, for instance, attaches to a cell, but then you can have different loads, like connectors, to it. So once it's attached to the cell, you can actually inject something into the cell, and now you can recombine these molecules. And so I think that engineering aspect of it, I think, is truly exciting. And the first sort of aha moment for us was, I think, in 2018, when we trained the largest language models for proteins.

It's called ProGen. Ali Madani is the first author of that paper. That was back in the day when I was the chief scientist at Salesforce still. And he's since started Profluent. They've now closed, like, multibillion-dollar contracts with Eli Lilly at Profluent, his company, because they've created new kinds of proteins that are, for instance, even better than CRISPR-Cas9 at gene editing and being even more specific and targeted for changing certain genes inside living people, potentially, and creating new kinds of therapies from that.

And so proteins, being such an important piece of all the building blocks of life and disease and health, making them programmable will unlock very, very obviously many, many exciting use cases. And I think you're starting to see this sort of in this recent trial that is making a lot of progress, where they basically created a different drug for every different patient in the trial. And this is a first for the FDA, too. And more will happen there. It's actually unfortunate how hard it has become in the US and certainly in Europe to run clinical trials.

And so a lot of folks are now moving to either China or Australia for their clinical trials. Interestingly enough, in China, it's cheaper, it's faster, but you also have to worry a little bit whether your IP gets sort of sucked into the ether and is gone. And in Australia, they had a clever move where they actually decentralized clinical trials, and every hospital can run its own clinical trials. So all of a sudden, you get competition instead of having one centralized sort of decider on which clinical trials to run and how to sort them and all of that.

And so, anyway, there's not enough people in Australia, so it would be great to get that kind of system happening in the U.S. too. But clinical trials will be more and more efficient over time. We'll collect more data, and then the AI will be able to automate more and more of that.

It's a good question and sort of touches upon what some people call the hard takeoff too, where some people think once we have RSI, and generally with AI, there will be this really hard takeoff, and then everything will just happen very quickly. And as bullish and excited as I am about AI, I'm not a believer in this crazy hard takeoff. I think, yes, things will accelerate, but there are certain things that will just require time because of physics and constraints in the real world, such as long-term trials that you want to know whether people have some issue, like three years after they stop taking the drug, and things like that.

And so there will be some delays. But the biggest difference is that the whole bio market—and somewhat contrarian take that we have at AIX Ventures too—a lot of folks think bio is just a terrible space to invest in because in the past, a lot of drug companies kind of spent eight, 10 years. They finally get—they have to be public because there's not enough late-stage bio investors. So they go public with one drug or maybe two drugs in late-stage trials, like stage 3, and then the stage 3 trial fails, and then the whole company is dead.

Now what we're seeing, the difference is we now have companies that instead of having one molecule or drug after eight years in late-stage trials, they actually, within six to 18 months, have multiple different drugs in late-stage Phase 2 trials already. And by the time they'll go public, it'll be with like eight-plus different drugs that are then also much more likely to succeed because we have better predictive models.

There's a lot to unpack there. Maybe at a very high level, I think if you care about the outputs of an industry or a company, then you love AI. If you get paid hourly, you probably hate AI. And so AI, in the positive instantiation of this future, is a huge force towards more entrepreneurial thinking. If you're an entrepreneur, generally you kind of love AI because it's making your things more efficient. You just get more done. You have an unlimited list of things to do if you're a startup founder or just running a company.

And to have an AI do many of those things for you just means you can do a lot more. But if you're basically being told you're training your replacement and all your data is being collected hourly, then you know at some point those hours will end and then the AI will just do the thing you just taught it how to do. And so it's understandable that people, if they have this very unentrepreneurial mindset of just getting paid by the hour and they don't own any equity in creating that IP, then they're understandably unhappy.

And then you can go one level deeper and think about, well, what is the impact on jobs? And my theory here, after thinking about this for quite some time, is largely dependent on the elasticity of the demand of the product when its prices go down. And so concretely, for instance, illustrators. Illustrators hate AI. The world needs a certain amount of illustrations. Because of AI, you cannot charge $200 anymore for one illustration. So now any little blog post has illustrations.

If your goal was just to see more illustrations in the world that are specific to a text, you love AI. Too bad, maybe you hate it, right? And so the problem was that the demand for illustrations didn't go 1,000x when the price went down by 1,000x. It didn't grow, because you just don't need that many illustrations in the world. Now, in coding, it was a very different world, actually. As coding got cheaper and cheaper, you had this famous Jevons paradox that everyone's talking about now.

I think I was the first, at least I didn't see it online for a while. It's like an interesting fact from history. And you actually, like, the thing got cheaper and cheaper, but we actually used more and more of it. And for coding, that will definitely be the case. And so we're seeing actually more demand for programmers now because they're so much more productive when they use AI. And anyone ultimately could have dozens of apps on their phone that are unique to that person, that are modified in some way and very special.

And there's so many other ideas that people didn't explore because it was like maybe the market wasn't that big. But now that you can just create an app really quickly, why not? And so I think that is another aspect of jobs. And so, go back to cancer. Yes, I do believe actually AI will play a big role in curing multiple cancers. We are seeing trials now where AI is being used to make a specific cocktail of drugs, create specific RNA sequences, and so on for the types of cancer.

And each cancer often is also not one homogeneous thing. It has different types of subcancers in it and so on. And you need to specialize treatments for each person and for the various different forms of the different cancers that you can have. And so all of that is much, much more feasible to be done with AI, and we're seeing it.

Exactly.

Exactly. So, yeah, it will take some time. And obviously, even if AI, let's say, had the perfect molecule and was like, okay, for this type of cancer, this is the molecule—an AI came up with it—you'd still have to run it through many clinical trials. It'll still take years to come out. So everything in biology just takes longer than it does in software.

I think really we can design bacteria that eat microplastics, and once there's no more plastic, they just die. I think that would be extremely helpful for the oceans and so on. Obviously, you have to be very, very careful that they don't somehow mutate into eating other things and so on. So when you mess with the environment at large scales, it's important that humans have done that many times, and sometimes it worked out pretty well. Many other cases, maybe not so much.

Forests are a good example. People deal with forests too much. They don't let small forest fires happen, and then they get even bigger because the small ones didn't clear out the underbrush and so on.

Everything is a complex system. And we need AI more and more to do some of that engineering better than we've done in the past. And so I'm excited for, at all the different levels, when you look at how to balance plasma in tokamaks for nuclear fusion, that's already a machine learning control problem. I think we'll have a better handle on that. So it's sort of at the lowest level of physics. Clearly, there are more and more materials, more efficient solar cells and solar panels that we can design with AI.

There's companies I've invested in that do that. Better batteries, better materials. So we don't need only lithium. We can try to build batteries with more abundant molecules that are easier to get and mine, have less pollution. We, especially again in biology, are seeing a lot of things. I think it sounds like science fiction, and I understand the famous saying of, like, if you want to know why something doesn't work, ask the experts. I think that was true in natural language processing and neural nets, and I think it is currently also true for longevity and cancer and other kinds of research for neural nets applied to biology and medicine.

I do think we'll see—we'll make more progress than the most skeptical people think, but we also won't have a hard takeoff again because things do require careful experimentation in medicine. And I'm personally excited for all of these things. I think if you're mostly interested in making humanity more productive and more efficient and create more outputs and grow, then you're going to love AI. But also, in some ways, it becomes a philosophical question. And I think we already observe many subcivilizations or subgroups of people.

I mean, cultures that have essentially off-ramped from progress. Like, if you're living on some beautiful island in Greece, you don't really think about AI. You don't have to think about AI, and you just enjoy life. You go fishing, and sometimes there's a storm and things are bad, but most of the time, the weather is good, the fish are abundant, and you just kind of live your life. And so I think there will be different groups of people who will want to off-ramp from civilization progress, right?

There's already people who prefer to live way deep in the countryside and never go into the big city and so on. And I think we'll have more of that. And in some ways, I personally love progress. I think scientific progress especially is what helped humans solve most of the hard problems that were in our biosphere. David Deutsch has a whole section in his book, The Beginning of Infinity, which I highly recommend people read too, where he talks about how there are all these different material, real problems, and we came up with solutions thanks to science and better explanations and better research.

And I'm personally all for that. But some people will not want to participate in that world anymore. And I think AI is such an accelerant that it makes that question even more pertinent for people.

Yeah.

Yeah. I mean, like, a sort of example that I'd love to visit, actually, is Bhutan. Bhutan decided we will not measure our gross domestic product based on money, but based on happiness. And happiness mostly for people who want to keep it simple and have a simple life, not want to build startups and so on. I'm pretty sure those folks aren't quite as happy in Bhutan, but overall, Bhutan is just very green and it cares about the environment and cares about a specific subset of religions.

And people are more often content in keeping things the way they are rather than trying to progress in various different ways.

Yeah, it's really interesting because clearly people use the technology. It's like if no one used ChatGPT or Claude Code, there wouldn't be a problem. People clearly like it. It's just that the people who get a lot of use out of it are not quite as vocal. And there are negative things. There's also some amount of moral panic about chatbot friends, in similar fashion to how novels used to be a really bad thing. Like, there's all kinds of stories of older people saying, oh, these novels are ruining the youth.

They're now living in these dream worlds and are distracting themselves from the real world. And, like, Die Leiden des jungen Werthers is a very famous book in Germany, actually led to some suicides, which is really sad. And now it's like the book every German kid has to read in high school, and it's just like a high form of literature in Germany. And then comic books were really bad and computer games were really bad. And there are various sort of levels of that.

And currently the chatbots are really bad, but there are also clearly a lot of people who get a ton of value out of these chatbots. And now all of a sudden you make access to medical advice cheaper, legal advice cheaper, and sometimes also emotional advice cheaper. But you don't hear many people, or the many people that clearly exist who are hundreds of millions of users of these technologies, talk about how much this helped them not commit suicide or something, or not be very sad and dysfunctional and so on.

So I do think you're right. In some ways, not just AI, but I feel like the future as a whole needs better marketing.

Yeah, economics is a really interesting field that unfortunately doesn't have obvious benchmarks the way computer science and many other sciences have, where you just say, if you do better on this benchmark, you clearly have the better ideas, the better algorithms, and we should all learn and study those. When we submitted these papers on two-level reinforcement learning systems to Nature and Science, they just desk-rejected them. In one case, some random ethicist who had no idea about AI was just like, desk reject.

I'm not even going to read the full paper because AI for economics with reinforcement learning is just a weird thing. And so it was just, like, gone. And so because of that, economics often becomes just a political field. And if you're in one economics department that has a certain political slant and direction they want to see the world move into, you just have to write papers that make sense for that political ideology. And so that unfortunately makes it very hard to do more objective research.

And so we tried to create this very simple simulation where you have a bunch of agents. This is from 2018. The agents were much simpler back then. They just had a certain utility function. They had certain hours in the day that they would be willing to work. They were sampled from certain priors that you may make assumptions about. Not everyone wants to work 14-hour days, but some people basically make all these assumptions. And then you let these agents collect resources, build houses.

They can block other agents from those resources to try to build monopolies and become even wealthier. And then you had a sort of meta-agent that looked at all of these other agents and basically chose how to tax and subsidize different groups of agents. And in that fairly simple simulation, you could essentially give it an overall reward. Like in our case, we said, let's maybe start with equality times productivity. You want the economy to grow, but you also don't want one agent to have access to everything and everyone else to be really poor.

And so obviously you don't want just equality and you don't just want productivity. So you have a combination of these two multiplicatively. Now, if you agree that that's a good reward, you could have politicians say, well, I'm going to do this and that to help, for instance, the middle class, or to do this and that. But if we had a much larger-scale simulation, you could then run their one proposal through billions and billions of years of simulations and of taxation and subsidization to say, well, will that proposal really result in that outcome that you say you have, the goal that you have?

Or maybe, probably, if you simulate billions and billions of years of different tax years, maybe there are better ways. And what we found is that the agents will try to avoid taxes by dumping a bunch of stuff before or making a bunch of gains just after the tax year, and so on. And the funny thing is that paper, basically the baselines that the field uses, one very famous formula is called the Saez formula in economics. And basically it's beautiful math, and it shows that provably it's the optimal taxation scheme, but it's the optimal taxation scheme in a one-step economy where you make one economic decision and then no other decision again.

And so we showed that this very complex RL system basically recovers that thing and does come up with the same solution. But now you can actually deal with the fact that economics is a temporal sequence of many different decisions, and you can learn and adapt, and there are counteradaptations from the agents to certain taxes and subsidy schemes. They're trying to play things, and then you can still simulate it. And so my hope is eventually that that paper will have kind of a GPT-3 moment where someone actually scales it up, builds a really realistic simulation, and then we could have AI give us feedback.

Obviously, we don't want to let the AI make those decisions without any human oversight, but at least have some economic policy suggestions on how to most objectively try to achieve the goals we want to set. And of course, humans then have to really formalize kind of what is the goal of our society. And in many ways, these are very deep questions that philosophy and political philosophy have asked many times. Socialism, capitalism, maybe social market economies where there's some regulation in healthcare, but maybe not in other areas, and you want competition.

You can actually define once what your real goals are. So I think hopefully over the years, this kind of system will help us run economics much better and make it a much more objective science.

All models are wrong. Some are useful. I think we can make those models more and more useful, and they'll be less and less wrong. I think we've seen surprising results where you can prompt an LM and say, you are now a 43-year-old from this region. Give them all kinds of sort of prompts on what they're supposed to act like. And then after having trained on tens of trillions of tokens on the internet, you can say similar things to what people might say from that setting.

And so I do think these models will get better and better. The fidelity of the simulations will get higher. And once they cross a certain threshold, then the recommendations from such a simulation with an AI could become more useful. I don't think this is very feasible in the United States for a very long time. It's just so much identity politics and special interest groups, and how super PACs and so on get funded, that it's very, very unlikely to be used. My hunch is Singapore or China will probably be more likely to try to use those ideas, say, hey, we all agree, or we at least make it very clear that this is our objective function.

And then we're going to really try our best to set the various taxes and subsidies and so on in a way that really achieves that objective function.

Yes, I think AI often is only as good as the people, the data, the systems, the infrastructure, the rewards that we give it. And we have to be very careful about how we design and filter all of those things. I think we have more and more control over it, but it is still surprising how poorly engineered some of the environments are and some of the sandboxes are that frontier labs use. So let's get into the machine itself.

So, yeah, the four pillars, I briefly alluded to them earlier, where the first one is just large language models, essentially to try to ingest the world's knowledge into the Eureka machine. And I think the interesting bit here, actually, is that in some ways, there's this weird cycle that happened that I don't talk about in the book as much, but I've sort of lived through this now, which is the few large closed labs, Anthropic and OpenAI, took almost everything they could from the open internet, trained a model, but then the Chinese open-source companies basically siphoned a lot of that knowledge out of those closed-source models by distilling it, but then they open-sourced the model back into the open domain.

So now the knowledge is back in the open internet where it started. And so I think it's very clear that that first pillar of just, like, having access to all the world's information, being able to reason through all these different concepts and the crazy large combinatorial space. So pillar two is a model of reality itself.

If you think about how limited human perception and the current set of human knowledge is, and how we could actually expand that, you have to look at scientific measurements, right? We cannot observe gravitational waves. We cannot observe dark matter. We observe sort of gamma rays, but we can build tools and scientific machines that measure these things for us. And so that is a clear second pillar that is different to human knowledge, that in some cases hasn't been fully described in human language, and in some cases might be very complicated to describe in human language.

We can already say, "Oh, a neural network predicted this word because of these 5 million parameters." But it's like, okay, well, you just list them all out, but you don't gain an intuition because the system is so complex. Similar to how no one can really say, "Why did you move this muscle fiber in your pinky when you try to move the steering wheel?" No one has access to that in their brain. And even if they did, it would just be like, because of this very complex system.

And so that is basically the ability of an AI to take in all of these measurements and try to start actually digesting it and taking real knowledge, extracting it from scientific measurements.

So one, you'd have to really collaborate with a lot of different sciences to put together this kind of foundational model of physics, chemistry, biology, and larger and larger systems. And then have many universities and labs work together to bring all of that into one model. So I think we've had sort of the projection of the humanities knowledge onto the internet, but there are just lots of things that just don't make sense to put up on the internet. And so those things are still hidden.

Many companies are now working on, quote unquote, foundational models. Some of them now rebranded them as world models, using similar technology, though, where they basically try to ingest as much information about one domain, and then they build a first example of a virtual cell that is particularly good at estimating particular gene variants or something, but not lots of other aspects of a virtual cell. A virtual cell is a good example of a goalpost where many different teams would have to come together and bring all of that data into one unified model.

No one in its full glory. That doesn't exist yet.

But there are obviously different levels of abstraction. And sometimes, for most aspects, you actually get away with not having to simulate all the quantum details of a very complex subatomic particle. You can just say, all right, these are the molecules. And then you know how, in chemistry, those molecules will work together based on the valence shells, blah, blah, blah. And then in biology, you sometimes can just abstract from, oh, I don't even care about that molecule. I'll just say this is overall this protein, and that protein connects to a cell at that level.

So as you try to build it all together, then you have to have computational efficiencies and abstractions that humanity has been good at building, and computer science is particularly good as a field at building. No one has to program in zeros and ones anymore. They can now program in English, and a lot of the abstractions can be ignored. I think similarly, in these physical simulations, we can ignore more and more levels down. But sometimes there is quantum biology, and there are maybe some effects that we didn't realize and we oversimplified.

And those might come out from a model where you're, like, from one large simulation in which the AI can then try to experiment.

In many small bits and pieces, right? The simplest example is a simulation of Go or chess. And that's like, okay, we have it. It's easy. An interesting new one that many people are working towards now is a virtual cell. If we had that, I mean, a virtual cell is so complex, like a real human cell is so complex. We're very far away from that. But I can see how, with enough people coming together, with enough funding, we can eventually get to a fairly useful model of a virtual cell.

That's right. At some point, especially in biology, but in all other fields, you have to engineer a system. You have to really put it together to see if you missed anything in your simulation, any confounding variables, and so on. You have to really run experiments in the real world. And obviously, in the smaller case of physics, chemistry, and biology, you can do that in a lab. At some point, you have to build real machines and really get out there, build satellites and whatnot, and take measurements of the universe at all kinds of scales.

And so there, I think it makes sense for us to put more and more resources behind that as AI has gotten really good in the first three pillars. So there's some sequence to it.

I love that there are first efforts in this. Periodic Labs is a great example of that. I love that we're starting to think about this personally. From an investing perspective, I feel like it's a little bit early, but in two to three years, I think it'll be right on time. We'll have figured out a lot of the software. We'll be really good at the LLMs, the scientific sort of data, and connecting that also potentially to LLMs, building even more high-fidelity simulations.

And then we can ask the AI to come up with really good, expensive experiments that can take sometimes hours or days or weeks to really run through. We'll have better organoids or tiny cell systems where you can basically use human-derived stem cells. Parallel Bio does this, for instance, for human lymph nodes, like immune cells. And then you can experiment with those cells more quickly. And that is done with robotics already. So a few real examples of that exist. I think there's a chemputer too, in chemistry, that can put together a small set of molecules. There are first examples of this with Parallel Bio, with organoids and doing clinical trials with that.

By the way, that alone, that company alone has already gotten FDA approval to skip certain animal trials. So you save many, many millions over the next few years and the lives of animals that are just bred to then be tested upon and then dissected and evaluated. And this, if you love animals, you can also love AI because AI is now already—not just eventually, but through this one company, Parallel Bio—already saving animal lives that are, again, bred for being tested upon.

And so I think there's tons of really amazing work that is very targeted. To build these out in more and more generality is kind of what is required to then allow the AI and the agent swarms to sit on all of these four pillars more efficiently. Yeah.

The agents will ideally work on as much of the scientific process as possible, similar to how scientific communities do it. And in many cases, the evolution of science and culture and even biology has aspects of open-endedness, which is very inspiring for us at Recursive also, and are actually basically exploring interestingly different ideas highly in parallel that then can be recombined. And so these open-ended processes have led in biology to everything from our fingers, eyes, and brains. In technology, there are lots of examples where—and Jeff Clune, one of our co-founders at Recursive, talks about this a lot—how you can't get a microwave if you just say, make this pot faster in heating up my food, right?

And you just add all kinds of pressure and so on. But you had to work on radar technology and realize some chocolate bar in your pocket was melting as you worked on radar, to then eventually get to a microwave to warm up your food faster. And so there are these different paths and recombinations of different research ideas that can be coming together, and we can model that better and better with agent swarms.

Indeed, compute is the biggest constraint. I think in the future, more and more humanity—and already we see this inside different companies—will have to decide what problem is worth solving, how much compute do we give to solving that problem? And then there will be new kinds of scaling laws where we give enough compute to really solve different kinds of problems. And yes, I think the majority of the public internet has been digested by a lot of these labs, but there's always new data.

There's always new things that happen in the news. We work with a lot of AI labs and other labs also to just give them constantly new search results when they ask about something that just happened last week and wasn't yet part of any training dataset.

Yeah. So Recursive started with the goal of building recursive, self-improving superintelligence to automate knowledge discovery and scientific discovery. And actually, the eight co-founders came together, and we all, in one form or another, came to the same realization, but from very different directions. Tim Rocktäschel and Geoff Clune, for instance, came very much from this open-endedness research direction of evolutionary algorithms and so on. I came very much from this idea of, well, we automated feature engineering to have word vectors, we have neural nets, then we automated architecture engineering by just having one unified architecture.

What's the next level of automation? It's like the actual ideation and implementation validation of general ideas in all of AI research. And that's clearly and obviously the next level to unlock a new set of capabilities. And when you think about the automation of science, and then you apply the automation of AI research to AI itself, before you know it, you're in this recursive self-improvement loop. And we believe that will be a great unlock to then apply that kind of intelligence to all kinds of other scientific discoveries.

Yeah, we raised in the end like $670-ish. And yeah, that will probably be one of the smallest compute deals that will happen in our future.

There will be a couple of interesting things coming up. I can guarantee you they will happen this year. We are in, and I struggle with this sometimes, the NeoLab category. I don't love it because I think a lot of them will not succeed. We are a real company, not an academic lab. We are building real products. We're talking to real customers, and we're very excited to take this technology and make it useful for real companies. I can't share the details yet of what we're going to release, but I think it'll be exciting.

And we already know from some first conversations that it is exciting.

There will be different—there's a sequence to it, and some things will be general, but then obviously at some point it'll be more and more specific. I think we did publish a blog post that gives you a little bit of a glimpse of things we're thinking about that are essentially milestones towards full recursive self-improvement that show that, for instance, when a lot of people use AI or do some auto-research on small models, our system, the sort of first instantiation of this Eureka machine—machine in a very narrow domain—can already outperform months and sometimes years of human endeavor on particular problems.

We also showed that they can build new CUDA kernels, which is very useful for faster inference, which is very useful for all large hyperscalers and people who provide tokens and run models. And we're very excited to keep pushing those. And we've heard very positive feedback from folks who are using these kernels now. And at NVIDIA, folks had created these benchmarks. The SOLiExec Bench is a particular example there. So yeah, those are all just simple examples of artifacts that this Eureka machine can produce when it comes to AI research on the path to a full RSI.

I mean, they're just ideas of, like, we want to explore this particular idea. And that particular idea is one of the many useful artifacts our Eureka machine could also produce, but it's not really a product. If you just want to try to think about how humans interact with AI in the future, that's not quite anything very concrete. And so you have to be very careful about founding teams and so on that have not just done amazing research but also shipped real products.

Yeah, that one will take us more than the time we have left. I feel like it's almost like a new book. I had to wrap it up at that point, the book. And so, one, I'm surprised no one has really defined intelligence in all of its complexity really well, neither in terms of the very foundational building blocks, which I currently think are prediction, action, and goals, and a combination of those three. Those are sort of the three principal components. Just like energy has one unit, we don't yet know what is the unit of intelligence, something I'm thinking about a lot right now.

We don't yet have a proper definition, sort of physics-inspired. In physics, we have kinetic and potential energy. But then it also makes sense to study chemical energy and mechanical energy and electrical energy in different forms. And some are still pure science fields, and others are very much engineering fields. And so I think a similar thing has happened in AI, where we have visual intelligence, language intelligence, physical intelligence, and robotics. And I define these 10 different spaces of intelligence, and each space basically has many different dimensions.

And I'll just give you this one example on visual intelligence, right? Humans can only see in a specific part of the electromagnetic frequency spectrum. But you can go much beyond humans when you think about what are the bounds of visual intelligence. How far could an AI or any kind of intelligent life form or entity in the universe push visual intelligence? And then you get into very interesting, sort of often physics-inspired, thoughts and loops.

For example, you can see everything from gamma rays to gravitational waves. So, very different, like the whole spectrum of electromagnetic frequencies. You can try to have not just two eyes, but you can have millions and billions of different sensors all throughout. But then how far could they go? Well, at some point you have communication bounds of the speed of light, and each sensor has sort of a speed-of-light cone around what it can see. And you quickly get into these thoughts around bounds.

And what you then realize is that, boy, are we far away from the true upper bounds of any of the spaces of intelligence. And there's still so much further that AI can go in research. And so when people think, oh, this set of algorithms or the field of AI is sort of like, the bubble is going to burst—maybe like energy, right? The cost, the unit cost of intelligence, may fluctuate depending on a bunch of factors, but we can still go so much further as a field and as a civilization in pushing that field forward.

That's right.

On Twitter, X, RichardSocher.com.

That's right. Com.

Thanks for having me. And wonderful questions. Great chatting with you, always.

Bye.