Perplexity AI: The AI-Powered Answer Engine with CEO Aravind Srinivas

Thank you. Thank you for having me.

Thank you. Yeah, we love giving information to people.

That's right, yeah, six years ago, approximately around this time, in 2017, I came here to do my PhD in AI. Back then, AI could not even pull this much crowd for an academic conference. But now it's so hot. None of us knew that at that time. So it was purely curiosity-driven. And also I really wanted to come to the US.

That was Berkeley. Yeah. Berkeley was like the number one school in AI at that time. I believe maybe it's around competitive with Stanford now, but at that time it was undisputed. So around 40 people got admitted into Berkeley in AI out of like 2,000 applications or something. I got lucky to get in. And I think about 38 of them ended up coming to Berkeley. Two of them went to MIT or Stanford.

So it was that hot in terms of academic interest at that time. I did good research that got me an internship at OpenAI the next summer. And OpenAI was like 30, 40 people, nonprofit. I think my first week or two, we couldn't even get a work laptop. It was like that much of a—

Yeah, Elon Musk was leaving at the time. So it was not the kind of OpenAI today, which is a formidable force in AI. So I got to learn a lot during my PhD, and that got me more and more excited about AI. And the next year I got an internship at DeepMind, which was actually the more prominent lab at that time. And I would read these books in their library, like In the Plex and How Google Works, and was very inspired by Larry Page.

This is not said in hindsight or to make up a story because we are working on a search engine. It's really true. And that's why I was very inspired to start a company too.

Yeah, I went back because I couldn't start any company after my PhD, either because the idea was too grand that nobody would want to come work with me, or I had visa problems to start it all by myself. So I felt the best thing to do was find a job and continue to explore and learn more. I saw companies that started making more money than even OpenAI at the time. And that was amazing. Okay, people are building products.

This is real. And GitHub Copilot, when they moved away from the waitlist to the paid version, they just had hundreds of thousands of people paying from the first day. And so that all made it clear that stuff that we were all thinking was just new ideas and research was more about strong execution, building teams, and shipping products. So I really wanted to be part of that too and reached out to two prominent investors in Silicon Valley, Elad Gil and Nat Friedman, and they both were willing to invest.

And people were like, these two guys are backing you, you should just do it. Even if it fails, you'll learn a lot. It's like getting an MBA and getting paid to do it rather than paying Stanford or Harvard. So that's why it's a good use of VC money. Exactly, exactly. Yes.

What was it like working at OpenAI in like '21 and '22? It was cool. I think at that time GPT-3 was already there. GPT-3.5 was being developed, and GPT-4 was not even there. The biggest hits at the time were GitHub Copilot and DALL-E 2. Both of them were really cool, and it was not clear whether, for a successful product, you needed the largest compute put on it.

That was not clear at that time because Jasper and GitHub Copilot were all tiny models making a lot of money. Similarly, Stable Diffusion and Midjourney were all pretty competitive with DALL-E 2 and actually probably making more money than DALL-E 2. So that part was unclear, whether it was 3.5 or 4. But obviously I was wrong. ChatGPT and GPT-4 are the reason why OpenAI is so formidable today. Nobody predicted that, and I didn't predict it either.

Yeah, so research scientists, I was more in the open-ended research team that tinkers and tries new things. So I was exploring new architectures for diffusion, like audio and text models together. There is no direction. You kind of are given the freedom to explore with limited compute. So you kind of have to be patient and try to strike gold somewhere. That's sort of the idea. And some of the hits through that mode of research for OpenAI have been, I would say, DALL-E is a big project that came out of that kind of research.

So I was also pursuing something of that nature for multiple modalities together. But it was individual contributor. I would work alone and try to get things done.

Yeah, yeah, there is competition internally for GPUs. And people at the top decide based on intermediate results you show and importance of other projects in the company and deadlines and stuff like that. So that was always difficult. And I think being a great individual contributor in research requires a lot of patience, and being a great startup founder requires a lot of impatience. So obviously that's why I ended up becoming a founder, because I was not patient, especially when you have to keep competing for GPUs and try to work individually and get to a point where you have a great demo, right?

So these are like two completely different modes of work. You can excel at anything, but you gotta pick what you want to excel in.

What's the high level of why I started the company?

Yeah, okay. So we kind of started with the thesis that LLMs are going to keep getting better and are amazing, and it's the time to build products around them, and it'll create a lot of value. But we had no idea what to build. So, obviously, I went by ideas given to me by my investors. So we started with prototyping text-to-SQL, just very, very—even now, no one actually has made any big progress there, right?

It's one of those things that is an amazing investment idea, but somehow nobody's capturing any market there.

Yeah, so our lessons from that is text-to-SQL is only a product-market fit when you have so much data and when you're such a large company that people actually need to keep munging data to figure out what decisions to make. And the people at the top need help here instead of bothering the data engineers. So it's kind of like a technology that you have to sell to a bigger company, like a more mature company. And those mature companies have most of their data in Snowflake or Databricks, and they're not just going to give you the data.

Like, you go to a Series B or C, or even more advanced companies, they're just not going to give you access to their internal data that easily. Neither are they going to move fast for you. And this is basically the lack of alignment between a startup trying to build and sell that technology and the company that's actually buying the technology. So that's where the fundamental misalignment is. And smaller startups that will actually move fast and work with you on these things, they just don't have enough data.

In fact, I would talk to some people who were doing sales at some companies, and they'll be like, "I just have a spreadsheet. Why would I use Airtable or Salesforce or HubSpot? I have only like 20 customers anyway, so I don't need all of these things. I can just keep track of it in my head." So that's the problem, and the bigger companies like Databricks or Snowflake might build these tools and offer it on their SQL editor or some kind of assistant, and that could probably solve the problem.

It's not a great problem for a startup to work on.

Yeah, SQL sucks. I mean, but it works, and it's scalable, and nobody likes writing it, but it just works. So you have to keep writing it. But what happened was we approached text-to-SQL like how we would approach a search problem instead of a developer problem. This basically only happens if you deep down care about something, that it just comes out in everything you do. And for me, search was always a passion.

My CTO, my co-founder's first job was at Microsoft Bing. And my other co-founder, he had basically built all the ranking systems at Quora. So we were all just deeply passionate about search. So even though we were working on text-to-SQL because we thought the VCs knew this was the right problem to work on, we would approach it as if it was a search problem. So we would just scrape a lot of data from the web that could be organized as relational databases in the form of tables, and then we would power search on top of them with text-to-SQL running under the hood.

And then the reason to do that was nobody would give us their internal data, and we had to build something anyway because, as a company, you've got to do something every week. And then if we scraped a lot of tables from the web, people would be convinced that we could handle a large database. So the first database that we really scraped hard was Twitter. This is before Elon was the CEO, so you could actually scrape data for free, which we did.

And so we basically built this thing that was an amazing way to do graph search on Twitter. Like, you could ask anything, like, who does Matt Turck follow that I could potentially follow, that I'm not following yet? So if I think he has great taste and he has some people that he's following that I should want to follow, what are the tweets that Musk has liked recently in the last 10 days?

These are questions that generally people are curious about but currently cannot do on Twitter easily. Those are the kinds of things we were working on. And this is actually what convinced a lot of great investors to invest in us, because when they used this tool, they found it pretty useful for themselves. And we were finding it useful for ourselves too, because as a founder, I would often want intros, and I would often be like, "Who can I DM directly if they're following me?"

Or if I know someone they follow, maybe they can DM them. That's how we started building this. And we realized that, okay, there's this version that's running on structured data, which seems like a great search engine. What about unstructured data, everything on the web? Like, what if I want to know a summary of one person's tweets? And then we connected it to a regular search index and just pulled up all the activity and gave a very neat three- or four-sentence summary.

With sources. And that with-sources part is because my two co-founders and I, all three co-founders, have PhDs. And PhDs, citations are like your currency. You're taught to just maximize it, just like VCs are taught to maximize their assets under management. Academics are taught to maximize their citations. So we were obsessed about citations. So we were like, okay, it'd be great to give answers as if an academic would write the answer, with inline citations.

And then we built that tool too. And so we were kind of a confused company. We had like two different products, like a SQL search thing and a regular search with answers, not knowing what to do. And then one day, ChatGPT launched. We had all these things internally. We were using it. Our friends were using it. And then I talked to my investor, Nat Friedman, and said, "Hey, I don't know what to do. The search thing is cool. I actually want to launch it, but everyone's going to think I'm taking on Google, and who's the startup that even deserves to take on Google?"

Like, you better not. Everyone's going to think about us like a clown. And then he said this one thing that was pretty hard-hitting. He said, "You're irrelevant anyway. Like, today you're irrelevant." And yeah, yeah, it's true. I mean, most startups are. And even if you launch this and it's a failure, you're going to be at the same level. But if it's a success, it's going to change your future. And that asymmetry, those kinds of asymmetric moments, are what kind of change your trajectory a lot.

So I got the courage, and all our investors were encouraging. None of them were like, "You're not supposed to do this." All Nat said was, "Get 100 users. Let's start with that." And we launched it one week after ChatGPT came out, and we got a lot more than 100 users, and it kept growing through the vacation. So we got the current— We also launched a Twitter search too. We wanted to get data on both.

And all we realized was that the Twitter search part was a pretty small audience. Like, there were very few—it was a niche, much smaller market. While the generic search was a much larger market. So we wanted to continue running both because we didn't have to actually maintain much on the other end. Obviously, Elon becoming CEO changed a lot, because we cannot scrape Twitter continuously anymore. So we obviously killed that version, and this regular search answers was growing a lot, and we continued to focus more on it and decided to commit to working on that.

So that's where you are.

Yeah, yeah. We have a lot of millions of monthly active users and millions of daily queries. So it's growing a lot, and it almost feels like a responsibility to keep improving it. I mean, Mark Andreessen and Lex Fridman had this podcast, and Lex asks him there, "Do you really think people are just going to keep consuming 10 blue links?" And he's like, "No, but this still exists." He's like, "Horses still exist today."

Like, some people still use horses to go from one place to another. So he gives that analogy, and our belief is that maybe today it's still niche. It's still niche enough that people view it as this research tool if you have deeper, more complex questions. But five or 10 years from now, you're not going to consume 10 blue links. It's going to be a chatbot or a system that you just ask questions, like how you would ask another person.

And in that era, I don't think Google is going to be the one single monopoly because they're already far behind on this interface. Yep.

Okay, sounds good. So basically, if you go to Perplexity and ask a question, you get answers and you also get the corresponding citations. So the content is only being pulled up from the cited links and nothing else. So unlike ChatGPT, it's not making up stuff. It's only taking stuff that already exists on the web, but giving it to you in a manner that you can ingest and don't have to open all those links yourself and read. It's almost like somebody did that job for you and gave you a concise summary.

And you can still verify in case there's some hallucination or incorrectness. You can still open the link and read yourself too. So this is pretty useful already because the links that get cited have almost been filtered for quality. And so it gives you already a much better experience, even if you just care about the links, and saves you a lot of time in reading them too. Now we have a version called Copilot, which is like a toggle on the search bar.

In the future, we don't think there'll be a toggle. It's just going to be one single experience that will resemble Copilot. The idea of Copilot is it's almost like a browsing companion. It's not just going to do one search query for you, whatever you asked. It's going to break down what you asked into more pieces, especially if what you asked was pretty ambiguous. Our philosophy for this thing was that right now all these products, Bard, ChatGPT, Perplexity, are all requiring people to be very precise prompt engineers.

Like, explain like I'm five, tell me this, or you've got to write these detailed prompts. Our philosophy is that if you want to take AI and make it widely accessible and regularly used by every person in the world, you shouldn't expect people to be great prompt engineers. You should work on the user's behalf and try to understand their underlying intent and keep expanding the prompt in a back-and-forth interaction. So that's what Copilot does.

You come and ask, like, "Help me buy headphones," and the AI is going to come back and ask you, "What headphones do you actually want? Do you want noise-canceling? Do you want over-the-ear? Do you want wireless?" And then you're going to click buttons. So this UI is dynamically generated. And then it'll ask you, "What's your budget?" And you pick your budget. And then it's going to ask you, "Which brands do you like?"

And you pick one or two. And then you just get the final answer that's been concisely catered to what you selected in these interactive questions. And these questions are going to be different for different kinds of queries you ask. And our belief is that that's the future of how you would discover information. Anything you have in your mind, you're just high-level curious, but you don't have to be so articulate and precise, and the AI will work with you together and get to where you want to be.

So that's our vision for Copilot. And in our Pro plan, we offer you unlimited uses of Copilot. As for Perplexity Labs and the API, everyone's been talking about us as a wrapper around GPT-3.5 and GPT-4 models from OpenAI, because that's the right thing to do as a startup with very little funding. Long term, you do have to try to ship some of your own models. You kind of have to be an orchestra of many models, like some closed proprietary models that others have, and some models that you are custom building on top of open source.

That way, you can drive more efficiency into your product, scale your usage, and don't run out of funding. And then when you actually make money, you can be profitable. So the first step towards that is making all the open-source models run efficiently on your own GPUs. So that's what we worked on. We have the best inference infrastructure for all these models, and we made them available in terms of a nice chat UI frontend called Perplexity Labs.

So you can use that as a way to chat with Llama or Mistral models. Think about it as ChatGPT that's running on open-source models. That's what we have in Perplexity Labs. If you don't want to use GPT-3.5 Turbo and you want to save costs, you can come use our APIs. Our long-term roadmap is to understand how people use it, battle-test our infrastructure for this at scale, start shipping these models into our core product too, and use them already in our core product for things outside of the summarization use case.

That's how we want to eventually stop being a wrapper.

Yeah, right now we've tied both together. So the only way to get access to our APIs is to be on our Pro plan, and then you get the Copilot unlimited experience as well as developer access to our APIs with a certain restricted rate limit. And those who want higher rate limits can come to us separately, and on an ad hoc basis we extend it for them. We have to come up with the usage-based pricing, obviously, and we have to build a whole API team for this, and that's sort of what we think we should be doing.

We are first trying to understand what is the market here. GPT-3.5 Turbo and GPT-4 already exist. And these are amazing models. And I think we can add a different value through our RAG, retrieval-augmented generation APIs, which are still not out yet, but we are working on getting them out. And in that world, I think we are committed to helping people deploy using our models and APIs for their applications if they're interested, right? So you want to start small.

I think the best thing is to start small. Allow people who want really tiny models to be deployed onto their product. We are a good fit for people who are trying to build consumer-facing applications because that way, they don't really care so much about SOC 2 Type 1, Type 2, and all these things. And we don't have all these things yet either. And they would benefit from all of the work that we have done on high throughput and low latency, because that's something we need ourselves to be users of our own API.

So they would benefit from all of that. So we are still going to be pretty focused and not go all in on the API and enterprise business, but see both of them simultaneously fueling each other.

Yeah, I've been thinking about this: how did we end up getting a lot of users despite not being vertically focused? And honestly, the answer is a lot of other people did market studies for us and sent their reports, and I read their reports. And the answer is that we are seen as this tool for research. We appeal to people who really wanted to do a lot of web research and dig deeper into topics and be curious about things.

That was a small total addressable market that we kind of cornered.

Citations and the fact that it's very accurate, very fast, and has a reliable UX. So we didn't actually take on Google. We never said, "Hey, guys, I'm the Google competitor. I'm gonna kill the monopoly," or something. Instead, we said, "Hey, this is a pretty useful tool. You guys are all spending a lot of time browsing and doing your research. Why don't you come and use this tool? It'll help you." And people started using it more as a research companion.

And even though that's a tiny fraction of the search market, you have to corner it first. Now we've cornered it, you've got a few million users. You start being useful to them in other ways, and then they're going to spread word of mouth to their friends and friends of friends, and you slowly grow. That's the idea.

Yeah, so we are expanding, right? For example, not every query needs an LLM. If you come and ask, like, time in New York or weather in New York, we would have written a small widget, similar to how Google does. You don't need GPT-3.5 Turbo. And that's why this is a fun problem to work on. This is not a problem that will be dominated by the company with the largest language model. Hence why I actually think Google is making a big mistake by trying to do whatever OpenAI is doing, right?

Like, going after them with the large—oh, they're having GPT-4, I'm going to train Gemini. When you're working on search, you're more about playing the orchestra. You're more about doing so many things simultaneously, together at once, that the user has a great experience but doesn't actually know what's going on under the hood, right? So our focus is to slowly expand the category of queries that we are really good at. We started being good at anything that requires you to read five or six different web pages and give you an answer.

Then we slowly expand in terms of what is in our strengths and what is in our competitors' weakness. And then be a greedy algorithm, just slowly start expanding. And if you have a pretty good coverage area, then you have a sufficiently large user base that uses it pretty regularly.

Yeah, we started with Bing, and I think we're not fully independent of other people's indexes.

We are building our own index. In fact, that is the most ambitious project that we are working on. Everyone's going behind training GPT-4. We believe that building your own index is even harder than trying to compete with GPT-4 because it's not a problem that's just solved with money. I'm not saying GPT-4 capability matching is just solved with money. You still need amazing talent and research teams to get it done, but there are at least two or three teams in the world that can potentially get there.

Building an index is really hard because it needs you to have a good team, some amount of capital, not necessarily a lot, and an actual product that people use. Without people using your product, there's no way to build an index. That's why most of the companies that were started earlier that tried to take on Google never really succeeded, because they never got the usage.

Yeah, you need the feedback loop, and you need to keep crawling more and more of the web that actually matters. And that's why we think we have a shot at this game.

Yeah, crawling is not the hard part.

You can write a crawler, rent a lot of EC2 instances, and keep doing it, deduplicating it, and keep expanding it. The size of the index is not important. It's the quality of the index and ranking. Quality of the index and ranking, these are the two most important parts. And that's why you need to get the good kind of queries. There's a lot of junk on the internet. The internet is Pareto distribution, power law, right?

Ninety percent of the value is in 10% of the pages. And you want to get that 90% value as a startup. Okay.

Yeah, I think open source models are definitely going to work out in the long run. Today it looks like they're lagging behind GPT-4, but that's also because the amount of compute thrown at them is way lower. It's also important for the world. Otherwise, if the person having the best model controls the prices, we'll end up in a situation like how NVIDIA, for example, controls the GPU compute, and nobody can tell them what to price H100s or something.

That shouldn't happen. If you want to access these large language models to build your applications, you shouldn't have only one single choice or two choices, especially when you don't control the price. So that's what open source models enable. They enable democratization, and they enable the fact that there's going to be a free market here. And we are supporting it in a way where we are not the ones building these open source models, because that requires a lot more capital.

But we want to make sure anybody can access them easily. So we are democratizing access to them through our APIs, through our Labs, and you can come and play with it. You don't need to go to ChatGPT at some point. These models will become pretty good that you can use these models through our Labs too. So that's the idea.

Yeah. So synthetic data is the trillion-dollar question. If you can actually make good use of synthetic data and use that to build the next most intelligent language model, that basically decides if the company that has the most compute in the world or data in the world, which matters more, right? And especially if we can get a smaller model to be really good through synthetic data, this changes a lot.

You actually don't need that big a GPU cluster for inference anymore. And that's why this is such an interesting question. And I think at some point, we'll all saturate on data that exists on the internet that's interesting enough for models to keep improving on that. The next big capability jump is probably going to come from the models teaching themselves. And then if that part is figured out, there is no upper bound to it. It can just keep happening recursively.

Recursive self-improvement, that's the holy grail of artificial general intelligence, right? And that part hasn't been figured out in the context of large language models. Currently, they're just still predicting the next word on the internet. But if they can be used to teach themselves, and if that's a scalable process that can be repetitively iterated upon without any human in the loop or with minimal human requirement, that's actually going to potentially lead to the most general form of intelligence.

Thank you.

Bye.