MAD Podcast
    MAD Podcast

    The MAD Podcast with Matt Turck

    Dust: Secure AI Assistants for The Enterprise with Co-Founder Stanislas Polu

    Stanislas Polu is the Co-Founder at Dust. We cover why company-wide AI adoption follows a traditional adoption curve, why focused assistants onboard non-early adopters better than general-purpose chatbots, and why retrieving the right company data remains a bigger challenge than model hallucination.

    10/11/2023

    Hosted by Matt Turck · with Stanislas Polu, Co-Founder, Dust

    enterprise AIAI assistantsknowledge retrievalLLM adoptionDust
    Listen now
    YouTubeApple PodcastsSpotify
    54 min · 1 chapters
    Contents

    Transcript

    Full episode

    0:00
    Matt Turck0:57

    Hey Stan, welcome to The MAD Podcast. Hey.

    Stanislas Polu0:59

    Thank you for having me.

    Matt Turck1:24

    So you are the CEO of Dust, which is a very interesting generative AI startup focused on productivity in the enterprise. The tagline is to build secure AI assistants with your company's knowledge. So we'll talk about what that means. And also, part of the reason why I've been looking forward to the conversation is that you're based in Paris, which obviously is near and dear to my heart. But I think it'd be really interesting to talk about what's going on in Europe in general and France in particular in the world of AI.

    Matt Turck1:44

    I know a lot of people in the US and internationally are sort of intrigued by everything that's been going on there. But that would be really fun to—

    Stanislas Polu1:45

    Of course.

    Matt Turck2:10

    Talk about this in a sort of educational way. So we'll do all of that maybe later in the discussion. But I'd love to start with your background because you're an entrepreneur, serial entrepreneur, and you've worked at some really interesting companies at very interesting times, specifically Stripe and OpenAI. So you started a company back in the day, right, with your current co-founder at Dust?

    Stanislas Polu2:37

    Yeah, that's right. We started a company, I think, 10 years ago. We met probably more like 15 years ago, studied in France. We both went to Stanford. So we were at Stanford also at an interesting time, which was at the time Facebook was just a few hundred people down the road. Twitter was even smaller. But anyway, it was interesting to see how it's hard to realize how big these things would become back in those days. But yeah, we started a company.

    Matt Turck2:46

    Is it something that people talked about on campus, though? Like, everybody sort of knew, or everybody's not?

    Stanislas Polu2:51

    So I was at Stanford in 2006, I think. So it was kind of the—

    Matt Turck2:52

    Very beginning.

    Stanislas Polu3:21

    Yeah, the downturn of CS. The CS department was tiny, tiny compared to what it became since then, or grew back to since then. And yes, there was a class by Dev Mackler, yeah, Dev Mackler, whatever, but a class about Facebook apps or whatnot. And so we were talking about Facebook, but I think it's hard as a student to realize the ecosystem you're embedded in while you're in that also kind of safe bubble that is the campus. So it's a very weird situation where your life can literally change by just going through a door a couple hundred meters away.

    Stanislas Polu3:57

    I didn't push that door at the time. I pushed another great door. I went to Oracle for nine months, which was kind of weird when you have Facebook and Oracle. It's like a bad decision, I guess. But I stayed there for nine months. It was awesome, experiencing what the big kind of computer science company looks like at the end of the—like when the journey is well underway. And then moved back to France, worked a bit there, and then we decided to start a company.

    Stanislas Polu4:29

    We started a company for the exercise of starting a company, so something stayed within us from Stanford, I guess. We didn't have a plan, didn't have a vision, didn't have nothing. And I think we made all the mistakes of the wannabe young, inexperienced entrepreneur, but that's the mistakes, that's the path you have to walk to learn how sausage is made, I guess.

    Matt Turck4:32

    And you pivoted a couple of times, right?

    Stanislas Polu5:00

    Oh, back in the day, yeah, a gazillion times, way too many times. We started with digitalized coupons for local stores. It was kind of the Groupon days. So we were like, oh, the kind of loyalty cards should be digitalized on the phone. Pretty tough to sell anything to local stores above $10 or something a month. So that was tough, but we luckily stayed in that business only for four months. Then we did an app that was aggregating all the publicly shared photos in real time.

    Stanislas Polu5:37

    So it was mostly Instagram, Foursquare, Twitter, and a bunch of other photo apps back in those days. So it was a B2C app where you could teleport somewhere, seeing the live feed of photos. That didn't work that well, but eventually we realized that there was a B2B interest. So we started building a B2B product around those kinds of feeds, real-time feeds of photos, and eventually realized that Instagram was taking the lion's share. So we pivoted on Instagram only and then realized that the brands, which were the only ones that had money to pay us—events and media didn't at the time—were interested in not the content, but more the analytics.

    Stanislas Polu6:17

    And so that's how we ended up doing an Instagram analytics platform solely focused on Instagram. And that's where we went from a long stretch of zero revenue to all of a sudden addressing a very nice niche. And we grew into a business doing between $1 and $2 million ARR with a small team of eight or 10. So it was kind of a nice little gig that we had.

    Matt Turck6:37

    Okay, very good. And so that company, as I was prepping for this, I think I read somewhere, I heard somewhere, you got into almost like a bidding war between Pinterest and Stripe. Can you talk about what happened? Well, at least you were in conversations with both.

    Stanislas Polu7:02

    I wouldn't call that a bidding war. No. So we basically, we were struggling to raise a Series A at the time. I mean, it was really hard at the time to raise a Series A because the growth was not explosive. The B2B on Instagram was not yet explosive. The B2C part was explosive, but the B2B part was kind of linear growth. And so we knew what we had to do to bump the growth, but it probably would have meant doing the same trick on another platform.

    Stanislas Polu7:41

    So start from scratch, a fully verticalized product on, let's say, Snapchat. And since we pivoted the entire team into that, it wasn't kind of a—we weren't born to do social media analytics. And five years in, kind of grew a little bit, to be perfectly transparent, a little bit tired of it, lost the kind of energy to push further. And that's why we started looking for a nice exit for the team, for the investors, for the company. And we chatted a lot with Pinterest, and the moment we got an offer from Pinterest, we were like, okay, is it the thing we want to do?

    Stanislas Polu8:15

    Is there a company that we're more excited about? And there was one. We were the biggest kind of fanboys of Stripe back in the day. And so we went and reached out to Stripe to ask advice and see if we could do something. We knew them because we were the first European users. And so that's how we started chatting with Stripe. We went through an interview process. It was quite accelerated and got a counteroffer, or an offer, from Stripe.

    Stanislas Polu8:23

    Within 24 hours, I guess, or 48 hours.

    Matt Turck8:30

    Okay.

    Stanislas Polu8:55

    The fun thing is that, just to give you an anecdote there, we went through the interview process and then Patrick called us back and said, sorry guys, I mean, we really love you guys, but you're not a US company. So it's one of our first acquisitions, it's going to be complicated. I mean, if you had been in the US, it would have been much easier and we'd probably go with it. But because you're in the EU, it's going to be tough.

    Stanislas Polu9:22

    So sorry. And knowing that news, we kind of chatted with a couple of people. On my side, I was working at the time a little bit with David Mazières, working on Stellar, who knows Patrick pretty well. And we were excited about me joining Stripe because Stripe and Stellar have a relationship, so we could maybe eventually sometime work back together again. And so David reached out to Patrick and said, you're making the biggest mistake of your life.

    Stanislas Polu9:52

    I guess time has flown enough that I can share that. And it's really funny how, first, Stripe decided to revisit that decision based on that new signal, and at the same time, how sometimes those kinds of things are really due to some very serendipitous stuff. And so eventually we got an offer from Stripe and joined them. It was 2014, I guess. Yeah.

    Matt Turck10:07

    No, thanks for sharing. I think that's always fascinating, like the sort of behind-the-scenes story about how those things actually work. So that's great. Thanks. So yeah, Stripe was super early, right? How many people were there when you joined?

    Stanislas Polu10:08

    80-ish.

    Matt Turck10:10

    80-ish. Wow. Okay.

    Stanislas Polu10:35

    Yeah. And so I stayed there for five years, all the way to 2,000 or 3,000 people. So I really saw the impressive growth and impressive trajectory that Stripe had. It was a fascinating experience. I really do not regret not pushing the Facebook door because it led me to eventually join Stripe, and really, no regrets on the Stripe journey.

    Matt Turck10:36

    Worked out.

    Stanislas Polu10:38

    Yes, exactly.

    Matt Turck10:53

    Like, any sort of stories or learnings from that phase of hypergrowth, like things you've learned that, I don't know, perhaps you apply to Dust these days?

    Stanislas Polu11:21

    Yeah, I think the biggest learning, at least for us today, and I think the biggest learning overall, was the incredible mode of operation of Stripe in the early days when we joined at 80 people. There were a very limited number of managers, very limited number of product managers. I think it was a forbidden word at the time, so no product managers. That function was kind of covered by engineers and managers, or engineering managers. And really, the company felt like it was really self-organizing based on a couple factors. There was a pretty strong density of talent, and so that's one factor which led to people trusting each other.

    Stanislas Polu12:00

    So if somebody said that they would do something, everybody would really trust that that thing would be done efficiently. And obviously Stripe kind of innovated in the open communication way of operating. So Stripe had, and still has to some extent, I think, open mailing lists that you can subscribe to for each team, so that information flows kind of freely within the organization. And so that enabled that group of 80 people, or more—I think it scaled to multiple hundreds of people—to really operate with very low coordination, which was very impressive to see, really fascinating to see actually.

    Matt Turck12:17

    What did you do there? What did you focus on?

    Stanislas Polu12:41

    Yeah, so I worked mostly on—we came in with our basically European hats. And so Stripe was a very US-centric company at the time, focused on cards. And I think we came with that kind of bias of saying, hey, in that distant country where we come from, my friends, there are other ways of paying than cards. And so I think I was motivated by trying to build that at Stripe. And so I mostly focused on—I started working with Greg Brockman on a small project for a couple of months, but after that, I really focused on building an infrastructure to add more payment methods to Stripe in a way that is somewhat scalable, which ended up being the V1 Sources API that covered a bunch of non-card payment methods.

    Stanislas Polu13:08

    And that now has been superseded by the Payment Intents and Payment Methods APIs.

    Matt Turck13:20

    So presumably Greg, who was the CTO of Stripe and then became the CTO of OpenAI, was your way into OpenAI. Is that what happened?

    Stanislas Polu13:35

    That is perfectly correct. That was kind of through Greg, who I met a couple of times before joining OpenAI, that I got to meet with Ilya as well. And that's kind of my entry point toward OpenAI at the end of my Stripe journey.

    Matt Turck13:53

    So maybe tell us about that. So when was this? When did you join OpenAI? And I guess, what was OpenAI like at the time? That was presumably before its evolution from a sort of nonprofit to the current commercial operation.

    Stanislas Polu13:58

    So yeah, I joined after the switch to for-profit. So I never knew the—

    Matt Turck13:59

    What year was that?

    Stanislas Polu14:24

    So I think I joined in 2019-ish. I'm really bad with dates, but roughly 2019. And it was after the for-profit switch, if I recall correctly. So, going out of Stripe, I started getting interested in AI. I played a little bit in a couple directions. I was trying to look for directions that had societal impacts. And so I played a little bit in the direction of autonomous vehicles, but I decided not to go further for many reasons that we can cover as well, but not necessarily interesting.

    Stanislas Polu15:05

    Then I played a little bit in the space of cybersecurity and AI and transformers, which is very much echoing today with the DARPA AI Cyber Challenge, the latest DARPA Grand Challenge, which is exactly about that. I played with that with a friend who is an expert in that space. We didn't find anything very interesting that worked. And then I started digging into the space of AI and transformers and math. And so that's the direction I started taking outside of OpenAI and eventually found that OpenAI was kind of interested in that direction and eventually joined them to work on that as part of their research team.

    Stanislas Polu15:46

    So I'm not a trained researcher myself, though I think the most surprising part to me was being embedded in a research organization, however unique it is, because I presume it doesn't look too much like a university research organization, but still a research organization. And so that was the biggest change for me. And yeah, if there's one thing I take away from OpenAI, it's that it's another level of talent, and the researchers that I got to interact with have always, and keep, really impressing me.

    Stanislas Polu16:30

    There was an incredible talent density at Stripe when I joined. I think that was an even bigger shock to me, the density of talent, because it's not talented people. It's like people you don't quite understand how their brain works type of people. And you have like 90% of them in the room that are like that. So it feels a little bit intimidating, but it was overall a very awesome experience. And then, obviously—sorry, go ahead. No, go ahead.

    Matt Turck17:05

    What you just mentioned is super interesting. I guess that's something that anybody paying attention to the world of AI has heard a few times now, but a big part of the success is around engineering as much as research, and that a lot of what OpenAI does is actually some very impressive engineering feats that support very impressive models, of course. But you mentioned organizationally the teams are embedded together. So how does that work? Is there a group that works on, whatever, DALL-E, and another group that works on GPT-3 or 4?

    Stanislas Polu17:41

    So what I've seen—when I joined OpenAI, it was really mostly a research organization. It was a bunch of researchers organizing teams, but you can't just top-down say to researchers what they're supposed to work on, obviously. So it's kind of a very free place to work on the direction you care about. In the era of large language models and transformers, obviously there is a way to orient an organization, a research organization. It's the way you allocate compute.

    Stanislas Polu18:07

    Which means that, as a researcher, it's often the case that you are free to work on whatever things you want to work on, right? But if the things you're working on are not aligned with the direction of the company, it's fair that you don't get the large chunk of compute. Or, to get good results, it's often the case that you need to get access to a large amount of compute. So you have kind of a natural incentive for researchers that don't want to be too much aligned to align themselves with the mission of the organization, because of that scarce resource at the core of it all.

    Matt Turck18:32

    So it's fascinating, but by default, people are not tasked with necessarily super precise things because that's how you recruit and retain researchers.

    Stanislas Polu18:57

    Yeah, at least in my time. So I think OpenAI doubled or tripled in size since I left. But in my time as a researcher, you were kind of free to explore risky things. But obviously, if they were risky and not very directly aligned with the mission of the group, then it would be hard to make an argument for getting hundreds of GPUs to test your thing. So any researcher has kind of a minimal set of GPUs that they can explore with.

    Stanislas Polu19:25

    But if you want to scale up, you naturally have to be aligned with the objective of the company. And yeah, overall, the research density was incredible. Sorry, I was trying to look at what I wanted to say next to echo what you were saying. So I joined not as a researcher; I joined as a research engineer. And one thing that is extremely interesting is that I think I saw the transition from research to research plus engineering being the key ingredients in building the largest and best models.

    Stanislas Polu20:09

    But even very early on, OpenAI was very open about the fact that you could be one or the other quite fluidly and change between the two. And it was so deeply embedded into the organization that, as a research engineer, you could be evaluated on a ladder that was both research engineering and research science, right? And so, as a research scientist, you could also choose to be evaluated on both at the same time. And so I did mostly research, but my projects were kind of engineering-heavy. I was evaluated as a research scientist, even if I was officially a research engineer.

    Stanislas Polu20:31

    So it's interesting how very early they understood that you need both together to really achieve great stuff. Absolutely.

    Matt Turck21:00

    And I guess we're recording this a few days after there were some rumors that OpenAI was now doing secondary transactions at a $90 billion valuation. And there was this Reddit thing where Sam Altman may or may not have said that AGI has been achieved internally. That obviously set the Twittersphere, I guess, X-sphere, whatever they call it these days, on fire. So maybe at a sort of high level and taking a step back, what's your current take on OpenAI as an organization?

    Matt Turck21:25

    And they're going in lots of different directions, doing lots of different things. Are you super bullish on it as somebody who spent years there? Do you have any reservations? I'm just curious if you have a take on that.

    Stanislas Polu21:50

    Yeah, so to answer the question, to be very clear, I think I had my insider hat for the previous questions because we're talking about a period where I was inside OpenAI. Now it's been a year that I'm outside of OpenAI, so I'll take my outsider hat to answer the question, with my knowledge as an insider obviously embedded in my brain. No, I think I'm still extremely bullish on OpenAI because I think this is a unique organization that was able to raise a very large amount of capital to power a research organization and a research project that is very hard to reproduce anywhere else.

    Stanislas Polu22:35

    Obviously, you have Google and DeepMind, Meta to some extent, Anthropic, but it still remains kind of uniquely positioned there. And the potential is humongous. So I understand why some people may decide that the expected value is humongous because the potential is humongous. I think the expected value is humongous because it's not if, but more when. But obviously, if the when is very long in the future, then the value starts decreasing because you apply kind of a rate to it.

    Stanislas Polu23:11

    But it's hard to believe that it's going to take 100 years to achieve something spectacular, clearly transformative for society. And today it remains clear that the most efficient way we know how to push that technology is by scaling it up. And now, if you think about the places where you can really scale up that technology, OpenAI came out as a leader. And being in that position that they built as a leader in that space has allowed them to hire even more talent.

    Stanislas Polu23:50

    I mean, I think I've seen from the outside that they've been on a hiring spree over the past year. And so that creates a set of unfair advantages for them that can't be ignored. So yes, I'm still extremely bullish on OpenAI, but also, if it takes too much time, then the world will move on and the investments that have been put in there are so big that the end won't be that nice, I guess. But if it takes five years or 10 years, then that's probably where it's going to happen.

    Matt Turck24:17

    Great. So moving on to Dust now. So you made the decision to leave that world of OpenAI to, I guess, scratch the entrepreneurial itch once again. So maybe walk us through the thinking and why you chose the specific problem that you're addressing at Dust?

    Stanislas Polu24:45

    Yeah, the original thinking, I think it remains very valid even if we explored in different directions along the way. But I think the original thinking is the following. So it was last summer, not this summer, the summer before, so summer 2022, end of summer 2022. And the realization is pretty simple: those models are already extremely useful. They're extremely powerful, and they are not deployed in the world at all. Like, nobody uses them. So it was pre-ChatGPT.

    Stanislas Polu25:09

    And so the obvious answer to that was, well, obviously the research is going to be an important part of making those models more useful. But there's probably another question that is completely underexplored today, which is the product part. There's probably a product problem explaining why those models are not used, despite them being extremely useful. So obviously that hypothesis has been proven true because a couple months later ChatGPT came out, which is kind of a nice product packaging and research packaging. Because if you think about ChatGPT, it's not only a good product, it's also reinforcement learning from human feedback that makes those models more useful, that showed that indeed there were unlocks at the product layer.

    Stanislas Polu25:43

    Now, despite the fact that ChatGPT kind of spiked the usage of those models in the world, they are still quite underdeployed compared to their potential. And we are really focusing at Dust on trying to explore that product layer to make that technology the most useful and the easiest to deploy inside a company to really save time for the knowledge worker in general.

    Stanislas Polu26:12

    So we're taking a pretty horizontal approach. Dust is a place where you have models that interact with humans in conversations. You can have multiple models, you can have multiple humans. So really trying to focus on the productivity setup and teamwork. And you can not only have generic frontier models, GPT-4 and Claude, but also custom assistants that you define and that will be focused on certain tasks with access to your company data, which makes them useful for achieving tasks within the B2B, or your company, or your work setup.

    Stanislas Polu26:55

    So this is really that exploration around making the AI work for your team and trying to understand what it takes to deploy those models within a team. It's very interesting because today companies are all early adopters, meaning that you take any company, the management will want to have more AI, more AI in product, more AI for their employees, more AI something. And it goes all the way to the biggest companies. Like the biggest companies, the boards talk about AI, and even the largest, oldest companies, they are all early adopters of AI.

    Stanislas Polu27:31

    But the really fun part is that even if the company is an early adopter, as you roll out the solution within the company, you're faced with a traditional adoption curve, meaning that within the company, you won't have only early adopters. You'll have a certain percentage of early adopters and a lot of normal people and a lot of late adopters. And so it's really hard to roll out a solution that gets adopted by the whole company. And so that's where there's a kind of very interesting product question of how you optimize that kind of propagation within the company from the early adopters that understand how that technology works and will use a product day one, and how you convince the later adopters to start using the product.

    Matt Turck28:20

    Yeah, that sounds fascinating. Let's double-click on that. So especially for something like an AI assistant, you would imagine that ChatGPT has done a lot of the hard work of turning most people into early adopters because it's so simple and intuitive. What are the blockers in the enterprise? Is it the question of hallucination and all the things that slow people down? What are you finding?

    Stanislas Polu28:43

    Yeah, so even if ChatGPT has made it very easy to use those models, I think a large chunk of ChatGPT users are only mundane ChatGPT users. They don't necessarily use it in a productivity setup. They just tried it a couple of times. And those people, I mean, a lot of those people really did, even after trying it a couple of times, really didn't get what it is as a technology. What can happen is that you try it and you ask a question on which you are not at all an expert, and you get an answer that looks nice.

    Stanislas Polu29:01

    And the natural reaction to that is that this is like a search engine. I ask a question, I get an answer, the answer is correct, which is far, far, far from being true, right? I think something that everybody should try: ask a question on something where you're not a domain expert, and try to ask a question on something where you are a domain expert, and you realize the difference of experience you get from that technology by asking those two different questions.

    Stanislas Polu29:36

    When you are a domain expert, you'll see that there is stuff that is generally not true, stuff that is made up, stuff that is imprecise. And when you're asking a question on something where you're not a domain expert, you don't see those kinds of deficiencies. And so I think even if some people have used ChatGPT a couple of times, they don't necessarily grasp the nature of the technology and really definitely don't grasp how they can use it in a productivity setup.

    Stanislas Polu30:15

    Because we want to interest ourselves in deploying this model in a kind of company setup, it's natural to give them access to the company data. And there, it's also extremely interesting. Basically, if somebody really deeply understands the nature of those models, they can make great use of a very general assistant with access to a large amount of data because they naturally find the questions that are the right ones to ask for getting value out of it. Well, if you're not an early adopter, asking such general assistants questions, you generally fall into traps and fall into kind of use cases where you're very disappointed.

    Stanislas Polu30:50

    And what we've learned from deploying within our users is that for a large amount of the people within companies, the best way to onboard them to the technology is to not give them a very general assistant that can do anything and everything, but instead focus the assistant on some use case that feels more like a tool, such that those people understand, see the affordance of the tool, understand what it's made for. That could be kind of just something that generates SQL queries based on table definitions.

    Stanislas Polu31:18

    You just really understand what it's made for, and there's much less magic, much less uncertainty, and much higher probability that the answer is going to be correct. And so if you build them those tools, they will gladly start using them because they are kind of magic tools, but they remain tools. Doing so, they start educating themselves on what those models can do. An example could be something that just creates an answer from a knowledge base, and maybe somebody will show that person that, oh, but you can also use GPT-4 to translate it in Spanish.

    Stanislas Polu31:38

    And they're like, oh wow, incredible, and start chaining those different tools all the way to learning to use those general frontier models.

    Matt Turck31:47

    So that's kind of what you narrow down. You increase constraints, so you just narrow down the scope so that people don't get lost.

    Stanislas Polu31:58

    Exactly. And that's the kind of entry point to converting the non-early adopters within companies into using the technology and being productive on top of that technology.

    Matt Turck32:19

    So does that mean that for now you find yourself, which I guess is pretty natural at this moment of the cycle, doing a lot of effectively services where you sit down and help people define the problem? We obviously do some of that as an early startup should obviously do.

    Stanislas Polu32:41

    But we really think that it's something that can be scaled up through product, basically. What you really want is a product where there is no question that the product is better than using ChatGPT because you have GPT-4, you have Claude, you have productivity features, you can collaborate and stuff. And those early adopters can become the builders for the later adopters, meaning that those people that are early adopters, maybe engineers, they understand technology, should be given a platform where they can easily create those tools for the other ones in a way that, at the end, we don't necessarily need to be involved inside the rollout of that product.

    Stanislas Polu33:30

    The product by itself starts with the early adopters, starts with the engineers, and the engineers might build tools for the ones that require a little bit more guidance. And I think all of that can probably scale through product and not necessarily handholding and kind of custom work with our clients eventually. But obviously, in the beginning, to learn and to build that product, we need to do that ourselves with them. Yeah.

    Matt Turck33:38

    And you're building this for developers? I mean, clearly for business users, but you also have an interface for developers.

    Stanislas Polu34:07

    So Dust, one year ago, started as a developer product, actually. So we are doomed to pivot a little bit, I guess, as entrepreneurs. And so that was kind of an early pivot, actually before Gabriel joined me. So I was still solo on the project. It started as a developer tooling platform to create LLM apps, so chains of calls to models and external APIs. And we always kept that product alive.

    Stanislas Polu34:30

    So it got buried down deep inside a product. We kept it alive because we always knew that customizability of those assistants is something critical that people will want. And so today, finally, after six months of iteration on a more general product that is usable by anybody inside the company, we kind of plugged back the old system inside the new one, meaning that today you can build an assistant and instead of having an action that is retrieving information inside the company, which is our base action that we have supported in the product, you can also tie it to a custom large language model app that might hit APIs, go do search on Google.

    Stanislas Polu35:18

    And so building those custom actions requires a little bit of low-code capability. So developers plus tinkerers, I would say. And so we have that in the product, and we do believe that being able to customize those actions will be a big part of our users being happy with the product. Great.

    Matt Turck35:45

    And we talked about the hallucinations a little bit and people wanting to search based on the company's data. So where does that fall into the emerging RAG architecture and how people solve the hallucination problems in general? Is that something where you will connect Dust to a vector database, or is the idea that you abstract all of this away into Dust so people don't have to worry about the architecture?

    Stanislas Polu36:15

    Oh yeah, we definitely abstract all of that into Dust. So basically, when you start using Dust, you can start using and creating custom assistants, and eventually, if you want to give more context to your assistants, you can connect your Notion, connect your Slack, connect your GitHub, and we take care of everything: the synchronization in real time and, obviously, the chunking, the embedding, and the retrieval part. So that's something that we do internally. At least, I think, to be honest, the frontier models, they don't hallucinate that much anymore.

    Stanislas Polu36:51

    The main, I think the biggest challenge today is the retrieval part: retrieving the right information. In particular, given a question, there are many questions that are not well fitted to semantic search. So that is a complex problem. There are many questions that have the expectation that you have access to structured data on which you would want to run a query. As an example, a Notion database that is, in fact, a board, a product board with tasks that are moving. If you ask a question about that and you only have textual information that's been chunked, the models will never be able to answer your question.

    Stanislas Polu37:17

    And so there are also a lot of interesting things to dig into here. And then for the questions that are a good fit for semantic search, it's really if what you're searching is too large, the answers, the chunks that you'll be finding will be a bit noisy. And the models, even the best ones today, they have a tendency to get a little bit disturbed by that noise, meaning that they might not hallucinate too much, but they might skip information or not attend to the right information, even if it's there.

    Stanislas Polu37:51

    And also getting that information there is really a challenge because those context sizes, they're getting bigger. Obviously, they will keep getting bigger. So GPT-4, 32K, Claude, 100K. It's pretty sizable, but if you put too much information in context, the model still gets a little bit lost. So I think the right product solution there is really to incentivize people to create, again, assistants that are more focused on a given task such that the scope of information that they tap into is more limited, such that the retrieval is better, such that there's less noise in the retrieval and the model gets more pristine and quality answers.

    Stanislas Polu38:28

    So it's basically context size and model quality that make it such that an assistant that knows it all within a company is still a little bit far-fetched because it'll get confused if it sees too much information.

    Matt Turck38:33

    Hence the reduction in scope and having multiple tools.

    Stanislas Polu38:45

    Yeah, that's both increasing the affordance of the assistants, but also, even for people that understand the technology, having much higher-quality answers, much higher-quality answers.

    Matt Turck39:13

    And you mentioned earlier that under the hood, you have multiple models: GPT-3.5 Turbo, Claude Instant, and you can have custom models. How does that work? Do people pick which model they want to use for, I don't know, cost reasons or whatever? Or are you building a routing layer that pushes certain queries into certain models?

    Stanislas Polu39:40

    So we're trying to stay pretty transparent there. So if you are a free user, you only have access to the small models, the ones you mentioned. If you're a paying customer, you have access to GPT-4 and Claude, and we foot the bill. We want to be in a business where you're not thinking about tokens. And as we build the product, we are fine using way more tokens if it increases the quality of the answers because the cost will go down. So that's fine.

    Stanislas Polu40:06

    Even if you're losing money today, you'll be earning some money tomorrow. So we expose the models that are available and we let our users pick the model they want when they build their assistants. So we try to be quite model agnostic. And today we indeed have GPT-4, Claude. We're probably rolling out, just for fun because we're in Europe, Mistral very soon, which will be a much smaller model. So the risk is that people start comparing GPT-4 with Mistral 7B, which isn't a fair comparison because it's a very good model, Mistral 7B, but obviously not at the scale of GPT-4.

    Stanislas Polu40:31

    But we really want to focus on the product layer. And so we're quite model agnostic, and we actually quite transparently expose which models are available, and you can pick the one you want to use yourself as a user today.

    Matt Turck40:55

    Okay. Do you think that's a space or an idea for a feature or even a company, this idea of having multiple underlying models and you just route prompts or queries into the model depending on what a given model is good at? Or ultimately, are the best models fairly undifferentiated?

    Stanislas Polu41:23

    Yeah, I think, to be honest, I think GPT-4 and Claude are a bit differentiated. I mean, they're obviously both frontier models, very large, very general, but they have a different vibe, right? Obviously, that's something that you can rewrite by prompting, because as you prompt, you bias the distribution, and so you can give it the vibe you want. But when you use the unprompted models, they definitely have a different vibe. And so the reality is that I think we're in a world today where the early adopters of technology embedded within companies, they want to have access to GPT-4, they want to have access to Claude, they want the raw model because they're used to it and they don't want any bullshit around it.

    Stanislas Polu42:07

    They just want the raw model because they know it and they kind of want to make sure they would have the same value by going to ChatGPT or going to Claude. So that's the reason why we definitely have raw models available in the assistants. And on the business of routing to the right model, I mean, it depends. You can scale down the model size if the task is very narrow, right, and fine-tune your model to that narrow task.

    Stanislas Polu42:38

    And if you manage to define that task precisely, then you can have that story about routing. But I think in the realm we are kind of exploring, which is productivity for teams inside companies, most of the tasks are pretty general, right? Most of the tasks are pretty like a human would do. And so you just want the best models. And as it happens today, the best models are GPT-4 and Claude. So that's what you want.

    Matt Turck42:45

    So what's on the roadmap for Dust? What are you guys doing in the next, I don't know, 12, 18 months?

    Stanislas Polu43:07

    Yeah. So right now we are really validating the current product iteration that we have. I think we are on the precipice of repeated sales processes. So that you could call PMF. I wouldn't quite qualify it as PMF yet. So really kind of validating the strong hypothesis we've made recently. I think going deeper inside the product—if you have something that kind of automatically discovers that some data that you found is a structured data source, and instead of just presenting random chunks to the model, you ask another model to generate a query and you get a table as a result and you present that to the model—and all of a sudden you're capable of answering quantitative questions that are based on structured information present within the company, are the kind of deeper product explorations that we're very excited about because, at the end of the day, it's about creating value, and many questions within the company are obviously of a quantitative nature.

    Stanislas Polu44:27

    And I think because we are on the verge of feeling very confident that we have a product that scales naturally inside the company, there's going to be, as with any startup at the stage we're at, a lot of focus on distribution and shaping the product. So that's along with our go-to-market strategy. And so that's not necessarily technical work, but more product work and customer-facing work that will allow us to go from a pre-PMF to a post-PMF company.

    Matt Turck44:29

    Very cool.

    Stanislas Polu44:29

    All right.

    Matt Turck44:56

    So maybe to close, I was hoping to spend a few minutes, as we said at the beginning, chatting about AI in France and Europe. It's actually really interesting you mentioned Mistral. Mistral, a few months ago, was the company that everybody loved to hate on Twitter because they raised so much money, and people reacted strangely to that, in my opinion. But at least that was entertaining. But it's yet another example, like with you guys, with Hugging Face, with D-ID, who is right on, others, of just the vitality of AI in France and in Europe in general. The UK has companies like Synthesia—that's my investor plug—as well as VidIQ, which is also my investor plug, but there's so much going on.

    Matt Turck45:32

    So why is that? Again, in Europe, but in France in particular, why is there such a heavy concentration of talent around AI?

    Stanislas Polu46:11

    Yep. So I think our education system is very well aligned with the kind of skills that are very useful in that new AI era. We've always been very, very—I mean, math is kind of the thing that is elevated as the skill that is synonymous with success in academia, I guess, and success during your education. Not for everyone, obviously, but French education resonates a lot with math. And there's always been a very strong scientific talent coming from France, I think.

    Stanislas Polu46:51

    What's so special this time is that we've seen the first wave of startups in France that have been successful and have created a kind of local ecosystem that is much stronger than what it was before. And there's been a lot of—I mean, I think DeepMind and Meta have invested a lot in France as AI labs over the past 10 years, I'd say.

    Matt Turck46:51

    Yeah.

    Stanislas Polu47:01

    And they've created a very large pool of talent, from PhD students going to Meta, PhD students going to DeepMind, staying in Paris, staying in London.

    Matt Turck47:30

    And maybe just double-click on that, because that's indeed interesting and I think was very important. So Yann LeCun, who's obviously one of the godfathers of deep learning and French, although he's been living in New York for a very long time. So he launched his team, which is mostly FAIR, and then head of AI at Facebook. He launched his team mostly in Paris, right? Is that right?

    Stanislas Polu47:37

    To be honest, I don't know the details of all Meta. I believe there was a lab in—obviously—

    Matt Turck47:39

    It's a lab in New York, but there was like—

    Stanislas Polu47:59

    There's a very big lab. What I know is that there is a very big lab in Paris, and there's been a very big lab in Paris for a very long time around AI. And a lot of talent grew into that lab, in the same way that some talent grew into the DeepMind Parisian lab and the DeepMind London lab. And so, as a consequence, the reason why Paris is so attractive today, I believe—that's my hypothesis—is that it's a pretty deep pool of AI research talent, but one that is not too coveted yet.

    Stanislas Polu48:27

    Meaning that if you want to build a strong AI research team today, it'll be 10 times easier to do it in Paris than it is to do it in SF, competing with OpenAI as a lab, as a competitive lab too, in the same hiring market.

    Matt Turck48:31

    Still hard, but easier.

    Stanislas Polu48:46

    Still hard, but much, much, much, much easier. And so I think that's why. So obviously Mistral was started by French people, and they have their connections in the French and London ecosystem. So that's kind of natural to them. The much more interesting case is the Poolside case, because they are U.S. founders—or I don't know if they're U.S., but non-French founders—multi-time entrepreneurs in the U.S., and they elected to put their main presence in Paris.

    Matt Turck49:03

    Yeah.

    Stanislas Polu49:33

    And my main hypothesis is that it's because the Parisian talent pool is still accessible at this time. Obviously, if many more startups and other labs come in, that might change. But right now, that's probably one of the main reasons why there's been an attraction of startups around AI in Paris. And the fact that the ecosystem is ready for that, meaning that we've had multiple successes, people starting to come back from stories in the U.S. And so, as founders, you have more support.

    Stanislas Polu49:50

    I think we're in a world today where U.S. investors are much more okay to invest in France, et cetera, et cetera. So that's kind of that set of things that come together and makes Paris an interesting place.

    Matt Turck50:18

    Is that a thing you're seeing indeed, like technical talent moving back to Paris? Because just, when was it, maybe last week or the week before that, there was the latest amazing thing that Xavier Niel announced, which is like another AI research lab. But part of the goal stated, at least publicly in the press release kind of thing, was just to attract back French AI researchers and technical talent back to Paris. Are you seeing anecdotally some of your friends or former colleagues from school moving back, or is that a trickle?

    Stanislas Polu50:50

    Researchers, I don't know, because again, the community of AI researchers, of French AI researchers, I think, is dense in Paris already, in a sense. Then in terms of technical talent, I think the same story can be made. We've had a couple of very nice scale-ups in Paris over the past five, ten years. And so it has really densified the talent pool in terms of engineering as well, which means that today also, it's much easier to create a very A-grade type of engineering team in Paris than it is in SF because the pool compared to the demand is probably much larger, right?

    Stanislas Polu51:34

    And it's quite high quality. What's going to be exciting is to see what that lab—I mean, I don't know if that lab will bring talent from the outside back into Paris, or if it will grow from the Parisian pool of AI talent, but it's awesome that there are so many opportunities. That's all for the positive stuff in Paris. I think the biggest question, obviously, is the question of funding because those models take a lot of money to babysit.

    Stanislas Polu52:04

    And so it's great. It's already awesome that Mistral was able to raise that much, that Poolside is able to raise that much. But it's a trickle compared to what OpenAI is raising, compared to what Anthropic is raising. And so there's going to be a big question of how that scales, where the money comes from, et cetera.

    Matt Turck52:23

    Okay, super great. Well, very exciting. That feels like a good place to leave it. Thank you so much. Where can people find you online? Where can they learn more about Dust, whatever? We'll put some of that in the show notes, but what's the—

    Stanislas Polu52:54

    Of course. dust.tt, that's our URL. They can try the product for free and play with it, create custom assistants, et cetera. And then they can contact me. I used to be pretty active on Twitter last year. I've been busy building stuff more recently, so I'm a little less active, but there's a good chunk of content on my Twitter account, @spolu, like Stan Polu, P-O-L-U. DMs are really open.

    Matt Turck53:08

    Okay, very good. Thanks again. That was a lot of fun. Really appreciate it.

    Stanislas Polu53:38

    I had a lot of fun as well. Thank you very much for having me. Thanks for joining us for The MAD Podcast. We're back here every Wednesday with new conversations with leaders in the machine learning, AI, and data space. And if you like this show, you can also find the video recording of not only this episode, but many, many more over on the Data Driven NYC YouTube channel. Thanks again, and catch you next week.