Notion: An AI Assistant to Work Faster with Head of Data Daniel Sternberg

Yeah, for sure. And what it means is a little bit different than even when I joined about nine months ago. So, when I joined Notion, we'd had a data team for some period of time already, for a couple of years. And the data team at Notion looked pretty much like you would expect, I think, at a lot of companies like ours: data science, data platform, data engineering. That's historically, pre this current AI focus.

That's really what the team was. That was the team I joined back in the last summer. Putting AI aside for a second, in that space, I think Notion is in a particularly interesting position. That is what got me excited about joining the company in the first place, which is: Notion, hopefully many of you have already used Notion, I hope. We are not just a, here's another buzzword, product-led growth B2B SaaS-type business. We also have a very, very large personal user base.

This differentiates us, I think, even from a lot of companies that are like the flagship companies of PLG, in that we actually have a very, very large personal user base. And so while our business model and business looks a lot like a B2B SaaS business, we have this—this is what got me interested. One of the things that got me intellectually interested, we have this very different, almost consumer-like availability of data about how people use our product. Because if we're doing our jobs right, whether you're a personal user or a user at a company, you are using Notion day in, day out.

Really, it's a core tool, the way we use it at our company, for collaboration, for knowledge management, for task and project management. And so people are in it day in, day out. And so we have an ability, I think, to look at the business and drivers of the business and tie them back to user behavior and customer behavior in a way that I actually think very few companies in B2B SaaS do. And I've worked in B2B SaaS before. And so I think that was something that got me excited initially.

And so when I joined, really it's a mix of this normal business internal analytics use cases of data, which were getting quite significant for the reasons I just stated. And then in addition, some data products, or some products that depended on our data infrastructure, that our team was supporting through building out and scaling data infrastructure for the company. Search—we were just talking about traditional search a moment ago. Search is a part of the product experience in Notion that we have to have because we have very large workspaces in a lot of cases.

We launched features around workspace analytics for our enterprise customers last year so they understand how people are using Notion in their workplaces, et cetera. All of those depend on infrastructure that we were also actually using for internal analytics as well. We had to scale that infrastructure further for these use cases. And then AI kind of came along about, I want to say, I guess, eight months ago now, almost.

Yeah, for sure. Some of those, we've let people have our logo on their websites. I feel like I can share those ones. I mean, we use Snowflake internally for the core data warehouse. We also have our own data lake infrastructure that we've built out ourselves on top of S3. We're in AWS, mostly in AWS today. We use, for internal analytics stuff, Hex as a product actually quite a bit. So we're an early user of Hex.

That actually happened before I even started. And we've expanded it quite rapidly across the company. We have used a lot of—I think maybe I'll step back a little bit. My understanding of the history of the data infrastructure at Notion is that it actually predates any data team at all. We had a head of growth who was there in probably 2018, 2019, who stood up pretty much exactly what I think you would stand up if you didn't have a data team and you wanted to use various vendors to solve your problems at that point in time.

And it was Snowflake—lots of companies use Snowflake—but we had a lot of other, some of these other vendors, I don't want to name all their names, but things you use for ingestion, things you use for data transformation. And I think what's happened over the last couple of years at Notion is there are parts of our stack, where there are parts of our business or product needs we want to support, where the scalability and cost-effectiveness of those tools starts to not quite work for parts of your stack.

So eventually, we went out and built our own event pipelines. We went and set up our own way of landing data for our largest datasets in the data lake because it really was not scalable for us to use some of these vendors anymore. In other cases, there's a long tail, I think, for a lot of companies, of different integrations in terms of vendors other parts of our company are using that it makes sense that we don't want to really build that long tail ourselves.

And so we've had to think about that. But what was interesting to me is as AI started to become more of a focus for us, some of this homebuilt internal infrastructure that we were already starting to develop for some of those use cases started to be something that actually helped us and kind of was a bridge to getting some AI product experiences out faster, as well as some that we're working on that I can't share too much detail on today.

Yeah. So we are in the early period of AI features in products like Notion, and so we were focused on getting something out into the world pretty quickly. Today, Notion AI, I would say, can add value to anything you're trying to do around creating content in Notion. What we ended up finding over time is that a lot of the most intensely used use cases of the current Notion AI experience are around summarization, improving content. One of the really standard ones that a lot of people use is generating action items from meeting notes.

Like, that's the thing that comes up a lot: summarizing that content. We have some automated ways where you can create a block at the top of a page, and it will automatically update with a summary as that page changes. So imagine you're working collaboratively with your team. You're building out, I think we often are doing, or like writing a product requirements doc or something like that. You want a summary of that at the top so people can quickly glance and see, is this the thing I'm looking for right now?

So those are some really common use cases. Another one that came up for us a lot is—and this was very much a commoditizable thing, but nonetheless really valuable really quickly—is we have a lot of larger enterprises and large-ish enterprises using Notion. And many of them are multinational. And Notion ourselves, we have field offices all over the world—in Dublin, Seoul, Tokyo today. And we find, actually, that automated translation is something that these models are quite good at really rapidly.

And being able to have that capability in the documents that everyone at your company in different locations is working in makes a really big difference. So imagine we're launching something like Notion AI and we're going into the Japanese market with that. They were very excited about it. We can actually generate content really quickly to help get us to really take all of the things that our product marketing team is building and have the Japanese sales teams have access to that and then build off of that.

So there's a lot of use cases like that as well that we've found really useful internally, and we've seen our customers using too.

I guess, how does it work?

Yeah, and this is evolving really rapidly. So I think we're taking a pretty pluralistic approach to how we're integrating with partners right now in this space. And while those are the two that I can say are, yes, publicly, like we have leveraged both of those partners, we are also designing things, I think, in a way where we know this space is changing really rapidly. You just heard earlier about this debate around open-source models versus these really huge LLMs coming from some of these companies like Anthropic and OpenAI.

We want to be set up in a way where we know what the core components of the stack that we're building are, but we can, in many cases, swap vendors out relatively easily if we need to because these things are changing. We already have some ML on the kind of search side of the stack, so we can leverage some of that infrastructure that we've built as well. But I will say it's very early, and I think actually a lot of the decisions we're going to be making about Notion's AI stack are going to evolve a lot as new capabilities come out in the coming months and through this year.

So I would say it's very early in terms of what we needed for that part of it. I want to go back to one thing I think you asked earlier that I didn't fully answer, I feel like, which was around Notion being very fast to market. How did that happen? One thing just to call out is while no one saw Notion AI capabilities before ChatGPT, I'm not going to give us too much credit. It wasn't way, way, way before, but we were definitely working on this a little bit before that.

And there was a pretty big focus at that point. The other thing we did was, once we realized that this was something we needed to start really investing in, our approach was to actually bring a small group—it was kind of a tiger-team-type approach—across my data platform team, across some of our product engineering teams, a very small number of people actually, with a lot of targeted effort from our co-founders as well, to get an initial product to market quickly.

And so it was a small group. And my job and the job of a few of the other of us kind of in that management layer was really mostly about how do we unblock this group to just get this done and reprioritize things at the company so it can happen quickly. Now, over time, that's already evolved maybe once since then. And there's actually an AI core team that is part of my group now that's going to evolve again as we continue to make these investments.

And so I do think the right way to get to market quickly for us, at least, looks a lot different than probably how I expect AI development to operate at Notion in three months from now, six months from now.

Yeah, I think it's going to look pretty different depending on what stage your company is at. I think for us, the biggest things that we are benefiting from are, number one, that a lot of the initial use cases around Notion AI were actually things that we could build basically by integrating some partners, by leveraging some of the partners that you mentioned, and just AI-aware product engineers, which we happen to have a few of.

So that was great. And it also helps that one of our co-founders, who still actively writes code, was in his own world working on this for some time as well. And so that gave us an advantage initially. Now, for where we're going from here, I think for us at a company at our stage, it's beneficial for us that there were some critical data transformation and processing problems that we were already working on. And that's an advantage for us, I hope, going forward.

So, for example, some of the ways we need to process all of the content in Notion workspaces, given the way that data is structured, depends on some really large-scale processing that we had started to do for the purposes of, to some extent, internal analytics, for search, and other use cases as well. And so now we're kind of in this mode where—and this is kind of fun because I feel like I can talk about the tech without telling you actually what we're building.

We're in this fun place where my data platform team, some of the folks over there, are working on, okay, how do we scale that processing to support some of these AI use cases that are going to be quite intense? And then we'll be looking at new partners and adding new parts to our stack on the AI side. One of the things we did a few months ago was actually move some of my data platform team over to be part of this AI group because they knew that infrastructure already and could build on top of it.

Meanwhile, the data platform side is thinking, how do I scale all of this up to work in a way that has the latency requirements and real-timeness and all of these things that we're going to need for some of the things we're going to be building and rolling out soon?

Yeah, for sure. So first off, I think the initial version of Notion AI that you all have seen is not a radical transformation of the Notion user experience. I think that's on purpose. I think we're all going to be going on a journey in the next couple of years, and this, I think, is a big part of what people talk about when they talk about a platform shift, or the term, the buzzword that we keep hearing, is that we need to start from a standpoint where people have an understanding of how they start to use this product.

Over time, I actually think there will be a more radical transformation to the user experience. I personally don't think it's all chat all the time, like the way we are all going to interact with all products. But there are going to be, we're already starting to think about, okay, as we go deeper in this direction, what are some of the ways our overall product surfaces are going to need to change and evolve? Given the overall mission that we have around making software toolmaking ubiquitous, around allowing really easy collaboration and knowledge sharing, I do think that's going to change over time.

But in the short term, it's really about how do you create ways of integrating AI into the experience in ways that are very natural. And I think Notion's advantage here as a company and as an organization is that design has been very central to our experience from the very beginning. So I'm going to correct one thing you mentioned at the very beginning. So Notion, nobody really used Notion in the world until 2016. But Notion actually started in 2013 and really was more of almost like a research project for a period of time.

The 1.0 release that came out in 2016, that I think started with people using Notion for docs and wikis, and in 2018 for databases, really stemmed from a period of time in which Ivan and Simon, who are two of our co-founders, were basically restarting this company, living in Kyoto. You can go read more about this online. You don't need to hear it all from me. But essentially, it was the two of them going back and forth for like 16 hours a day. And it was one of them working in Figma doing wireframes and mocks, and then the other one implementing it, and then them playing around with it and repeating over and over again.

And when we were starting on Notion AI, we brought design in very early too. So part of that team, one of our earliest designers, started working on that team. And I was watching this in Slack almost like I wasn't that in the weeds of that part of the product development process. But really, day by day, here's some mocks, getting feedback from all the way up to the co-founders, getting feedback from them on it, doing another rev on it, testing it out.

And then the last thing I'll say is we use Notion so intensely internally. We dogfood the crap out of it ourselves. And so we were obviously playing around with this for quite some time. And that's why, if any of you were in the alpha before Notion AI was available to everyone, which was for a couple of months, there were things that we added in over that period of time that we were testing internally constantly. And so I think that makes a huge difference for us as well.

Yeah, that's a great question. I am excited about agents, which is the thing that came up recently, and I'm excited about that development. And it's definitely something that I'm keeping in mind, and there's ideas around it. I think we're excited about, for us, and this is one of the things that I think is really exciting, what we want to be able to do today. If you use Notion AI, generally the generative things that it will build for you are built off of the content basically on the page that you're working on in Notion.

And we see the opportunities around AI as being much broader than that, around really, again, how do you share and transmit knowledge across the workspace in Notion? And I think more broadly, we're going to see a real transformation in the way that knowledge work is done. And there are tedious aspects of this that we have to do today ourselves that we think AI can help a lot with. So one thing I can share here is that we are excited about some major improvements and a much more cohesive experience around how you manage projects in Notion.

And some of that's going to be coming very soon. And it's going to be very much, I would argue, the first AI-enabled experience of how to do project and task management. And I think there's some really exciting things we're going to be able to do with that. It's going to depend on some stack and infrastructure that we haven't shared yet. But it's a really exciting area for us. And I think it's a big part of the value proposition of what makes Notion different and unique, which is that all of this is happening in one place.

And maybe the kind of philosophical thought I'll leave you with there is, one of the most interesting, most transformative use cases, I think, of generative AI and large language models is the ability to bridge between structured and unstructured information much more easily. And that's right in our wheelhouse because Notion ideally is a place where you both are storing a lot of structured information and unstructured information. In fact, the mere concept of a Notion database was the idea, and the kind of innovation of it in part was you have structured information in terms of the fields in that row of the database.

But also each item in that database is also a page that can have tons of unstructured content in it. And so I think there's going to be some really interesting things we're going to be able to do there. It is going to depend on some of the capabilities from some of the vendors, the types of vendors and companies that spoke earlier today. I can't say I'm working with them specifically, but those technologies are all relevant for that process and that development.

Thanks, Daniel. You might recall Evernote got some negative attention due to them changing their terms of service to let employees and data scientists look at data for their modeling. How have you thought about the privacy and proprietary information, especially if you're trying to use third parties as well for some of your AI functionality? Yeah, that's a great question. The privacy and security side of this is a really important one, and it's something we take very seriously.

And it's also something that I think this entire industry is grappling with right now as well. So for the first part of this, are internal people at Notion looking at your data for these use cases? The answer to that question is just no, unless you're opting into specific things we can do with logging. So we do not actually look at pieces of content from external users' workspaces. And the good news for us is actually we were able to go a long way often in testing out new AI capabilities with our own internal Notion environment.

So we do that quite a bit. In terms of the third-party part, it's a great question. So first off, today, users and enterprise customers can completely opt out of that experience altogether. Second, we have agreements with our partners that they are not using that data to train models, et cetera. You can turn off Notion AI for an entire workspace if you want to do that. To your point, I think this is one of these areas where we're going to be figuring out this industry in the next few years, and probably even sooner.

Obviously, there are advantages to open-source models in the sense that we stood those up ourselves. Now you're not thinking about subprocessors and other third parties having access to that information. So it's definitely something we are focused on, our legal team is focused on, and we want to be transparent about where your data is going. And so today, we have a limited number of providers we're working with, and we are only sending content to them when you are choosing to use Notion AI.

As we start to see use cases potentially that are sending more data from a user's workspace, from a company or a user's workspace, to these third-party providers, obviously we want to have a way for people to make a decision about whether they want that. Hi. So you mentioned practical applications. Just a fun thought exercise for you: if there was one thing that you could build, practical-wise, with generative AI, what would it be? There's one thing I could build for Notion or just generally?

In general, you personally. Oh, that's an interesting question. Trying to think of what comes to mind first. I think for me personally, I would love to have a way—I have two young kids—so there's some examples for me that come to mind with my kids where I would love to be able to create more personalization around how they learn. So personalized learning, I think, is something I'm excited about. I'm very curious about how this is going to transform the education space because I have some kids at various ages entering school.

And so that's one personal use case that I'm interested in. I think for Notion more generally, I'm really excited about the ways that AI is going to make the development of ways that we collaborate and use knowledge inside of our Notion workspaces, is going to kind of take away a lot of the tedious parts of doing that and really focus on humans doing the parts that humans do best. So, for example, what's really important in a team is: how do we work together collaboratively?

How do we resolve conflict? How do we make progress together? And there are a lot of things we have to do along the way there that we have to jump in and work on ourselves that are quite tedious. And I think AI can help a lot with a lot of those use cases and help us focus more on those human parts as well.

Yeah. Thanks for having me. Thanks for listening to The MAD Podcast. If you liked this episode, be sure to leave us a review. firstmark.com/events/data-driven.