Scott Belsky: AI & Creativity

When people think about huge public companies making big bets on AI, people tend to think of NVIDIA, maybe Microsoft, but I think that the world is very quickly waking up to the fact that Adobe has been doing incredible work at the intersection between AI and creative work. So maybe walk us through some of the stuff. So there's Firefly, there's Sensei. How does that all work? Yeah, well, I mean, to take a step back, over the last six years, the lowest-hanging fruit for us to grow our business and make our customers happy has just been to help them be successful in our product.

If you've ever opened up Premiere Pro or After Effects or Photoshop and you see this cockpit of 40 years of features, in some instances, it's overwhelming. And a lot of the work over the years has been improving the first-mile experience and all that kind of stuff. But the dream scenario is if you could actually just automagically say things and have them done for you, right? Talk about the onboarding. In the ultimate, if product-led growth is people feeling successful more quickly in your product, AI is such an unlock for us.

So that's one of the obvious reasons why AI is impacting our business so much, is that for all the people that download any of these desktop products like Photoshop every day, the number of people that get lost or can't feel successful in the product is really significant. And if we can just help those folks, one of the most important insights was, and maybe as important as building a great family of models—and I can talk more about Firefly and the underlying models and how we trained it and how we didn't train it and everything else—

But it was simply the insight of what we call the context bar that shows up by default in the product. So you open up Photoshop and literally you see a blank page and you see this context bar whenever you click anywhere, and it prompts you to generate something. And so you can generate a landscape and then boom, suddenly you're like, oh my gosh, I'm using Photoshop. So if you go through it, you see why this has been such an unlock. But of course, for Adobe, AI is also about empowering creatives to do much more.

And we can talk more about some of the tactics. Yeah, since you kindly offered, let's dive into Firefly. So Firefly was announced in March of this year. Yeah. So what did you do? We first teased the Firefly capabilities without the brand last October. So not this October, but last October. And we kind of showed some of these generative AI capabilities that we were building. And at the time, I was chief product officer, and I had one of my VPs whose side thing was always exploring these things.

And I was always sort of not sure, should I have him do his job, or should I really support him doing all this rogue on the side? Of course, he's now one of the leaders of this entire effort. So that's a management lesson: really follow your people. Certainly lead them; don't lead them, follow them in instances where they're so talented. But from that moment forward, it was all about, okay, let's make this real. And internally, it was around making sure that we had really competitive models.

Now, one of the biggest choice points for us is how we were going to train this. And of course, at the time, compared to Midjourney and a lot of others out there that really scraped the internet, ironically, ultimately scraping Behance. I mean, we had built a platform of 50 million creative profiles with projects that are immaculately tagged, and it's like the perfect scraping source for a generative AI model, right, for creativity. But we weren't going to train off of that.

We wanted to train off of licensed material that we had a license to train off of. And this was not only a moral obligation to our own customers, but also when we went to our customers on the marketing side, and I had meetings with some of the biggest brands and also advertising agencies, and I would talk to them about GenAI and I would show them these demos, they were all nervous. I was like, guys, you have to use this stuff.

Like, this is transformational. And they're like, we would never use it for an ad because we can't attest to how the content was trained. And a light bulb went on, and we were like, wow, we have to be the commercially viable model. So when we launched Firefly in March, immediately, of course, so many people started comparing us to some of these other models out there. And we were really happy with what we had launched from a quality perspective.

But people were putting in things like Spider-Man, and you get Spider-Man, and then from us you'd get a spider that looks like a man. And people would be like, oh my God, this sucks. Like, Spider-Man, they can't even get Spider-Man. And I would always try to reply as fervently as I could, that's a feature, not a bug. That shows you that we're not training off of content that we don't have a license for. And are you adding to the training sets now, like with licensed datasets?

Yeah, so we're constantly adding to it. I have a whole team under me that is focused on content ingestion and curation. And then also we're doing a lot of work now with our customers to make what we call tune-your-own-model, where whether, let's say you're Lucasfilm at Disney, can you take all of your collateral for the brand and make a tuned model of Firefly that is commercially safe with content that you have a license for? It obviously wouldn't be leveraged by other customers that are not within your organization.

And those are your own models, you said? Is it completely homegrown? Yeah. So all of our generative AI models for Firefly are homegrown. And then we have partnerships on the LLM side for some of the LLM-powered capabilities across our products. But we realized that not only did we have to become best in class in how these models were trained, but also we have a lot of information from how customers use Photoshop and Lightroom and the tendencies of photographers and how they make their images better or improve their videos.

I mean, that is a lot of usage data. It's not using the customer's work, but it's using the way they use the product that we can also use to actually make the models better. Great. So that's Firefly, which is the image generator. What is Adobe Sensei? Yes, Adobe Sensei actually is 10 years old, and this is our AI effort overall. Internally, Sensei is the research team and the AI development team. But over the years, we've had all sorts of technologies.

It's really like the first—we had neural filters three years ago in Photoshop where you can kind of, with just a dial, change someone's facial expressions or things like that. And those are all AI as well. One of the moments where this all really hit me, though, and the power of it and also the problems with it, and I could talk about some of the work we've been doing around fake media and a lot of that stuff because we've been at the head of a lot of this stuff and we're working with a lot of the governments on it.

I was sitting in a room like four or five years ago, and I was getting a demo from the After Effects team of a tool called Content-Aware Fill, which was an AI tool from the Sensei team. And Content-Aware Fill allows you to remove an object or a person from video. Now, you could always do that. It just took a long time to go frame by frame, removing someone and the dust and their shadow and the footsteps and everything, right? And now you could literally just select and mask an object, click a button, wait like 30 seconds, and then boom, this person is just gone.

And the team was so excited about this. Our customers are gonna love this. And I'm like, wow, this is amazing. But oh my goodness, what is our responsibility to make sure we ship this and make sure that people can tell what's real and what's not real? And that was the time where we founded a nonprofit effort called the Content Authenticity Initiative. And we participated in the founding of an underlying format for this called C2PA, almost like Unicode, like a consortium that everyone aligns around.

And we have now something called Content Credentials. And so whenever you use AI in any of our products, we automatically add credentials to the asset that says what tool was used and what model you used. All these things, I think, are going to matter a lot in the future, and we can talk more about that later if you want. Yeah, actually, that's super interesting. Let's double-click on this. So, how does that work? That checks work against an existing database?

No. So what we realized is this: we're never gonna catch the bad actors because anyone can make anything, and it's a cat-and-mouse game to have algorithms that tell whether something was edited or not. What we can do is really double down on provenance for assets and then help people be trusted. I like to say that the new "trust but verify" is "verify, then trust." We're gonna see content thrown at us constantly in our life, and we're not gonna know what's real and what's not.

And if it has a credential mark and you can click on it and see the photographer with a certain Nikon camera that then edited it in Lightroom with these types of edits, then you can say, oh, I feel comfortable trusting that piece of media from the BBC or whatever it is. And so, fast-forward, we now have 2,000 partners. Nikon, Canon, Sony, and Leica have all come out with cameras that now add Content Credentials to assets by default taken in their camera, so that when they come into Photoshop or Lightroom or third-party products of other companies, the provenance can be tracked and added cryptographically with metadata.

Fascinating. All right, so Firefly, Sensei, you had a bunch of big announcements maybe a month ago, rolling out more AI across products. Can you talk to that? Yeah, sure. So we came out with the very creatively named Firefly Image Model 2, which is so much better than 1. New and improved. New and improved. And it's really very high fidelity. I think that, in one kind of research, with nine vectors of comparison with some of the other market-leading models, we were better than many of them on seven out of the nine.

Again, we won't give you Spider-Man, but we will give you a lot of this other stuff. We also came out with our vector model, which is kind of mind-blowing. I mean, you can be an illustrator and just say, "I want a lion," and it will give you a fully vectorized lion with all the endpoints and everything. You can say, "I need an arrow," and it just gives you tons of variations of arrows to choose from. So I think that the vector landscape and illustration as a whole was really changed forever in October when we launched that.

We also launched another kind of spicy feature called Generative Match. What this allows you to do is take any piece of work that you have access to, and we tell you you should have a license to it, and then you can upload that piece of work, and then anything you generate matches the style of that work. Another great example. I saw this demo and, like many others, was like, oh, this could be used in nefarious ways. Like, how do we do this right?

And so what we decided to do is say, number one, we're gonna add Content Credentials that show that you used Generative Match. So we know that a style was referenced. And number two is we're actually gonna save the image that you uploaded so that there's always provenance of the asset that you used to inspire the image that you made. So again, we're trying to find ways to do this the right way, but also make sure that the technology drives us forward.

Anything in the labs or roadmap you can talk about? I know you have this concept of sneaks where you can see some of the stuff that's brewing. Yes, we did show some of our generative video stuff and 3D stuff, but one of the things I'm most excited about is we did a sneak of something that we call just the Firefly Editor. Very creatively named. But the Firefly Editor is pretty wild because you have an image, and it's an image of a family, right?

And so you could always select the five-year-old boy, but if you move him, well, there's gonna be a blank spot behind him, right? But what this does is it actually allows you to select objects and move them, and it generates all the pixels behind whatever you're moving in real time to fill in what would be there. And so it's like this crazy—you feel like everything is layers, but you captured every aspect of the image in endless layers because you can just do anything with it in real time.

And it's one of those experiments where we were like, wow, this is going to change image editing forever. People are just going to move their kids around in family photos, and it's going to be effortless. It's intensive from a generation perspective, but it's the future. So, taking a step back from products, almost at an organization level, Adobe, as you mentioned earlier, has been doing machine learning and AI for a while. Actually, as I was prepping for this, I read that it goes back 20 years, like just building OCR technology, which was AI at the time, into products.

But equally, Adobe is a $275 billion market cap company, huge company. How did it happen that the organization seemed to have moved so quickly in a few months, building so many AI features into the products? Well, if you looked under the hood when we were building the Firefly models and also the Firefly Playground, we surfaced these capabilities first in one simple website before actually bringing them into a lot of our products. And it was a great way for the team to just ship really fast and for us to get customers' hands on them.

And we had a mantra internally that we were not gonna announce Firefly until people could literally use it that day. That was very important for us. The other thing I would have to say is that big companies with program managers and product managers reporting to other product managers with two biweekly meetings, I mean, it just can get so big. And in this instance, we just had created a small design organization within the larger design organization that we just tasked on interface development for these generative capabilities.

And then we paired them directly with the engineers who were building these models, and we were just starting to prototype rapidly and just start to feel it out. It was a really important thing for us to do. That contextual task bar, for example, really made this work in Photoshop. I feel like if we had tucked it somewhere in some menu, it would never have had the success that it's had for Photoshop as a franchise. That contextual task bar actually was something that the designers put in our Illustrator on the iPad product because they were trying to figure out a way to have Illustrator on the iPad.

Without all the chrome, without all the buttons and all the distraction. And so the insight of this contextual menu came up, and we launched that with that product, which was a good product. It's not like a sensationally blowout successful product, but that insight, that design pattern, we just reused to solve this problem. And it was like the absolute most important thing to do for Photoshop. You alluded to some of this, but organizationally, so there was that team working with engineers, but are the AI folks and the data folks distributed across the organization, or is there a central kind of AI research team?

Yeah, so we centralized the research team, which includes all the AI and data scientists that are involved with Firefly. You mentioned some outside vendors and LLMs. How do you think about build versus buy? On the LLM side, people are now talking about multimodal models, and that's something that we're exploring as well, which would include kind of all of the above. But I don't think that any company should build everything themselves. You have to consolidate your resources to be great in the things that matter most.

So, for the example of LLMs, I mean, we have a great relationship with Microsoft and with OpenAI and with NVIDIA and with Google, and we have a number of partnerships leveraging some of their stuff. My strategy for the team has been: let's do everything that is our competitive advantage ourselves, but the stuff that is not our competitive advantage, we need to partner. And that's been the drive. Great. All right. So moving on from the Adobe hat and putting on your thinker and writer hat.

I love all the content you've created and stuff you've written about the intersection of AI and creativity. Maybe give us some summary of some of the thoughts. What do you view as the future of how we work and how that gets impacted by AI? Yeah. Well, I've had a lot of discussions, as you can imagine, over the last year or so of people asking, "What does this mean for the creative industry? What does this mean for advertising?" And we're entering a world where, first of all, every brand's going to absolutely flood the zone with content.

That's going to make the bar go up for the digital experiences that really engage us. Whenever you talk to any great creative and you ask them how they come up with a great solution to a problem, they always say it's a function of time. I mean, as you give them more cycles, they'll come up with more possibilities, and then they'll find the better three to present to their client at the end and get a choice. So if you give them 2x the time, they'll do 2x the exploration of the surface area of possibility, and they'll come up with better solutions.

AI basically truncates the time. So what we're seeing in our products is that instead of an illustrator testing three color palettes for packaging for a perfume product, they'll test 30, and then they'll have so many better choices to choose from. So I think that AI is going to raise the bar. Now, when you get more capacity out of a creative professional, does that mean you need fewer of them, or does that mean you actually want more of them? Well, in the last 20 years, engineers have gotten a lot more productive every single year, and yet we keep wanting more.

And I think the reason is because we want to make more. We made two products with 20 engineers, but now they're more efficient, and so they can continue to do those two products, but they can do a third product. But since we get more ingenuity and performance per person, let's have three more people so we can do five products. It's like this capitalism-driven desire to do more, to do more marketing, to do more content, to do more formats and more segments and everything.

So in our data, we're still seeing this revolution drive hiring more as opposed to reducing the hiring. Now people are doing different things. The job of someone turning squares into rectangles all day for banner ads, that job's been replaced by AI. But guess what? That person is really thrilled. They don't have to do that anymore. They never really liked doing that in the first place. So the bar goes up. I also think that we lower the floor, though, and a lot more people are getting into the fold of creativity and marketing.

And it's interesting, you look at a big organization that does marketing for their products. Increasingly, they're doing more of their marketing spend on social, a significant amount, maybe 50% plus. And that social content, they're realizing, performs better if it's personalized, if it's culturalized, if it's made to be of the moment, if it's responding to a meme, maybe one of Matt's amazing tweets. If it's in the moment, it'll be more performant. And so they're realizing they can't rely on the old school, okay, there's a marketing brief and it goes to the agency, which goes to the designers, which goes to the account managers, and then we meet three weeks from now.

This stuff has to happen now, right? And so to do that, you need to outfit people to be creative without having the skills. And so that's another sort of, yes, creative professionals are doing better stuff, but also non-creative professionals are starting to get into the fold with this technology. So that's the future, is personalization at scale? Exactly.

That's my bet. This concept in one of your talks that I really liked, which was AI is eliminating work in the word workflow, removing work and keeping the flow. What does that mean? Well, it's kind of Mihaly Csikszentmihalyi, the guy who wrote that book, Flow, the iconic book that is all about creativity as really a state of flow. And creators, and probably all of us to some extent, are constantly fighting friction every day to just get our ideas out. And what gets in the way of us getting ideas out is the friction of the work.

Right? The work in the workflow, having to open something and find that place and then drafting it out and then all the meticulous stuff you have to do in Photoshop, for example, that's redundant, mundane, repetitive work to get something done. And so if you can use technology like AI to reduce a lot of that, I think what it does is it unlocks more flow. And that's really appealing to a creative. But I think for all of us, to be able to just have less friction from the thought and the insight into the expression of it is a big opportunity for humanity.

You also wrote about the impact of AI on business models. So some business models are going to go away with AI? Yeah. Well, I think about this a lot because a lot of business models still today are all time-based. So you're paying a lawyer hourly, you're paying a designer hourly sometimes still. And so something's wrong here because these great, talented people are getting access to tools like we're discussing all evening, and they're inherently far more productive. But should they be paid less as a result?

And so I'm starting to wonder if value-based pricing models are really going to replace time-based pricing models in a lot of the functions of very sophisticated work. And also, what are the tools that allow us to know how to price our work? Right. But all of that is going to have to change because this hourly model in the age of AI simply is broken. Now, with your investor hat on, I liked one of your tweets where you said you never want to be first to market in a Cambrian explosion.

I often wonder what amazing iconic companies are the Prodigy or the CompuServe of the AI era. Yeah. And by the way, Prodigy and CompuServe were successful companies, right? But they didn't last. I'm oftentimes asking myself, what are those companies? I mean, we're all working in this moment where we had these great ideas, and then three weeks later there's some general API that makes that possible for everyone. And then three weeks after that, the world is just inundated with audio translation apps.

And it's like, oh my goodness, everyone—there's just dozens and dozens, if not hundreds. And a lot of people are just sort of, in this sense, flooding the zone with apps. They're just like, I'll just capture some degree of the TAM for people who want to translate. And you see all these tweetstorms about people building $300,000 ARR businesses working alone, which is exciting, but that shows you just how fast the pace this is moving. And it's unclear—that's a technology that a year ago would have been, like, breakthrough, raising huge amounts of capital from top-tier VC firms.

And today it's just, like, a redundant thing that everyone can do. Not to say that there aren't quality differences between some of them. I'm trying to think about what are the moats, but also what are the areas where there was actually no great technology solution at all? One of my favorite investments is this company called KoBold, and they use AI to crunch a ton of data to decide where to excavate for minerals. And as I did a deep dive in that space with the team, I learned that these huge companies like BHP and others, these massive, massive excavation companies, they do very little R&D now for a whole sort of other reasons.

And also, their teams that do new sites and stuff, they've been in their jobs for 20, 30 years. So they have a way of doing things, and they're certainly not leveraging AI models with all this sophisticated spectrum of data to determine the likelihood of mineral deposits in certain places. And so it was a great example of, I think, an application of AI. And I don't think that that's going to be an OpenAI API. So I think you look for things like that.

I think you look for certainly some data moats, whether it's in where there's, like, a network effect in the usage of the product. So, for example, some companies that are in the legal space where the more the product is used, the better the outcome, the more the product is used. So I love those sort of flywheel-like businesses as well. But I'm not sure what are those Prodigy and CompuServe going to be?

Probably some of the companies we love today. Would you play it back? Like vertical applications of AI, you think, are so KoBold-style? You think that's a rich vein? I think the verticals will be important. I think the LLMs will be abundant. And I actually think at some point, maybe relatively soon, we'll realize that some LLMs, even locally running LLMs on our machines, are enough for a lot of the actions that we want to do every day. And so that's, like, maybe a contrarian view in the sense that it's always about GPT-5 and then GPT-6.

And maybe at some point, though, we're going to want to route things locally to do certain things, and that will change the landscape as well. Have you invested at all at that either foundation model layer or middle layer of orchestration tools and that type of stuff? Is that interesting to you? Yeah, I do like the picks-and-shovels stuff. I think that the AI developer workflow on a daily basis is definitely different. The role of data analysts. I'm an investor in this company called Lightning, where it's sort of a console to be able to test what you're doing across different sorts of models and different sorts of stacks and that kind of thing.

So those are kind of developer tools that are interesting to me. How do you evaluate an investment, especially at the pre-seed or seed stage where you're most active? Like, what's a good AI entrepreneur from your perspective? Well, I think a very good AI entrepreneur is very skeptical and somewhat paranoid of their own ability to stay differentiated. I worry when founders seem to be either naive or not wanting to face the fact that that could actually just be a feature, a ubiquitous feature.

I love it when folks are like, well, we're not going to do just that because that's not special. Like, I mean, it is now, but it won't be in a few months. We all know that. I think it's a very refreshing characteristic or trait that you need to have right now. There's a lot of passion for solution right now, which in my experience always loses to empathy with customer. So too many of us start businesses, and I've made this mistake in my experience with projects, where it's like, so passionate about a solution, and at the end you're like 30 degrees off of product-market fit and you wonder why.

And it's because you didn't spend time shoulder to shoulder with the customer, realizing the dirty little truths in enterprise. People want to look good to their bosses. People are lazy, vain, and selfish oftentimes. They want to—I mean, there's a lot of practicalities that actually impact product-market fit. And so unless you have real empathy with that problem you're trying to solve and that customer, you're liable to probably build something that might be extraordinary, but just maybe off. One last question from me because I want to leave time for folks.

As an investor and entrepreneur, it's been a really interesting year, as in 2023, because on the one hand you got all this craziness around AI, and it's all exciting and amazing for all the right reasons. At the same time, it's been a super hard year for most entrepreneurs. You're a specialist of the messy middle. When you talk to people you've invested in or friends in the community, what do you tell them? I think that the question I'm getting most often these days, probably from companies that I invested in late 2021, is: should I quit or should I stick with it?

I think that a lot has changed in a lot of these businesses. Like, the whole environment has changed, underlying technologies have changed, the willingness to pay in some cases has changed for the customer. And so that's a really tough question. And I always try to do an exercise with these founders where I say, first of all, you had a vision for what the world needed to be in the beginning when you knew nothing, and you had high conviction, and now you've learned a lot.

Do you have as much conviction in the end state of what this world needs to be or not? And have a real honest conversation about that. If the answer is unequivocally, like, I have so much conviction that the world needs to be this way, and we just talk about the problems and the fits and starts, I remind them they are just in the messy middle. And the competitive advantage of most startups is simply sticking together long enough to figure it out.

And you just need to maybe iterate and try a different approach. That is just par for the course. However, if it comes out, and it often does, that they've lost conviction based on all they've learned and all that's happened, I'm like, there's no pride in continuing to do something that you've lost conviction in. And in fact, I've never seen that work. You don't just by accident hit it big with a product. And so pivot, do something different, shut it down, call it a day, and just regroup, open the aperture again.

I think a lot of us are at that moment. Ultimately, it's a conviction test. And I think that we should always be doing that with ourselves because it helps us have that compass of what we should do next. Wonderful. Thank you so much for sharing.