Glean’s Breakthrough: CEO Arvind Jain on Scaling AI Agents & Search

Thank you for having me.

I think all indications are that there's going to be a very large number of models that are going to be available to us in the industry. It's going to be both closed models from companies like OpenAI and Anthropic and Google, but also there's going to be a lot of open-source models that are like, for example, Llama, and then a lot of derivatives as well that are going to be available to us. So that is already happening. It's just going to be more and more of that in the future.

What it means for us is we will be in this world where it's going to be hard to actually always keep up with the model technology, for all of us. I think it'll become increasingly more of a deeper infrastructure-level thing that software developers actually make sense of and figure out, for a given application, for a given use case, what's the right model technology to use. And hopefully for enterprises, for customers, the LLMs become a technology that they don't really think about. It's just something in the background.

For most customer use cases, the companies are using the closed models at the moment, at least the ones that we are familiar with, as we go and work with large enterprises and we think about their core business processes that they want to automate with AI. Right now, the closed models are the ones that tend to dominate. And I think it's for the right reason. As you're in the early stages of the journey and you're trying to build an application where you're going to actually use AI as a key component, this is not the time for you to optimize and figure out what's the best model or what's the most cost-effective model because you're building an application and nobody uses it.

So it doesn't matter how much it costs. First, go and use the best, the state-of-the-art model, which today are still the closed models. And so, for that reason, you're seeing that largely people are starting with these larger models from OpenAI or Anthropic or Google. But as things reach scale, let's say you have an application that now has millions of users and hundreds of millions of daily interactions, now cost becomes a real factor. And that's when open-source models start to shine because you can actually take the LLM technology and make it work very specifically for your application, for your use case.

You can distill a model which is much smaller. It becomes not only a lot faster for you, it becomes hopefully a bit more accurate, but most importantly, it becomes much cheaper for you to run. And so that's sort of how we see the transition happen. Initially, you start your exercise, you work with the state of the art, the largest models, and then you start to fine-tune and distill to the right model as you start to get to scale.

Well, for certain businesses, there are actually real regulations and restrictions where you cannot send your data out. Having everything run in an air-gapped environment is actually a must. So for those, you have no choice but to work with open-source models. So that's one driving factor for it. Another one is cost. As I mentioned, when you have a large-scale application, you have to actually look at the needs that you have from LLMs and use very specific models for them, because they're going to be both better for user experience because they become faster for you.

They will hallucinate less for you because they are trained to do a very specific thing. So their range of errors actually also reduces. And then finally, they're more cost-effective. But there's another reason also, like the third reason that I would say why people actually want to work with open-source models is control. You feel, number one, you don't want to have dependency for your core business processes on other companies sometimes. And you want to actually, for that reason, run your own models.

You also feel like over time you'll be able to customize them better and better if you have the expertise to do that, that you won't be able to do if you were working with a closed model.

Well, the more LLM companies that are out there, the better it is for the industry. So we actually like seeing new names. We like seeing new models. And in fact, we benefit from them today in a big way. There's so many model choices that we have today compared to last year, models that are of comparable quality, models that are all getting better at certain things. For example, today, for a lot of coding use cases, people tend to use Claude a lot more.

For reasoning, they're using GPT. And so you're seeing these models get specialized to specific skills. And so, for application companies like ours, where we're trying to use this innovation and solve real business problems, the more innovation that we see, the better it is for us. And not just from a point of view that we have more technology to use, but also it creates that competition and it ensures that we can actually get the LLM advances in a more cost-effective manner that we can then actually bring our products in a more cost-efficient manner to our customers.

Well, I think for me, an agent is simply an application. It's a unit of work that used to be done by a human before. Now it's being done with the help of AI. So typically, any agent, like you think of, is going to work on some piece of information, some data, some knowledge, which can be brought from the public internet or the web, or it could actually be brought from within your company's systems. The agent works on this data. It has a task to perform.

It uses LLMs to actually reason and perform those tasks. There may be steps in this agent to actually self-reflect so that it can actually do the task better. And ultimately, it's going to produce some work, which is going to get saved in, again, maybe some of your internal enterprise systems. So an agent is nothing more than an application. We're all used to what software applications are. It's just that AI is being used in a fundamental way inside to do a lot of this work.

And so that's how we think of agents. They come in wide shapes or forms. Like today, everything from a simple workflow to a very deep, complex application, people are choosing to use the word agent to describe all of them.

So ultimately, an agent is automating a business process which was before managed by a human. And so that's sort of like, I'm talking in the context of agents in businesses. And of course, there are agents that you can be using in your personal lives as well. But for a moment, let's stick to business agents. So these are agents that are actually taking a specific business process and transforming them and automating them using AI. So, take some examples. You could actually, in your legal team, there's a process, like when you get third-party contracts, you have to redline them, make sure they conform to your company's guidelines.

So for that, today, you have somebody in the legal team, a lawyer who's actually reading these long documents, and they're going to redline it and pick all the places in the document where you need to actually change it to conform to your company's guidelines. And so this process, it requires a human. It requires them to actually go through a long document. They read it, they know what is the standard to which they're trying to actually align this document.

And you can now go and replace this whole process with an agent, an agent that actually automatically redlines your third-party contracts. And the way you build this agent is that you actually tell it exactly the process that human was following before, which is that, well, read the document, compare it with your version of the document, and then go and highlight or redline the problem areas in the document. And AI is pretty good at reading instructions, reading documents, and is able to actually go and now do this thing automatically for you.

And you can imagine that for many business processes today. Let's take a few more examples. Take customer service. One of the key activities there is customer service agents resolve tickets. And to resolve tickets, they're reading knowledge bases inside your company, looking for where the answers to these questions could be, and then use that to actually respond back to the customer. Again, AI can actually come and do that work. It can search over all of that information, understand the question, and answer that question automatically for the end user.

So, you can start to imagine that all these business processes that you have, you can automate them with AI. And this has been a desire of the industry for a long time. Before the AI wave, you had robotic process automation. It was a big thing. Now you can think of AI agents as just a much more advanced version of RPA. And there's a lot of business value from it. You can actually double the efficiency of every person in your company with it.

And that's why there's so much excitement. And it's actually like, the thing that's happening is that when you think about AI in this specific way—that you're going to build agents and you're going to automate your business processes—now it's super appealing to you because you can see direct ROI impact from building these agents, which is what is causing all the excitement in the industry today.

We are in very, very early stages of this journey. If you think about agents, theoretically, an AI agent is promising you to do any piece of work. In fact, that's how, when we talk about even our product, Glean, the way we describe Glean to our customers is that you can come to Glean and you can ask any question to it, or you can actually give it any task, and Glean will use all of the world's knowledge as well as all of your internal company data and knowledge to answer those questions or complete those tasks for you.

Now, when I say that, I'm actually making this assertion that I can do everything in the world. And that's not the reality. Today, agents are still quite basic, I would say. They all still need a significant amount of supervision. I would think of agents as more being, well, I think I have a piece of task, I still own it as a human, but an agent can potentially do it for me. But it'll do its work.

I'm going to still go and review it. We barely see any application where customers are running agents in a fully unattended, unsupervised setting. So it's early, but the impact is actually very clear. But remember this thing: even if human supervision is required, that doesn't make the agent worthless. That same example I was giving you before about a legal person reviewing a 100-page contract and redlining it, that was a two-week activity that if you actually get an AI agent to redline it for you, it's going to do it in one minute.

And it won't be perfect, but now I can go in and look at the work that AI did and potentially spend like two hours instead of a week to actually go and redline that. And that's more than 90% savings for me. So I think that's sort of where things are today, that there is real value being delivered to customers, but agents are still better run in a supervised manner where a human is in charge and looking at the work of the agent.

I would say it was not so much a vision or a realization that this was the right time to solve a problem. It was just personal frustration that I had, which forced me to start this company. Finding information at work has become increasingly difficult over the years because, well, number one, there's so much knowledge that we have in our companies these days. We live in a data-driven world. Number two, that information has increasingly gotten fragmented as we went through this SaaS revolution.

Every business has ended up with hundreds of these cloud-based applications because buying applications is so simple now. So businesses will end up with hundreds or sometimes even 1,000-plus applications, and all of your company's data and knowledge is spread across all of those systems. And my company, before we started Glean, was one of those companies. We built in the modern SaaS world with tons and tons of information and data with 300 different applications, and nobody in our company could ever find anything.

That's right. So it was a personal problem, pain that I had. And I did feel like this problem had to be solved because it was not unique to us. Every person who I talked to, they would all say the same thing: that it's hard to find information. It's hard to get the answer that you need to do your work. And even before Rubrik, I was used to at Google, and ironically, we were making it easy for everybody in the world to get answers to their questions, but not helping ourselves internally.

Inside Google also, it was really, really hard to find information. But in 2019, there were a few key technical trends which actually made this problem more tractable, or actually more acute. The same thing, like I talked about how SaaS fragmented your information so much, making it hard to find. But SaaS also allowed you to actually tap into enterprise data and knowledge more easily because we have open APIs. And so, if you're building a search engine, it's much easier to build it now because you can actually tap into these standard APIs to these products and actually get that enterprise content all in one place and then make it searchable.

But the second big trend that we thought was super powerful was transformers. So it's 2019, and nobody as such is really talking about transformers and language models. This was still largely within the domain of search. Like, in Google, people were using these models to try to actually advance Google Search. And our team, me included, most of us actually came from Google, and we were seeing this big impact that the transformer technology was having on search. Suddenly, you could move from that keyword-based word-matching technology to find the right information to conceptually understanding questions and knowledge and doing the matching at that semantic level.

So we saw a big opportunity and we decided to actually use that to build a really good enterprise search. So that's the origin of how Glean got started.

That's right. So these models kept getting better over the years. I think for us, using transformers in 2019 made us one of the first companies to bring LLMs to the enterprise, but we didn't necessarily have the foresight. I should correct myself. We didn't have the foresight, and we didn't know how fast these models would get better. In 2019, they were good at understanding content, but they're not good at generating. They're not good at reasoning.

And these capabilities started to come over the years. By 2023, we were seeing this amazing ability for these models to actually generate their own answers to questions that people have. And we felt that actually allowed us to really advance our product in a significant way. So we started out with being the Google for your workplace, and now we could actually become the ChatGPT for your work environment. When people come and ask questions in Glean, instead of just surfacing the right information back to them, which we think is most relevant, we could actually make AI read that information and generate its own precise answers so that you could save our users even more time.

So we saw that opportunity naturally as the models got better, and we transitioned and launched our Glean Assistant, the product that you can think of as a more powerful version of ChatGPT inside your company.

Exactly.

So our journey with agents started early last year when we launched what we used to call Glean Apps and Glean Advanced Prompts. The idea behind agents for us was that we had already built these deep integrations into all of our enterprise applications. We could read the knowledge inside those systems, but we could also take actions within those applications. And this was a core part of our assistant product. So when you come into Glean Assistant, you could actually say, hey, do I have the following benefit?

Can I actually avail of this benefit or that? But you could also actually do things. You could say, hey, can you file a PTO for me? And so we are seeing a lot of demand from our customers wanting to take actions and do some work while they're within that Glean conversational interface. And so we started to build support for actions into these enterprise applications. And so this is where we were. We still are only a one-product company. It's this conversational chatbot like ChatGPT, but that knows everything about your company.

But this platform that we built was actually quite powerful, and we started to get a lot of demand from our customers. They wanted to do some very specific things with it. They didn't want this general-purpose assistant that is connected to all of the world's knowledge and the company's knowledge. They wanted to say, look, I want to actually build an HR first responder. This HR first responder application actually does only a few specific things. It answers people's questions.

If they are on human resources-related topics, and it can actually resolve some of their tickets using automation. And so our customers were saying, look, allow me to build this one specific application, which does this very specific thing, and which can actually take some additional instruction from me on how to do this piece of work. And so we launched this concept of applications last year that allowed you to do exactly that: work on a specific set of knowledge, solve specific tasks using AI.

And then over the year, we got a lot of customers using our app platform. And now we come into 2025, and everybody's talking about agents. So the first thing that we did was, in February, we renamed our applications product. What we used to call AI applications are now called AI agents. So we don't confuse the market, we have to sort of adopt the terminology that is becoming more standard in the market.

But agents are now getting a lot more powerful. They're shifting from basic two-step RAG kind of application flow, where you take a task, you find some information, and then you make AI work on it to generate the right artifact. You're moving from that to actually running very, very complex business processes. There could be a hundred-step business process that you want to automate with AI now. And so the new version of agents that people expect now in the industry are significantly more complicated.

They have things like evaluations, insights, self-reflection, and things like those built natively into the core agent functionality.

Okay.

Since we started as a search product, where our objective was to help people find any piece of information that they're looking for inside the company, we built this horizontal platform. We built integrations to hundreds of enterprise applications, and we look at data and information inside each one of those systems so that when somebody's looking for something, we are able to immediately point them to the right pieces of information. So we built the platform before knowing that it would be used to build AI agents.

But now that we have this platform, which is connected to your Salesforce and your SharePoint and Workday and Google Drive and all these different systems—if you think about agents fundamentally, agents are working on some of your enterprise information and applying AI to perform some business logic, and then maybe saving the resulting work that the agent did back into your enterprise applications. So we felt that agents in general all tend to have these common elements. And the Glean platform actually solves for all of those because now, whether you're building an agent for an HR team or finance or legal, all of those systems that those teams use are already connected to Glean.

So that allows us to actually help our customers build agents across these functional use cases. So it's sort of like the reason why we started with the horizontal approach was just timing, because the first product that we built was search. But it has proven to be a very popular thing with our customers because today, one of the challenges that enterprises feel with AI is that there's so many vendors, there's so many products out there, and how many new products is an enterprise going to buy?

And I think the fact that Glean is this one horizontal platform that can actually do a lot for you is appealing to them, that they're able to try out AI across their different functional teams with just one purchase.

Maybe first, let's understand the overall architecture of the product. So the first thing that you have to do to build a search product is you have to get hold of the information that you want to actually search over, which means in an enterprise, you have to build integrations with all the systems that a business uses. So that's the first part of our Cortex stack. We build these connectors and deep integrations into products like Confluence, Jira, Workday, Google Drive, and Slack, and so on and so forth.

Now, once you have the data, you have to actually solve a whole bunch of problems. Number one, enterprise search is fundamentally different from web search in the sense that the information inside your business is protected. Not everybody can see every document inside the company, unlike on the web, where we can all see all the web pages that are out there. So when you build a search product, you have to understand an individual. You have to understand permissioning of content inside your company for any document that is out there or for any message that's been exchanged in Slack.

You need to know who are the people who have the rights to see that information. And that has to become part of your core search stack. You have to build a secure search experience where, knowing who the user is, you only surface information to them that they have permissions for. So that's a big change. Nobody had done that before in the enterprise. Search systems traditionally have been built in a model where you dump all the content to the search system and it makes it all searchable for everyone.

So we do actually make it more personalized. The third thing is that enterprise knowledge is actually quite complex. Think about a company that's been there for the last 50 years. They've got tons and tons of documents that have been written over the years, conversations that have happened. And most of them are actually obsolete now. They're no longer—in fact, they have wrong information in them for the most part because things have changed.

And not only old content versus new content, but even if you think about a lifecycle of any project inside a company, you'll write a document and then you'll write another one, then another one. And there'll be 20 revisions of the document before you get to the final blessed version of a PRD, as an example. You have people making copies of those documents for their personal purposes. So ultimately, you end up with this environment where, on any given topic, there's tons and tons of information and you don't know which one is the right one.

And that's where a good search product comes into play. And you have to figure out what content is high quality, what content is written by the right subject matter experts.

Exactly. Yeah. You have to look into those factors. You have to look into engagement. Typically, if there's a document that is viewed a lot by people, it's probably for a good reason. It's probably a high-quality document. It's probably still up to date. So you have to observe. You have to actually look into the enterprise. You have to look at how people are interacting with knowledge inside the company and use that information. Because ultimately, a good search product learns from humans.

Humans and human behavior. And so that's what we did. These are all the different pieces of challenges that you have to solve. And then language models play a big role in terms of doing that matching. Language models are good to, on any given question, especially if you customize and fine-tune embeddings for your own enterprise, it's a pretty effective way of taking any question, bringing the right information back, but then using all those traditional search techniques to figure out which information is the most recent one, which one is the highest-quality one, things like that.

So that's sort of roughly how the search stack works.

Yeah.

Well, that's the reality of an AI company today, is you have to fundamentally learn how to work in an unstable environment. This technology is moving so fast. And so, yeah, you have to do that. As new models come, you don't have the luxury to not look at them. You have to look at them. You have to see the new capabilities. You have to actually build the right evaluation frameworks to quickly see, as a new model comes in, we need to have a way within 30 minutes to know how well it's going to do on our product.

I mean, and you can do those things. You can actually build the right evaluation frameworks, things like that. Now, the way we ship these models to our customers is, one, we just give them the choice. So as they build agents in Glean, or when they ask questions in Glean, the user can manually choose what LLM they actually want to use to solve that particular question or that task. As you're building these complex multi-step agents, for each step, you can actually choose what's the best model to use.

So part of it is just working with advanced users who can make that selection for themselves, but that's the minority of the people that are going to be using your products. So when it comes to individuals, we have to actually make those smart choices ourselves. Given the nature of the question, can we use a fast and small model? Because we think the complexity of the question is not as high and a small model will produce good enough. So we have to make those kinds of judgments to actually automatically do the routing or choosing of the right model.

Yeah, so I think those are all actually, by the way, techniques that you have to use, all of them, in reality. I think we don't have computer use support in our agent platform. That doesn't mean that we've not chosen it. That probably means that we haven't gotten to it. And we prioritize based on customer demand. So right now, when you look at the kind of agents that our enterprise customers want to build, most of them tend to have this pattern, which I mentioned before, that they want that agent to work on a certain amount of business information.

They want AI to sort of do some work on it, and then they ultimately want to go and save the work of that agent somewhere within your enterprise applications. So in this model, to actually fetch the right data, you use the tool-based architecture. You build individual tools to fetch information live from different systems. You also build tools to actually save information back into those systems. And so that's the natural architecture that, in fact, everybody uses.

Any agent framework that you will see out there will have the support to actually use external tools.

So, two things. One, we actually learn a lot from that connectivity that we have into all of these different systems. I'll give you a simple example. Let's say that somebody asks a question in Slack on some technical topic, and some other individual answers that question and attaches a link to a document that contains the answer. So you can actually learn from this interaction. There are a lot of things to learn from this interaction. Number one, you've learned that that particular document that was linked in the answer contains the answer to that particular question that the user asked.

And that association is going to be super helpful to you in the future when somebody asks a similar question. You're going to use this document with more confidence than you could. Second thing is that you also learn that the individual who answered that question is potentially an expert on this topic. And so you can now, when there are more questions on this topic, actually, if AI cannot answer the question, point to this person as the person to go to, or you could actually take content written by this individual and weigh that more heavily on that particular topic in your retrieval systems.

So you're constantly learning. You're constantly learning from human activity. In fact, that's the source of all intelligence within the Glean platform, is that human behavior. And as you see more and more of it over time, it just gets better. And us being horizontal allows us to learn at the maximum pace because we are looking at what people are doing across all of these different systems.

So in fact, the prompts—it's like we're combining prompts and apps into this new concept called agents. Yeah. But the idea is that, yes. And look, the interesting thing with AI is that nobody knows what these models can actually do for you. There's a lot of uncharted, sort of unexplored frontier when it comes to these models. But you as an individual, sometimes you make a cool discovery, and that discovery ends up being something really cool for you. If you had a really easy way to share that with the rest of your team, that's how you sort of get those learnings going.

And so that's what our prompt library basically allows you to do, is when somebody has a great result with AI, we allow them to share it with their teammates.

This is the question that our employees have been asking us quite a bit as the market heats up, as more competition comes our way. My belief with any startup is you don't think about what the moat is. You just have to work hard on solving user problems. So we work with our customers. They have so many demands for us, so many things that they want. And I think just working hard and solving those problems one at a time ultimately allows you to build a very robust tech stack.

And the faster you build, the bigger that technology stack becomes. And I think that's sort of what ultimately the moat is: the hard work over the years. In enterprises, it's sort of hard to explore. It's hard to actually have network effects like the kind that you have with eBay. Enterprise software doesn't work that way. So we fundamentally just believe in that: build a good product, move fast, and that's how you stay ahead of competition.

So first, I think we believe that in the future, AI is going to be delivered in a way where there's going to be a set of horizontal AI technologies that are sort of organization-wide inside a company, LLMs being one of them. All the LLM technology is being delivered in a horizontal way to your enterprise. Similarly, the search and the retrieval and overall AI data platform is going to be delivered horizontally. And that's a platform like Glean, which is connecting all of your enterprise knowledge in one place and making sense of it and making all of that knowledge available to AI in a safe and secure way with the right policy, governance, and security.

Now, on top of that, you'll have agents which will proliferate, and they're going to be like thousands of these agents inside a large enterprise, and they're all functional and they are vertical in nature. Some of those agents will be built by you as a customer. Some of them may be specific vertical agents that you may buy from third-party specialists. And that's the architecture. Now, our vision, the way we want to be relevant in this world, is number one, we want to be the world's best personal assistant.

For employees at work. Glean Assistant, we believe, already is that today. You can think of it as a much more relevant, much more powerful version of ChatGPT inside your work. And the future that I see with AI is that every person who works is gonna have this amazing team of assistants, coworkers, and coaches around them that is going to actually make them a lot more effective. And this team is going to be proactive and help them whenever the individual needs help.

And today you have these kinds of teams. The CEO of a company has the luxury to have that team, but in the future, everybody, regardless of their role in the company, will have such a great team around them. And that's Glean's vision, that we want to be that team of AI agents around every individual that helps you do your great work. And the fact that we are connected to all of your enterprise systems will allow us to actually continue to be that, the best personal agent.

And then on the other side, we actually serve as a horizontal layer to power functional AI applications inside your enterprise.

It was a pleasure being here. Thank you so much, Matt.