Hebbia: Saving Smart People from Mundane Tasks with CEO George Sivulka

Yeah, happily. And I'll have to send Mike a picture so that he can see us up here. I think the genesis of Hebbia begins pretty far back when I was a PhD student. And I was one of the few math students who actually wanted to teach. And what they did was they threw me in a class, which is pretty famous at Stanford, called Math 51, where there's many hundreds of students. It kind of reminds me of this hall. And I got to see those students as they kind of left Stanford, continued on their careers, and years later or months later or whatnot, whenever I synced back in with them, probably the most common job outside of working at FAANG was, "Oh, I'm gonna go work, I'm gonna be a knowledge worker."

"I'm gonna go work at Morgan Stanley or Goldman Sachs." And I had some of my most brilliant students do this, like savant-level mathematicians that I was scared of, end up in these other roles. And when I would sync back, you could see in their face and their microexpressions that they had not smiled in months. And some of them had gone through physical transformations. It was really, really bad.

And the thing that Stanford scratches into your head, kind of imprinted over and over and over again in every entrepreneurial class, is, "Build a company where there's a lot of pain." And I had never seen such smart people going through so much hell, doing such stupid things over and over and over again. And I said, well, this is a good place to build a company, and forgot about it. And I continued on my research as a computational neuroscientist.

I was just kind of heads down: I'm gonna be an academic. To me, professors made more money than God, and I was really excited to be a professor. And at one point, some of my friends in the lab that I was working in started to show me the earliest transformers. This was like 2016, 2017. And there was a distinct moment where I remember them doing something that I'd never seen done before. And I kind of freaked out. And then I saw it again when Google Translate got really good.

I kind of freaked out again. And then my friends started to apply it to search, to neural information retrieval, and I kind of freaked out again. And I freaked out three times, and I said, okay, there's something here. Long story short, I ended up creating a Jupyter Notebook which ran a very early version of a neural information retrieval model, and I gave it to some of my former students that were working in the Morgan Stanley Menlo Park office to look over one type of document called the DEF 14A.

And I don't know if many of you know what that document is. It's like 400 pages long. It's kind of really, really gnarly to process. And they started to copy the Jupyter Notebook and paste it, and copy and paste it. And apparently a bunch of people that I'd never met started to use this software. And at one point it kind of hit me: there's this huge amount of pain. This is a thing that I've heard about three times. I think it's a good time to bet on large language models.

And I said, hey, I'll take a leave of absence. And I have never gone back from that leave of absence. So the genesis of Hebbia and kind of the product that we've built is all around taking the smartest people in the world that do very silly, mundane, honestly stupid tasks and using the power of large language models to help them not have to do that.

So yeah, there's kind of a product evolution as time has gone on and we've refined that customer pain. And what we noticed was people didn't want to have to learn how to run a Jupyter Notebook if they were bankers. And a lot of the time, people didn't really just want a simple search index like a Chroma or Pinecone database or whatnot, and some generative response over them. The people that really were power users of Hebbia wanted more than anything else to wrangle large language models and apply them in different ways in almost like a WYSIWYG fashion.

And so what we built was a productivity tool. It started off being very geared towards search and neural information retrieval and making that readily available to everyone. And then, as people added more and more features, we built out a suite of different large language model tools where anyone with no technical training can wrangle them and wrangle any documents to get any sort of output, whether it's an ETL pipeline, whether it's a search that you'd like to expose to users, whether it's a chatbot that we can spin up for enterprises in two seconds, or even just other ways to extract and find data that you might like.

So the one line is: we are the first large language model-native productivity tool.

A really similar analog to what Hebbia does actually harkens back to Excel in 1985, where Excel took a technology, the SQL database, and allowed just normal humans to wrangle it in an application and create value. And they started in a niche in fixed income, and now governments run on Excel. And in 2,000 years, archaeologists will be looking at Excel files. And to me, it's actually the most important software to ever have been made. On a similar vein, I think that large language models are this new, very powerful technology, like SQL once was.

And we're trying to build Hebbia to be just like Excel, honestly, a way to programmatically wrangle, in a WYSIWYG way, those large language models. And so what that means is it's an application that you can have in your browser or on your computer, and you can have any documents that are easily ingestible, and you could have any large language models that are kind of easily loadable over those documents, and you could run different workflows.

It comes with batteries included, but you can also pick or choose your models.

And I think the thing that's very interesting is no one really cares what the backend of Excel looks like. It just works. And it's really easy to get data into Excel. And it's really easy to run analyses and get data out of Excel in the form that you'd like. And for Hebbia, a lot of the technology that we built, a lot of what's proprietary, is actually that input and output flow, and then that flexibility.

We have not. So we work with many of the largest asset managers, we work with many banks, we work with the U.S. government, and we have not needed to fine-tune or kind of personalize any part of the software. In the same way where that would be crazy to be fine-tuning Microsoft Word, we've tried to build these things in a way that's very easy to use for anything.

Yeah, so DEF 14As weren't the end-all, be-all from those earliest days. And honestly, we spent a very long time trying to find the perfect market niche, our equivalent to fixed income, where Excel, in the span of 18 months, took over 90% of the fixed-income industry, and everyone was using it, and they started using it for everything else. And so it was like, where could this technology be immediately leveraged where we could go to market, where we could take over 90% of the market as fast as possible and then go and be leveraged for other things?

And the way that our team really thought about it was, hey, these large language models and the tool that we've created are very good at mapping from one ontology to another ontology, one set of unstructured data, some taxonomy, some set of files into, hey, here's a list of questions and a list of answers. And in private equity due diligence, you have a data room which has all of a company's files. And it's your job as a private equity analyst or a third party to come in and fill out a 100- or 400-question-long tracker that has all these questions about a business.

So in private equity, there's no numbers or reporting or anything else. It's all in the unstructured data. And you have the people that are the highest paid that hate their lives the most, whose job is entirely to, hey, go map from this data room about a company to this set of criteria that dictates whether or not we pour hundreds of millions of dollars into it. And so we started there. And so we've kind of gone to market with many of the largest asset managers.

In North America and now Europe. And we solve basically their most pressing pain: how do you deploy more capital more quickly in a more accurate fashion? And six months ago, there wasn't a single private equity CEO who'd even talk about Hebbia. And now they're all like, oh, I need a large language model strategy. And I've heard that my team is getting beat by this other team that has it. And it's helping us go to market really, really fast, which is quite good.

So I think the thing that I fundamentally very much believe, which maybe the last demo made me believe a little bit less, is that humans are really, really important. And making humans better is actually the ultimate goal of what I do. And if you look at, I'm gonna hearken back to Excel once again. Bookkeeping before Excel and the structured database was literally in books. A spreadsheet is called that because a spreadsheet was spread across the two different pages.

And so you'd open up a big book and you'd see a spreadsheet. And when this technology came out and when it was easily available in these productivity tools, people's jobs changed and their job actually became, hey, I'm not gonna use a slide rule and all kinds of calculators to do my job. Instead, I'll use a computer. And I think Hebbia is very similar. Like, we are building a tool that is aligned with humans and aligned with creating value.

And I think to date, not a single job has been replaced by Hebbia, knocking wood. But I think one day there'll be many more jobs that are created to run Hebbia.

Yeah, I'm happy to take questions as well. I don't know if that's something we—

The three buckets that I can definitely talk to are input, output, and then flexibility. And these are all the kind of different things we've done a lot of R&D into. One thing I am obsessed about is not being replaced by an increasingly large language model or better language model. So, kind of side anecdote: today, OpenAI released, as many people thought they would, a new API that allowed you to call functions, right? And so it was a kind of big thing, and there's a ton of startups that had done a lot of work into doing R&D around how to create a tool-form implementation, yada, yada, yada.

And they released this today, and that work is now moot. So what I care a lot about is, how do I build tooling and functionality around a large language model to make it much better? On the input side, just like there's entire companies, industries even, built around getting data into a SQL database, we've done a lot of work on integrations, on parsing documents, on actually creating multiple types of indexes to make sure that these models have access to good information in a way that actually matters.

On the output side, we've done a lot of work to find ways to actually successfully programmatically wrangle them to get to outputs that our users want and to do that in a generalizable way where the user's still in control of what that output looks like. And then on the flexibility side, it's a beautiful product, in my opinion, that allows people to very easily understand what they're doing. And there's a lot of work into building reliability and trust for end users because people just don't, by default, trust the output of a large language model.

And if their job is on the line, if they're allocating a ton of money or they're making a mission-critical decision, it matters that they're right.

Yeah, so those are all things that we do. So we expose a neural search chatbot. We allow you to compare two documents semantically, which is a pretty hard problem that we're pretty excited to have solved. As I mentioned before, for the private equity diligence case and other use cases where you're mapping between two ontologies, we allow you to create an output, like a question list or something even more complex. And you could say, hey, go fill this all out. And then we run an asynchronous operation to execute all of those tasks.

Okay, yeah. So if it's about mapping between one taxonomy and another taxonomy, we can have a large language model do that very successfully.

And that's not just an information retrieval task. A lot of the time, it's contextual and self-referential, which is pretty interesting. In terms of alerts, a lot of our customers care that, hey, when something new comes in, they're aware of it, even if they don't have alerts. So Google has alerts where every time someone mentions Hebbia, our marketing team can get notified. For Hebbia, if every time we get sent a new contract, I want to know who the contract's with, who the senior stakeholder is, what the fees are, and what our updated customer concentration is.

This is just for my internal use case. I can set an alert to do that. And every time we sign a new contract, I, as a CEO, know what the salespeople have agreed to in terms of terms, which is pretty fun. So that's some of the features.

This is very funny. The generative AI hype, in my experience as a generative AI CEO, has been very much: no one knows what I'm talking about, and then all of a sudden, everyone's asking me about something. And then they all think they can do it themselves with their own internal database and their developers. And now it's this part of the cycle where everyone's coming back to me and they're saying, oh, this is very difficult. And it's a gift.

Actually, I had the CTO of our largest customer explain that to me the other day. And I think what many people are finding is that the technology's very, very good, right? It's adequate. It's good at doing very simple tasks. But when you start to do these tasks where it requires multiple steps, or this programmatic wrangling of things, it's not a prompt. It's not going to be LangChain and the right logical operators. None of this stuff is actually going to solve it.

And that's because if you have a 90% good system and then another 90% good system and then another 90% good system, and then your output is the sum—or rather, the product—of all those 90% good systems, you're going to have something that is 10% good, depending on how many steps you have. And so what we're finding is that these models are good at first-order tasks. You can get 90% good at a first-order task. They don't quite have the context window, which will be solved, the contextualization, the repeated back and forth that allows them to solve nth-order tasks.

In a very general way. I think context windows will solve some part of it, but being able to programmatically run them is so, so, so important. And it's not just creating some logical tree of operators in LangChain and then having the right Pinecone database hooked up. 99.99% accuracy. So, great.

I often joke internally that if we stay in financial services, I will have failed. I really value the go-to-market of productivity tools that started in a vertical but built something and had a product vision all along that was generalizable. And a lot of the time, I've actually made hard calls where I could have made a smaller cut, a simpler cut to just serve our ICP, our initial customer profile. I actually chose not to do that and build something more broad.

So we're starting to serve law firms. We just started to serve the government in a bigger capacity. I care that when my children are in sixth grade and they're in computer science class, in their system tray, they have to learn how to use Google Chrome, how to use Microsoft Excel, and how to use Hebbia. And anything less than that, I'd be remiss.

Thanks for listening to The MAD Podcast. If you liked this episode, be sure to leave us a review. firstmark.com/events/data-driven.