MAD Podcast
    MAD Podcast

    The MAD Podcast with Matt Turck

    AI, Gaming, and Building a Data Science Brand - A Conversation with Carly Taylor, Founder of Rebel Data Science

    Carly Taylor is the Founder at Rebel Data Science. We cover how machine learning security focuses on outliers and changes adversary behavior, why generated NPC dialogue can make open-world games less repetitive without replacing existing work, and why startups need data engineering before expecting data scientists to own an end-to-end pipeline.

    09/13/2023

    Hosted by Matt Turck · with Carly Taylor, Founder, Rebel Data Science

    AI in gamingmachine learning securitydata sciencemodel observabilitytechnical marketing
    Listen now
    YouTubeApple PodcastsSpotify
    40 min · 1 chapters
    Contents

    Transcript

    Full episode

    0:00
    Matt Turck1:12

    Hi, Carly. Thanks for joining us on The MAD Podcast. I know you are pretty bombarded with requests to appear on those things, so I particularly appreciate that you accepted to spend a little bit of time with us today.

    Carly Taylor1:14

    Anytime, Matt. Anytime.

    Matt Turck1:49

    So, to set it up, you are a woman of many talents. You do machine learning at Call of Duty, which, of course, is one of the biggest franchises in the gaming world, part of Activision. You are also the founder of Rebel Data Science, which is your consulting shop. And then you are also a top voice for data science on LinkedIn with over 100,000 followers and generally pretty prominent on social media. So we're going to talk about all of this, a bunch of things today, including the intersection of gaming and AI, but also your work in general.

    Matt Turck2:24

    Lessons learned through your various parts of your journey in the world of data science. So it should be really interesting. So maybe let's start with a quick version of your journey that led you to the world of gaming and starting Rebel Data Science.

    Carly Taylor2:49

    For sure. Yeah, I actually started in chemistry. So, not usually where you'd think you'd end up in gaming. But really, I was a gamer my entire life. So it was always something I did in my spare time. But I had never really considered if I could make a career out of it. So when I finished my master's degree in computational chemistry, I was looking at going into this new field that they called data science. And it was a very exciting time for everyone.

    Carly Taylor3:18

    People were trying to learn what it meant. What were all these algorithms doing? There were no degrees in data science then, so people like me, chemists, could come into the room and be taken seriously. So that was much appreciated. And so that was when I started my career in data science, after making that career transition. It was definitely a difficult time that I'll just yada yada yada over. But once I got into data science, I really started thinking seriously about my future and where I really wanted to grow my career.

    Carly Taylor3:54

    I've worked at a variety of different startups on product teams, on marketing teams. I had kind of floated around trying to find my people, where I really wanted to land was doing more applied research, machine learning research, machine learning engineering. And one day during the pandemic, I was sitting at home playing Call of Duty: Warzone like everybody else. And I thought, wouldn't it be fun to work on a game like this? And honestly, within a week, I saw a job for Activision Call of Duty on LinkedIn.

    Carly Taylor4:20

    And I was just like, wouldn't this be wild if I got this job? They're hiring a machine learning engineer. It's close by. What are the odds? And I ended up getting the job. And so it's been three years since that happened, and I'm super grateful every day. It was a lot of luck and a little bit of preparation, but mostly luck.

    Matt Turck4:51

    Wonderful. So let's get into the intersection of AI and gaming. And, to put it up front, I know that there's only so much you can talk about in the context of your work at Activision, interestingly, because presumably AI and machine learning is such a strategic topic and part of what the company considers to be a really important part of their IP and strategy in general. But to the extent you can talk about it, what does machine learning mean in the context of a game like Call of Duty?

    Carly Taylor5:24

    Yeah, that's a great question. So I think back even five years ago to some of the gaming conferences I would go to. And you think about the things that people were discussing, right? It's a lot of the same usual suspects. Graphics is a huge area of research. But you started to see more and more this idea of machine learning applied to gaming-specific problems. And now I feel like I can't look at any gaming conference without seeing something that's, like, deep in the weeds for gaming.

    Carly Taylor6:03

    Maybe it's, like, synchronizing sound to movement in the world, right? Like some interesting rendering/audio problem, but there's a machine learning component of whatever they're talking about. So it's like, how did we 10x our workflow with machine learning? Or how did we unlimit our artists with machine learning or generative AI, right? And so it's a lot of rapid iterations on making all of the people who were already doing amazing things so much more effective. And it's just been insane to see the explosion in this.

    Carly Taylor6:31

    Like, I feel like now you can't turn around without seeing another application of AI and machine learning in the gaming space. And it makes sense, right? Because what we do ultimately is very deep in research, like I said, right? Like computer graphics, obviously. Who were the biggest purchasers of GPUs way back when? It's always been gamers, right? And then now we have this machine learning evolution where people are realizing the power of everything that machine learning can bring to the table for organizations, for individuals.

    Carly Taylor7:06

    And so I think it really makes sense to me, almost like two industries that would merge together, come together. NVIDIA is huge in this space, right? And it makes sense. Like, why wouldn't they be? They basically own the graphics market, right? And the GPU market. So strategically, they're making a lot of good moves there too. Yep.

    Matt Turck7:22

    And I think I heard you somewhere, because I think that was part of one of the several jobs you had at Activision, that there was a security component as well, like an intersection of AI and security. What is that application or use case?

    Carly Taylor7:46

    Oh, let's speak more generally so that I don't get in trouble for divulging some of my trade secrets. Stepping back from gaming, because there's a lot of similarities, the way I like to describe this is: think about you work at a bank, right? How would you go about detecting fraud on someone's credit card? You might start with, all right, let's build up a history of their purchases. Like, what kind of stuff does this person usually do? If it's me, they're buying video games, they're going to GameStop.

    Carly Taylor8:13

    Like, so if I end up buying, I don't know, like a handbag in Greece one day, you might say, that's weird. What's weird about it? Well, the item is strange. The location is strange. There's a lot of weirdness going on here, right? And so this kind of idea of anomaly detection is pretty well known when you think about spaces like banking. I'd say that there's similar overlap in gaming as well, right? Because you're just looking at behaviors at scale over time, trying to figure out what seems normal, what seems abnormal, and how can you build in safeguards for abnormal behavior?

    Carly Taylor8:51

    Because not every abnormal behavior is bad, but abnormal behaviors are something you should look at. The way I describe this kind of machine learning is there's a tenet of machine learning where, like, you just throw out the outliers because they're gonna mess up your distribution and you kind of don't wanna deal with them. For security, what you do is you find the outliers and you hyperfocus on them because that's where the most interesting information about your problem set lies. And I'd say there's one more component about this that makes machine learning particularly interesting.

    Carly Taylor9:22

    Thinking back to the credit card example, if you have a situation where someone is committing credit card fraud, they did buy a handbag in Greece, it wasn't their credit card, you flag it and you say, okay, this is suspicious. What's the next thing this person's going to do? I mean, you hope they change their life and turn it around and become a functioning member of society. But if they want to continue to commit fraud, they'll just learn from what you did, right?

    Carly Taylor9:32

    So the act of intervention into this cycle of fraud, attempted fraud, stopping the fraud changes the nature of the behavior of the person trying to circumvent whatever you're doing. And for machine learning, this is very interesting because when you think about how you train a model, you train it on known behaviors that you've seen, and you hope that it can handle certain edge cases well enough, and you retrain it and hope that it continues to catch those edge cases and it doesn't go off the rails or explode too badly when something completely new happens.

    Carly Taylor10:14

    But what I've never really considered until I worked in security was the act of classifying someone. Let's say making that machine learning prediction, you classify, rubber-stamp something as fraud. That part of the machine learning model augments the behavior in the wild and is going to change the way people interact with that model in the future. And this adversarial nature of it is very interesting to me, and it's something that I think traditional machine learning hasn't really been agile enough to deal with, right?

    Carly Taylor10:35

    Like, the act of doing machine learning is fundamentally changing the problem you're trying to solve, right? And it's like, well, where do you go from there? It's a very interesting space.

    Matt Turck10:41

    That's a super interesting thought. So there's sort of like societal and behavioral response to—

    Carly Taylor10:41

    Yeah.

    Matt Turck10:42

    Machine learning, right? Like, which is—

    Carly Taylor10:42

    Exactly.

    Matt Turck10:46

    Which we're just starting to experience because now machine learning is deployed at scale.

    Carly Taylor10:47

    Thousands.

    Matt Turck10:49

    Across enough use cases that we—

    Carly Taylor10:49

    Yeah.

    Matt Turck10:50

    Like, how do we get—

    Carly Taylor11:01

    We see with GenAI, right? ChatGPT, you're talking to it, you see its safeguards, you can kind of play with it and mess with its guardrails, right? And that's a security problem for OpenAI, for sure. Yes.

    Matt Turck11:28

    And then for generative AI specifically, are you seeing, in particular in the world of gaming but not necessarily, emerging use cases? So you mentioned graphics, and then you mentioned the specific use case of security, but what do people at a place like Call of Duty slash Activision do with generative AI? Do they experiment with it, play with it, whatever you can talk about?

    Carly Taylor11:55

    Yeah, I'd say more broadly, what I've seen publicly discussed at other gaming companies, especially at conferences like the Game Developers Conference, I can probably talk about more because that's all obviously been disclosed. I sat in on a presentation of—I forget who it was, maybe it was ZeniMax, I'll need to remember. They were basically trying to figure out if they could use generative AI to insert the player's name into the game, right? And so I see actually a lot of this because people love open-world games, but there's a joke on social media right now, right, that everyone's an NPC because they're like these unbelievable two-dimensional characters that don't really have a personality and they don't really interact with the world in a way that makes sense.

    Carly Taylor12:20

    And so there's a huge push to make NPCs not NPCs, if you get what I mean, right?

    Matt Turck12:20

    Mm-hmm.

    Carly Taylor12:38

    And I think that this is a really cool use case because the stakes, to me, are a little bit lower. Like, it's no one's full-time job to make an NPC say your name. It's just a functionality we've never had. So you're not really replacing anything people were doing. You're just making something that didn't exist exist. And it's kind of cool. So they were actually—you'd put in your name at the beginning. You'd say, like, I'm Spaghetti John or something weird, and it could handle it.

    Carly Taylor13:09

    It was super cool, super strange, because you'd think, like, oh, it could just say Frank. But if you said your name was Felix the King, it'd be like, what? And it could parrot it back to you, and the NPCs were saying your name, and it felt a little bit more immersive in that way. And that's really what I would love to see because I love the open-world games where you can get lost. But it gets tiring after a while when you hear them say the same four canned phrases.

    Matt Turck13:28

    Yeah. And that's because generative AI enables you to create the code to do this very simply in a way that you would not assign somebody to spend X many hours at X-dollar cost per hour to do it.

    Carly Taylor13:28

    Yeah.

    Matt Turck13:32

    But if generative AI can do it for you, you might as well, kind of thing. Is that what's behind this?

    Carly Taylor13:54

    Yeah, I'd say that's definitely a fair characterization. Like, why would I pay someone like a voice actor to record, like, John, Johnny, Jonathan, right, for every single name that anyone could possibly be? And then you'd leave people out, you'd have a bias in what names you think are important, when you can actually just generate that on the fly. So, yeah.

    Matt Turck14:03

    Do you think there's going to be more kind of on-the-fly personalization of journeys within games?

    Carly Taylor14:26

    I would love that. There used to be—I'm blanking on it. There was a game a really long time ago that tried to do something similar. It obviously wasn't generative AI in the way we're thinking, but it was a very in-depth, choose-your-own-adventure kind of situation. I would love to see that because, again, you're talking to something in the world and you know that there's two paths laid ahead of you. At any point, you can almost see the decision tree of where you're able to go.

    Carly Taylor14:55

    It would be fun to kind of obfuscate that a little bit away from the player and have them feel even more immersed. You don't know where you're going to end up, right? You could go on an adventure no one's ever seen before, a completely new quest that didn't exist for the last time you played through it because you made choices that were different and the game has learned and augmented around you. That's such a cool idea to me. Let's patent it and build it, Matt.

    Carly Taylor14:57

    Yeah.

    Matt Turck15:30

    That feels like the ultimate gaming experience. So again, not necessarily at Activision, but what have you seen in the gaming world in terms of the role of data science, like the way data science organizations are put together in those companies? Is that a reasonably new thing, that there would be data scientists in the first place? Like, how do they find their way in the world of gaming, which historically has been pretty insular?

    Carly Taylor15:58

    Yeah, that's a great question. I'd say that it's not necessarily new, like within the last couple of years. Organizationally taking data science seriously, elevating it within the org, relying on center-of-excellence models, I'd say is newer to gaming, right? I think everyone, all industries, are kind of figuring this out as they go. I would put gaming right up there with everyone else, trying to decide where in the org should your data team live. There's actually a decent number of choices.

    Carly Taylor16:27

    And when you think about a publishing studio, not necessarily Activision, but any of the big companies that have a couple of different gaming studios within them, right? You have different owners of different pieces of different projects that you're working on. So, do you centralize your data team and they kind of help everyone? Certain teams are going to want embedded data scientists because they have a big need for it, and what they're doing is iterative enough that they really need that expertise.

    Carly Taylor17:00

    And you need to be close to your subject matter experts. And there's a constant trade-off between those two things. As soon as you centralize something, you will inevitably lose the deep expertise you can get from embedding. But as soon as you embed everyone, you lose that center of excellence where everyone comes together and you set standards for your data science and standards for your models, and you're able to learn from each other, and you're able to scale your infrastructure. And since data science doesn't exist in a silo, you need data engineers. I think the centralized model is becoming more prevalent, if I'd say one is winning.

    Carly Taylor17:16

    But it doesn't come without a cost, at least from my perspective.

    Matt Turck17:45

    Maybe almost at a personal level, and I know you wrote about this a little bit, your journey into the world of gaming. I think you mentioned that that felt like you found your people. But again, the gaming industry is fascinating, but it's historically been kind of insular and kind of quirky. What was your experience sort of discovering that industry as a data scientist?

    Carly Taylor18:16

    It's interesting because I'd say on the data science side, from what I've seen, we have a lot more representation in terms of different genders and people from different backgrounds. And I think it's because data science has historically been something that a lot of people like me, who before data science was a thing, could get into. So we were already used to being like, one of my colleagues is a philosopher, I'm a chemist, we have physicists, right? There's no standard for that.

    Carly Taylor18:40

    It's changing a little bit, but it's traditionally been that way. And I see more representation in data teams than I do for something, let's say, like hardware-level programming, which has just historically been like a lot of these deep, nitty-gritty computer science fields have been more skewed towards men. And so I'd say the representation there is better, but I think that gaming as an industry as a whole, you can look at the data yourself, it still suffers from this idea that only boys play video games, and so only boys make video games.

    Carly Taylor19:04

    And it's really changing, I think, but they're like these big ships, you can't turn them around instantly.

    Matt Turck19:18

    Very interesting. Okay, maybe switching tacks a little bit and talking about Rebel Data Science. So did you want to maybe explain what that is, what the firm does?

    Carly Taylor19:41

    Oh yeah, for sure. So that's my consulting company. The name came out of actually one of my followers at one point when I was really early in my journey was like, "You should follow Carly. She's a rebel data scientist." And I was like, "Wow, has there ever been a more apt description of my personality?" I would have gone with, like, chaos data science maybe, but rebel fits as well. And so, yeah, I started this as a consulting company basically because I was getting a lot of questions from either people looking for help in their careers, but mostly brands who were looking for help with their data science branding, how to be taken more seriously as thought leaders in the space.

    Carly Taylor20:22

    They were looking to partner with me on projects where we could amplify each other's vision or give away something that they were excited about. And I realized I needed kind of some umbrella to encompass all of these things that I was getting asked to do. And that's kind of where Rebel Data Science came from. I have a friend working with me on it now. So we are a team of two, a company of two whole people.

    Carly Taylor20:26

    It's an exciting time.

    Matt Turck20:44

    Yeah, I'm sure there's a lot of interesting learnings there. And it's something that, as a VC, resonates with me because the world of data science is obviously very technical, and there's a lot of startups doing very interesting technical stuff. But as all VCs in particular, and a lot of other people as well, know, success for a startup is like, basically half of the battle is distribution and marketing and getting known and building a brand, and having the very best products in complete obscurity does...

    Matt Turck21:35

    Exactly. So what have you found has been sort of interesting, whatever it is, lessons learned or any recommendations you have for, let's say I'm a stealth startup, raised a seed round, and my entire team is super technically strong, but marketing is just not what I've personally gravitated towards for a long time. So I need the help. So what do you recommend for people in that case?

    Carly Taylor22:02

    I'd say that I think if we can all learn anything about what's been going on the last few years, really with social media and the way people use products, to me, and I'm probably biased, your community is your most important asset. And so what you really need—you can have the best people, like you said, you can have the best product, but existing in obscurity isn't going to get you anywhere. You need someone, or you can do it yourself, who you trust to build a community around your product and a community of people who trust you and who will evangelize for you.

    Carly Taylor22:43

    And without that community and that support, it's going to be really, really, really hard to cut through the noise because there's so many people with interesting ideas nowadays. And attention is a commodity just like anything else, and you're going to have to fight for it and you're going to have to pay for it and you're going to have to be strategic to get it. And acting like that's somehow selling out or it's focusing on the wrong thing—I know because I'm also a super technical person.

    Carly Taylor23:17

    Sometimes you can over-index on the technical, and getting maybe your platform to be 5% faster or supported on one more browser might not 10x you the way that spending that time building a community would. Because ultimately, without that, it doesn't matter what browsers are supported. It doesn't matter how good the buttons on your homepage look, right? You really need that. And it's hard to get that organically. It's hard to get that authentically. You can spend a lot of money and come across as inauthentic as well.

    Carly Taylor23:33

    So there's a lot of pitfalls. And so I'd say that finding someone or something or a strategy for your community would be, to me—you can't exist without that.

    Matt Turck23:53

    Yeah. So let's dig into this. That's super interesting. I guess, what are some of those pitfalls and conversely, what has worked? I mean, it sounds like a lot of it—and my words, not yours—but a lot of it seems to revolve around content and quality of content. So maybe let's double-click on that.

    Carly Taylor24:19

    Yeah, I think let's think about some pitfalls that I've definitely seen. Misaligning your messaging with where you're at or with where you want to be, right? So who are you talking to, and why are you talking to them? Having clearly defined audiences that you want to reach, right? Let's say, for me, you come to me and you want me to talk on LinkedIn about your data science product that sells plants. I don't know how we do that, but it does it.

    Carly Taylor24:48

    And what you tell me is you really want to reach decision-makers who are going to be building office buildings because they buy a lot of plants. You don't care about the individual consumer right now, but you don't ask me any questions about my audience. You're not trying to dig into what my messaging or my strategy would be. Maybe you're not even actually explaining to me your funnel. Maybe you don't understand your funnel at all, right? Lacking all of that context, you're never going to get out of social media what you put in, because you're either going to have too broad of a message that people are kind of going to be like, "eh," or you're going to have way too hyper-specific of a message with misalignment with your audience.

    Carly Taylor25:29

    And people are also going to be like, "eh." And that's where the understanding of your community is going to come into play. It's extremely important. And you might have to divvy up your audiences too, right? You will have a place where you speak to decision-makers, and that's where they live, and that's what you do. Maybe it's making a podcast, right? It's a great place to start. Maybe you also have a YouTube channel where you're trying to reach a couple more general people.

    Carly Taylor25:57

    And then on LinkedIn, you're looking for a big group of users because you gotta have the base users if you're gonna build anything else about the product, right? How are you gonna get the decision-makers on board if no one's using what you build? And you need to be very strategic about each one of those subgroups, and you can't just hit them with a mass message and hope it lands. Or even worse, hit them with a mass message that talks to all of the groups, but you just do it in one go and hope that they read the whole thing and get to their part that resonates with them.

    Carly Taylor26:09

    Right.

    Matt Turck26:35

    And in terms of channels, so you're particularly prominent on LinkedIn. Again, if I'm a startup doing technical data science stuff or AI stuff, would you recommend I start with one channel and just focus all my energy on that, or do all the things that you mentioned? So, like YouTube and podcast? Or, like, how do I start?

    Carly Taylor26:56

    I would repurpose content as much as possible to make it as easy on yourself as you possibly can. So if you're already writing blog posts, why not convert that into some sort of newsletter that you can put on LinkedIn and on Substack, right? You're literally just copying and pasting stuff, and you're kind of trying to reach a couple different new people. But you think about it as just top-of-funnel inbound. You'll deal with it later and see who comes through those channels once you get a little bit more established.

    Carly Taylor27:23

    Right? You're just kind of trying to start someplace. You have a newsletter, right? So cut it up into some smaller pieces for some LinkedIn posts as well, right? Take some paragraphs, take some high-level learnings, ask people some questions. LinkedIn is a great place to get to know your audience as well. People are really involved in the comments and leave thoughtful comments, from my experience. So that's where you can start to ask people about what they're excited about, what part of this resonated with them.

    Carly Taylor27:48

    You'll notice who responds to what kind of content where on your LinkedIn. Now you've got that kind of going. Let's say you found some LinkedIn posts that worked really well. Well, why not pop on a video and read it, and read through some of the comments and discuss what you learned? Now you can edit that into short-form clips for Instagram Reels. You could throw that on TikTok. You could throw it on YouTube. Take the longer edit, put the whole thing on YouTube.

    Carly Taylor28:14

    You can record a podcast with video, throw it on YouTube and on podcasts, right? The more you repurpose your content and understand how you can leverage a similar message but in a different way, you'll start to get a feel for where the people are that you want to talk to about what, and then you can start kind of trying to diverge your messaging streams once you have that base established, I think.

    Matt Turck28:39

    Yep. And not to spend too long on LinkedIn, but you've found that LinkedIn is particularly effective. And the context behind my question is I've been personally surprised, pleasantly surprised, by the emergence of the data science or data or machine learning and AI community on LinkedIn, which seems, at least from my perspective, sort of a recent thing, maybe the last couple of years.

    Carly Taylor28:44

    Yeah, we have a great community there. Yeah, it's awesome. We have a pretty deep bench.

    Matt Turck28:56

    And that's a good, again, like stealing the use case of a stealth startup, right? LinkedIn is a place you would recommend.

    Carly Taylor28:57

    Oh, for sure, for sure.

    Matt Turck29:05

    Yeah. Now, which is somewhat non-intuitive, I think, to many compared to Twitter/X.

    Carly Taylor29:05

    Yeah.

    Matt Turck29:06

    Has to be like the—

    Carly Taylor29:26

    Twitter might get you more impressions, I think. And you might get your message out wider. So I don't think it has to be an either-or. But if I was going to spend a lot of my time, especially if I was a data science or AI startup, I would learn where my audience lived on LinkedIn. And I'd also learn where the people who are talking to my audience live. Right? I've got a few creators who I follow, who I love, who are way over-indexed in the strategy decision-maker space, and that's their following.

    Carly Taylor29:59

    And I love to read their content and look at their comments because it's like a different league from what my people are talking about. We're talking about gaming, and it's a little more casual. And once you figure out not just your people in your space, but you figure out the people you want to follow, you'll start to see there's a whole lot of power in that, and those networks are extremely powerful.

    Matt Turck30:31

    Okay, great. So maybe switching away from content strategies into maybe some more of the technical stuff. Curious, through your practice as a data scientist, machine learning person at Activision, but also as a consultant, what are some of the tools that you like, open-source projects, favorite kind of things that you would recommend to others?

    Carly Taylor30:42

    Oh, that's such a great question. For tools, I've been using ChatGPT every single day. I think everyone definitely has.

    Matt Turck30:45

    To do what, out of curiosity?

    Carly Taylor30:46

    Oh my gosh, everything.

    Matt Turck30:47

    Everything.

    Carly Taylor31:15

    When I have to do anything now by myself, it feels so tedious. Like, wow, I can already feel myself becoming reliant on this technology. It's kind of scary. Let me think about some open-source tooling. I have a friend who's working on some amazing open-source deep learning frameworks that I'll have to send you after this, if we can put a link in the show notes, because what they're doing is really cool. I still am really attracted to computer vision.

    Carly Taylor31:46

    Projects and generative—or, sorry, GANs, so the adversarial networks, just because I had started in that before I moved over to the anomaly detection stuff, and it's always been something that's been fascinating to me. I also really like—in terms of libraries, I'd say my go-to machine learning library is probably SHAP, which is Shapley Additive exPlanations. I harp on observability a lot because I think it's probably the most important thing a data scientist can focus on.

    Carly Taylor32:09

    And so any tools you can surround yourself with that help towards the goal of observability, I think, are probably something that, if I had to pick one thing, it would be that. Because if I can't explain my work, what is it?

    Matt Turck32:17

    And any favorite tools there, more like commercial products or open-source products? I mean, there's a whole industry around observability.

    Carly Taylor32:26

    Yeah, there's a ton of observability platforms, no one that I necessarily want to plug because I love them all. I don't want to play favorites.

    Matt Turck32:46

    Yeah, yeah. And lessons learned around observability: what matters most? Is that the data lineage part? Is that the detection? Is that the fixing? Is that the reporting of it, or all of the above?

    Carly Taylor33:11

    What I've seen have the most impact, though, and where I see data science projects fail a lot, is actually observability into drift and label drift. So after the fact, I think a lot of when we started building data science projects, we were like, you have to be end-to-end, which is fine, prototype to production. But what production means and what that maintenance means is something that's actually kind of difficult to define. Because how often do you go back? How much time do you have in a day to go check the distribution of your labels?

    Carly Taylor33:30

    And check for drift. A lot of people are working in this space, which I think is great, but it's just one more thing that you don't have to worry about if you can get an alert if it seems like the distribution is changing of your labels or something weird is happening. I think that could save a lot of projects from going off the rails once they're kind of in prod and you kind of don't know what they're doing and you don't have time to check.

    Matt Turck34:05

    And maybe to close on lessons learned through your consulting practice in particular, like, I know you spend time advising companies on how to build teams. And we talked about teams a little bit in the context of the gaming industry. But if I'm an early-stage company or even slightly later-stage startup and I want to build a data science team, where do I start? What do I do first? Do you worry about data infrastructure first, then people, or people first, then infrastructure?

    Matt Turck34:14

    How do you think about it?

    Carly Taylor34:36

    I think about it—they're almost inextricably linked, in my opinion, because it really depends on what problem you're trying to solve. Sometimes you'll already have a little bit of data and you just don't know necessarily what to do with it. Sometimes you won't really have your data yet, but you know where you want to get it. And those are two very different places to be in. If you already have a level of data maturity, I would say that getting probably a machine learning engineer, if you're trying to do some data science, might be a good move because you can kind of Swiss Army knife them into doing a little bit of the data engineering and some of the data science and production support.

    Carly Taylor35:18

    If you don't have your data ready yet, you can't skip the data engineering piece of this. And I'd say you probably need someone who's going to be like your cloud data engineer. You just have to have those basics. And I wouldn't expect even a really good data scientist to own all of that pipeline end to end. I don't think that that's fair for one person. And it kind of sets the startup up to fail in certain ways because you have a single breakage point for your whole pipeline.

    Carly Taylor35:40

    And you really don't want to end up in positions like that because you end up pretty vulnerable. But it's more expensive. So you just have to decide, right? Hiring a data engineer is not going to be cheap or easy. And depending on your data...

    Matt Turck36:02

    Having your data ready, where is your threshold? And I guess what are the ways you recommend people get started? Is that, you need to have, like, a whole modern data stack in place, so go buy Snowflake, or just put a bunch of data on S3? Like, what do you typically advise?

    Carly Taylor36:21

    I think it's somewhere in between there. Like, let's say you have enough data that you're kind of, if you're not super technical, you're noodling around in spreadsheets, but it's too much for you to handle by yourself. You're kind of like, we can't really do what we're trying to do. We can't scale at what we're looking for. But we do have the data available to—if I were to hand this to someone today, I'd be like, can you build something? And they could get it done in a week, right?

    Carly Taylor36:52

    If part of your equation is we also have to make the data available, it's a fundamentally different conversation. And it's one that you perhaps should hire both people at the same time, though, right? Because the ingester of the data and the producer of the data need to be on the same page. You don't have to do that. But if you have the option to, I think that that's a decent way to move forward. Okay.

    Matt Turck37:18

    So maybe to finish, any part of the AI data science world you find particularly interesting or exciting? We talked about the data tools you use, but what's, like, any company that you find interesting or project or anything that people should learn about?

    Carly Taylor37:38

    Yeah, I'm actually noodling around a little bit in the health tech space. So this is newish for me, not really with my background. I obviously worked—I did some molecular drug design in grad school. But I think that healthcare is historically like eight years behind everyone else.

    Matt Turck37:38

    Mm-hmm.

    Carly Taylor38:04

    But there's also a very massive upside there right now for people who are going to be using machine learning and data science to solve problems. So I think that there's some really, really cool people in the space. I will plug Health Universe, which is one of my friend's companies. They are a platform for healthcare researchers, a machine learning platform for healthcare researchers. They're great stuff. And there's tons of companies like that that are just making huge strides in healthcare and putting patient outcomes first.

    Carly Taylor38:26

    And yeah, I think that data science can have a huge positive impact there. We just need to be mindful and have experts there and trusted voices to make the right moves. And so I'm trying to get my toes in a little bit.

    Matt Turck38:39

    Very cool. Awesome. So where can people find you online? So we talked about LinkedIn, so they can follow you there. Other places? And what are your handles?

    Carly Taylor39:04

    Yeah, so Carly Taylor Data on LinkedIn is my URL. It's just under Carly Taylor. Carly Machine Learning, I think, on Instagram. I don't honestly, I don't even know. I should be better at this, but I'm not. Mostly LinkedIn. And then you can find my Linktree on there and find the other things I do. I'm on Threads, but I don't really talk about data science on there. I mostly talk about shower thoughts.

    Matt Turck39:31

    That's like a whole different avenue and very interesting as well. Cool. All right. Well, thank you so much. As I said, I know you get a lot of requests to speak, and really enjoyed it. Very inspirational, very interesting. And thanks again, and look forward to the next conversation. Wonderful.

    Carly Taylor39:33

    Of course. Yeah, it's been a pleasure. Thanks, Matt.

    Matt Turck39:35

    Thanks so much, Carly. Bye.

    Carly Taylor40:01

    Bye. Thanks for joining us for The MAD Podcast. We're back here every Wednesday with new conversations with leaders in the machine learning, AI, and data space. And if you like this show, you can also find a video recording of not only this episode, but many, many more over on the Data-Driven NYC YouTube channel. Thanks again, and catch you next week.