Episode Transcript
[00:00:00] Speaker A: Okay, Julia, thank you so much for joining me. It's been so much fun getting to know you over the years and working together in sort of our parallel universes. Why don't we start with you introducing yourself a little bit and tell us a little bit about your career and how you got into the learning space and where you are now.
[00:00:17] Speaker B: Well, Josh, thank you so much for having me on this podcast. It's a real delight to be here. So, I have been in education and education technology my whole career. I started as a teacher America teacher in East San Jose. I worked for tfa. I went to grad school and got super excited and passionate about how technology could change education and make it more accessible. I ended up going to Google, where I ran the Google Apps for Education team, and then was recruited to Coursera by the founders right when the company was starting, when There was about 10 people in the company, I was the first business hire. It was, like, totally unclear what my job was or what I should be doing or if we even should have a business model at that point, but I ended up staying for over six years. My last role at Coursera was leading the enterprise team. From the very inception of it to when I left. We had about 1300 customers. And then I left to become a partner at GSB Ventures, investing in innovative education solutions. But it was the middle of the pandemic, and I kept on seeing how horrible Zoom schools were all over the news, and I kept on thinking, there has to be a better way to do online learning. Like, we have not cracked this nut, and the world needs a way to scalably learn online.
And the two things that we saw were changing were how people engaged online socially. And then the second one, which has turned into actually, the sort of part of what we're doing is the role that AI could play in creating new types of experiences for the learner and making it easier for the admins. And that was about a year before ChatGPT launched. And so we've been building up limit, really with that vision of how do we make online learning actually work for adult learners, how do we do it at scale and using AI to make that possible.
[00:02:02] Speaker A: Wow, that's an interesting background. I knew. I knew you spent a lot of time at Coursera, but I didn't realize how much experience you've had in the whole space.
So just go back. One more question about the sort of Coursera world as I, you know, I was an outsider observing you guys kind of built a bunch of content with universities, and there was no fee for it for a while, I believe. Wasn't that the model and then there was fees at added for certificates and so forth. Before we talk about uplimit, what's going to happen to all those content companies in your opinion? But you can AI generate stuff like with the flash of an instant?
[00:02:38] Speaker B: Yeah, yeah, I think it'll be, I mean I, I, I think the brands still matter of the, of the credentials and of the institutions and people wanting to mark the fact that they have mastered something and have learned something that is a marketable skill in the economy. I think that's still will matter. People getting the Google IT certificate as a, as an example to help open the door for potential jobs, I think, I think that will matter a lot. But I think that there's a lot of sort of interesting refactoring that can happen as you think about those content libraries as resources for creating backbones of new experiences. And I think it'll be interesting to see what Coursera and the other players do as they re envision the future.
[00:03:20] Speaker A: So Udacity got acquired by Accenture and then I guess Coursera could build an AI version of the whole thing if they decided they wanted to do that. Well, you guys could sell them your platform maybe.
[00:03:31] Speaker B: Yeah. You could pull different parts of the experience and to make it more relevant, to make it more personalized, to make it more engaging, I think the form factor could change substantially in order to drive higher levels of engagement and personalization, make it more relevant.
I think thinking about how it could fit into companies content, you know, you don't need the whole library, you need pieces of it that are really relevant. Yeah. And the credibility still matters and maybe the cred matters even more in the AI world where, you know, knowing where the content's coming from, knowing that there's an authority behind it having that stamp, maybe that matters, right?
[00:04:05] Speaker A: No, I think that it's certainly true in our case. So let's talk about uplimit. I mean, most of the people listening to this podcast know what I how I feel about AI driven learning and how transformational it is. And I really call it a revolution because it's, it, it changes the role of instructional design and gives people dynamic content and many sources of content.
But you guys are taking a very unique approach. I mean, you have a lot of AI in your platform, but I think you're taking a sort of a more focused approach initially. Tell us about what you guys are trying to do because you have some pretty interesting clients.
[00:04:38] Speaker B: Yes. Yeah. So, you know, as I mentioned, our Sort of core thesis was how do we make learning more engaging? And where we started was taking cohort based learning which we knew worked, we know, oh it drives higher level of engagement, we know learners love it, we know that companies use it for their most mission critical learning but it's very difficult to administer. And so we started with making that easy and making that far more scalable than it ever been before.
And so we partnered with companies like Databricks who is running their customer education program. Before us they were capping their live and training and workshops at about 20 people.
Now on a weekly basis they're running cohorts of over a thousand people.
And we've made that possible. And there's a number of different ways, lots of automation.
[00:05:25] Speaker A: I know a bit about it, but for the people listening, how do you scale a thousand people in one cohort?
[00:05:31] Speaker B: It's a combination of a ton of automation around calendaring and things like that, which is sort of less sexy but like extremely important. And then the use of AI to do things like analyze the data, pull out subsets of learners that are struggling, draft interventions and then we keep a human in the loop that's sending those interventions. As we've seen when it's sent from a human, it's far more effective. We also use AI to make the actual experiences within the cohort a lot more engaging. So use of AI role plays where you're practicing an experience and getting a feedback, open ended assignments where you're getting AI feedback, being able to upload a document and get AI feedback in it.
[00:06:05] Speaker A: So in the case of Databricks is, is it for technical training or sales training or what are they using it for?
[00:06:10] Speaker B: They're using it for both internal and external enablement. So the biggest population by far is their customer and partner enablement, but they're also using it for sales training.
[00:06:20] Speaker A: So I'm taking a databricks certification of some kind and I'm and there's a scheduled course with a thousand other people in it, but I don't see those people or do I get a little bit more specific on what it feels like for a number learner to go through this.
[00:06:39] Speaker B: So you join the course, you have a thousand other people, you might ask questions in the discussion forum and other or a human TA will answer questions. There's live sessions each week, there's normally one or two live sessions each week. And that was actually one of the barriers that they had to scaling where in the live sessions there were just way too many questions being asked that one instructor or one, you know, one TA couldn't handle them. But we have an AITA that's in the live session and we're tapped into databricks, all of their documentation, latest documentation we can answer and we know all the questions that have been asked over the last few years in any of these live sessions. And it enables us to very accurately answer people's questions as they're coming up in the live sessions. And that makes it far more scalable. And so I think that, you know, the paradigm of thinking about what's live and then what is asynchronous. When you start leveraging technology in smart ways, the equation just changes. And especially when you're thinking about how do you actually drive efficacy. Like this is not a check the box training, this is training that they need in order to be able to leverage the technology. And databricks obviously cares a lot about that because it's gonna drive adoption for them. And so they care that people complete and that they engage and that they do the projects and complete it. Having live sessions, a certain amount of live sessions, getting personalized nudges, having it be project based, getting personalized feedback, all of that contributes to so very, a.
[00:08:02] Speaker A: Very important sort of educational program.
The students go through these exercises, they're interacting, asking questions, getting answers. Do they know it's an AI or do they think it's a human or they know it's an AI, right?
[00:08:15] Speaker B: They know it's an AI. The place of the. We have humans in the loop for aspects of it like with the nudging. And that's where it is a human that pressed the button. Technology made it possible for the human to manage, you know, a thousand person cohort or multiple thousand people cohorts each week. Um, but it's still a human that's reviewing it and it will be a human that answers it if they respond.
[00:08:38] Speaker A: And then in the live session you get to meet the experts and interact with them directly.
[00:08:43] Speaker B: Yeah, and that's one example. Leadership is another.
[00:08:46] Speaker A: So one more question on that one.
So you're getting all, you're getting massive amounts of data about each person's activity. Does the system really understand how well each person learned this stuff at a more granular level than they finished and passed?
[00:09:04] Speaker B: Yeah, yeah, that's a great question. Yes. And there are some companies that are taking that further and doing really interesting things with it. So one of our customers is one of the top management consulting firms in the world and they're using us for all of their new consultant training and they care a lot that their new consultants are able to communicate effectively. Like that matters a lot for them being effective on the job. And so during their first six months having them practice and do we have them do AI role plays and they have rubrics associated with those and they're getting data to see their progression over that six month period and at what point they are, how effective they are at the end of it. And one of the things they've said to us is they've never had that data before. They've never been able to see in their onboarding of these new consultants what type of progression did they actually make in skills and where are they benchmarked at the end of it. And so and that that's new and exciting and it's.
[00:09:57] Speaker A: So at the end of the program the, the client can actually look through the list of the roster and see what each person's strengths and weaknesses are.
[00:10:05] Speaker B: Yes. Wow. Yeah. Yeah, well they're doing then they're doing additional data analysis on, on the metrics that are coming out from us. But yeah, they can see how people are doing, how they progress, what they're good at, what they're not good at. And so that is now possible.
[00:10:18] Speaker A: And it's your system, does your system dynamically generate learning content also?
Not just in question answering and exercises?
[00:10:28] Speaker B: Yeah, yes. And that's a place I would say like our first version of that we built right after ChatGPT came out and it's proven a lot since then. Like the first version, you know what was generated with the underlying models was not. Was okay but not great. Now it's really good. And I think what's interesting is it's not just content that's AI generated, it's also the role plays are AI generated and the open ended assignments are AI generated and flashcards are AI generated and so you get a complete interactive experience that can be generated by AI.
[00:11:04] Speaker A: And you're doing leadership development as well with companies?
[00:11:07] Speaker B: Yes. So yeah, I would say one of external education, leadership development and sales enablement are the three use cases that we see. We're working with some exciting companies like Sony Interactive Entertainment who's using us to scale their leadership development and make it much more engaging and hands on. Or companies like Kraft Heinz is also using us, Roblox, all for leadership development.
[00:11:32] Speaker A: So one more question about the company and then I have a quite one question about sort of career stuff. Is this a solution where the client licenses the. I know when you started it was more of a handcrafted solution, but are you are you moving to a point where the client can do all this work on their own or do you do a lot of it for them? Where are you in that continuum?
[00:11:53] Speaker B: Yeah, it's a great question. So it is at this point all done by the client. So they use our AI system to generate classes and run them. And then we actually are launching a new product. We haven't even talked about this, Josh, yet. Very, very soon. It's in beta. It's called Sparks. So, you know, our overarching thesis was how do we engage learners? We know cohort based learning works, but what we started seeing our customers do, Sony was actually the one that was the first one that did this and then Procore did this as well, is they started taking our interactive AI exercises and then rolling them out without the rest of the cohort infrastructure to their learners. So as an example, Sony rolled out something that they called feedback dojos, where right before performance season they had a small amount of content which consisted of their feedback model and a couple of role plays, and they pushed it out over slack to all of their managers so that they could practice giving feedback before the performance reviews. And we thought that was super interesting how they were using our technology in a, in a way that was very.
Procore did as well. Procore actually was doing something around AI upskilling. They called it Quick Bites with B Y T E S, which is they very clever names and they were using it for AI upskilling. And so they actually were taking Gemini videos for the Gemini launches and then rolling out a small amount of Gemini content coupled with a role play that they used as sort of a Socratic teacher for people to internalize the content. So it was, we thought that that's super interesting, super exciting. The response from learners was great and very high levels of engagement and realized that that could actually be an awesome compliment to our existing product. And so we have this new product, Sparks, which is currently in beta. Sales enablement is a big use case for this where you can take a product document, upload it into our system rapidly, generate a small amount of content coupled with personalized AI roleplays, and then push it out to all of your sellers.
And so that is. Will be officially launching in October, but is live in beta right now.
[00:13:58] Speaker A: Very impressive. So let me, let me ask you sort of a bigger, higher level question.
And you know, I've been doing this for a long time and you've been doing it for a long time in a different world, but very similar. It's odd to me that There are so few AI native learning platform companies. There's you guys, there's Sana, there's Dispers, there's a few small ones, but there's. And then there's a hundred LMS companies that have AI or skills features.
Why is this, you know, how did you get this off the ground? Is it. Was it. The vision is you've got to hire AI people. You're in San Francisco, so you're competing with OpenAI and Anthropic for skills. I mean, you're right down the street from them.
How do you do it and why do you think there are so few competitors? I mean, maybe there are more competitors that I'm aware of, but I don't see that many.
[00:14:47] Speaker B: Yeah, yeah. So, I mean, I think we got it off the ground because we have amazing co founders, amazing team, amazing set of investors, and I think there's a lot of people that want to build something with purpose and want to build something with a mission. Um, and I think when you think about what AI can do in the world that's good, like education is one of the areas where it can, it can do a lot of good. And the use cases are so clear.
There's really can like transform who has access to skills and what learning looks like. And we often on that team, our two co founders came from Coursera too. The things that we're doing are things that we like could only dream of 10 years ago. Like the idea that you could, that each person could have an interactive role play and actually practice and get personalized feedback. And we, Coursera, in the early days, we experimented with using AI to give personalized feedback on answers. And the amount of engineering and content work that it would take to build that for like one assignment made it, you know, totally prohibitive to actually build.
[00:15:55] Speaker A: Do you think it's like even LinkedIn, they've added a lot of AI features to LinkedIn Learning, but it's, it's sort of small, small role play summaries of existing content. Do you think it's because the bigger vendors have so much legacy content and infrastructure they can't start fresh? Is that, is that what's holding people back?
[00:16:13] Speaker B: What's holding the bigger vendors back from being forward? I think they have a lot of legacy. I think there's two things. One, they have a lot of legacy technology and then two, the talent. Like it's one of our top AI engineers graduated from college three years ago and he has as much generative AI product experience as anyone in the market, right because it's been a, it's been three years and, and it's keeping that talent. Having that talent is hyper competitive because they could go to open AI at any point and, and, and have, you.
[00:16:44] Speaker A: Know, very lucrative $100 million a year.
[00:16:47] Speaker B: So how do you attract that talent? How do you get that talent that has experience and can build generative AI tools? And, and it's a. I think one thing that people at sort of larger companies don't always understand is just how quickly the ground underneath us is moving. When you are building in this space, you, you launch a feature and, you know, customers think it's so exciting, three months later it is old news.
So, like, it's speed as a differentiator and speed as a competitive advantage is, is, is essential. And having teams that are both able to know where the AI market is going so that they know where to go and then be able to move very, very quickly is incredibly important. And that's, you know, different than how when you get to a certain scale, you, you operate differently, you operate in a, in a slower way and you have more checks on things.
[00:17:43] Speaker A: And that's one more question that occurs to me that you would be really interesting to sort of pawn on about the content business.
So as I've watched this all evolve, because I've been doing it since the beginning of elearning, all of the niche content companies, specialized providers of industry information technology stuff, leadership models, all that stuff all tended to what happened in the, the prior model, the sort of, the LMS model is they would build courses and then they would sell them to companies and the companies would plug them into their LMS and their lxp. And it was actually pretty good for them because there wasn't a lot of, you know, proprietary technology needed for them to get their content into a customer's environment. And then the customer ends up with this gigantic library of, you know, 5,000 vendors, courses, and they go through every year and try to figure out what to do with it all.
Where do you think AI is going to go relative to that? When these content companies do they have to build their own AI platforms? Are they going to partner with you? What do you think is going to happen?
Because you come from, you know, really the content world originally.
[00:18:48] Speaker B: Yeah, yeah. Well, I think one of the things that I have thought about a lot is like, what is the job to be, that is being asked to be done by the content companies when an L and D leader brings them in? And the job to be done for the most part is I need to provide all of my employees with some type of training. It's showing up in our pulse surveys that people want development and I need to provide them with a scalable way to access that training and I need it to be easy and quick and to get in there. And content companies will still do that. I think that they're still solving that need and I think they, you know, they still will be solving that need.
I think there is a bigger opportunity for them in what you can do for actual AI enablement that's tied to business priorities that can drive performance.
And I think that's where we're focused, where it's how can you use AI to create learning experiences that actually, you know, transform how somebody works that then would transform.
[00:19:49] Speaker A: So if an up give me, here's the scenario that's going through my mind. So you're an up limit customer, you're building some great stuff and you've got great stuff coming from you guys. And then a vendor comes along, say Franklin Covey or somebody and say, oh, I love their stuff. And they don't know anything about AI. You just have a bunch of, you know, content and they don't want to put it into your platform because they don't know how to use your platform. And are we going to be in a world where the customers are going to say, look, just give me your courses and I'll figure out how to take them apart and put them into my AI platform? Or you see what I'm getting at. What is the model for ip?
[00:20:23] Speaker B: Yeah. Yes.
I, yeah, it'll be interesting. Like I think they're, you still have catalogs and I also would not count any of the, the content companies out. They have massive distribution.
[00:20:36] Speaker A: They can all do the same thing as you guys. They can, you know, Udemy is doing it. I assume Coursera is going to do it.
[00:20:45] Speaker B: But I think it's, it's the, where I'm seeing some of the most forward looking L and D leaders go is being more prescriptive in terms of what programs they're focused on and running and that they think the business actually needs. And those ones, maybe they're working with external content providers and, or maybe, and even if they are, they're likely customizing that pretty heavily themselves to make it the most relevant for their company. So it's, I, I think, I think it's less of a, you know, we need this massive amount of content.
[00:21:18] Speaker A: It's less buying a library and just throwing it out there and hoping people know how to find it.
[00:21:22] Speaker B: Yeah. And then I like, as I've also been reflecting as talking to chief people officer this morning, when you think about what leadership or any of these areas, it's actually not that much content.
It's more about like practice and application. Think about learning a feedback model. There's not that much content in anyone's feedback model. It's not, you know, it's not that much content.
Getting someone to be able to give really good feedback is hard and that takes practice, takes time.
[00:21:51] Speaker A: So we're moving, you know, and I think you're absolutely right. I think we got stuck in this pedagogical dead end of selling people courses because they sort of mimicked classrooms and now we can sort of skip that and let's teach them to do the things that they need to learn and coach them along the way and maybe they don't have to take a four hour course in feedback.
[00:22:13] Speaker B: Yes, exactly. And it's the end that you don't know whether or not people have actually, actually can apply it. A four hour course on feedback where you have no idea whether they've internalized it or applied it back on the job. Like that's where we need to move away from. And instead let's do a 10 minute, 10 minutes, like read 10 minutes and then do a role play where you practice it and then go off and give feedback to your, your direct report and reflect on what you did well and what you didn't do well and submit that reflection and have a complete feedback loop on feedback and, and internalize it. And that's the shift that was not possible before AI, but is possible now. And I think that's like, that's where I think companies should focus. The content is just going to keep changing. The other thing I'll say that it opens up is the potential for, you know, business units or people within teams to, you know, own a little bit more of the experience. Like it doesn't, you know, you don't, you can farm some of this out to your teams and I completely agree.
[00:23:16] Speaker A: I think it's going to be much more distributed and decentralized in companies because this is so easy to use. Yeah, but it's a good point. So for somebody like you, Julia, who has a real background in this whole domain, you can go back to basic principles of how do people learn and how do we teach them this. And as opposed to assuming that if I buy a bunch of content, everybody's going to learn everything.
[00:23:36] Speaker B: Yeah, yeah. And it's, and again, it gets to like, what is the what am I actually trying to do? Like, am I trying to check a box and, like, give everybody access to catalog so that they feel like they have opportunities? Or is there real skill development that I'm trying to drive in my organization? And if it is real skill development, content doesn't get you there. And so you have to go further than that. Fortunately, AI can get there and you don't have to have that much content.
[00:24:06] Speaker A: Julia, thank you so much. It has been such a pleasure to get to know you over the years and work together and watch what you guys are doing over there from the beginning. It must be really fun.
[00:24:16] Speaker B: Well, we really appreciate all of your advice and guidance and enthusiasm over the years. So thanks so much. And thanks for having me today.
[00:24:24] Speaker A: You bet. Okay. Thanks a lot.
