The Potential of AI in Revolutionizing Personalized Tutoring with Brent Beck

[Automated Transcript]

Deep: Hey there, I'm Deep Dhillon, your host. And today on our show, we'll be talking about AI's potential in revolutionizing and personalizing tutoring. Our guest today is Brent Beck, CTO of Littera Education. Brent's got a proven track record of innovating industry changing educational technology, making him a leading voice in integrating AI with education.

We're going to dig into the potential of AI tutors and academics and emotional well being and contemplate a future where AI chatbots become pivotal and like super helpful for academic advising throughout a student's educational journey. So, yeah, so Brent, thanks so much for coming on. I'm really 

Brent: excited to have you.

Thanks for having me. It's gonna be a cool conversation. 

Deep: I'm super excited. So maybe start us off today Tell us, um, some innovative ways in which Littera or other ed tech companies are like leveraging AI to, you know, tailor some educational support for individual students.

Brent: Sure. Um, you know, in general, Ed Tech, I don't think it's quite grasped how to how to use AI.

So I would say, are we innovating out there yet? Some are Khan Academy is doing some cool stuff with how they're going to use AI to show people learning paths and as they go down different ways to help them pick up other things. But I think in general, you know, we know education is always slow. To grab onto technology.

Um, sometimes for good reasons, sometimes just because that's the nature of that education space. Um, illiterate. We're looking at all different ways of how I will get used. Um, you know, data for us is extremely important and translating that data and doing it quickly and doing it with some some real skill.

I think it's going to be extremely important, you know, explaining to a district. How their tutoring is going, what they can do better is the content working, being able to give them enough evidence to go back for funding so that they can even fund some of these programs. And I think that's where AI is going to play a big role in the next, you know, let's say 24 months is is more, uh, you know, uh, predictive analytics, um, predicting how students doing in in these tutoring sessions.

Are they growing at the rate they should? Are they growing at the same rate as others? Um, it's going to help us. In areas where, uh, you know, tutoring used to be a thing that only rich people could do, uh, and the people that needed it most didn't have access to it. I was going 

Deep: to ask you like, what's driving the need here?

Are we finding that, that, you know, one to 30 student to, uh, teacher ratio or teacher to student ratio is just. Not leaving enough time and it's not activities for personalized attention. Like what's what's kind of driving it?

Brent: It's absolutely not in in high dosage. Tutoring is really the answer for that.

And that's where what Littera has really that's where we shine. That's what we do. We look for a 1 And we're doing some research with, uh, the Bill and Melinda Gates Foundation right now to understand if there's a drastic enough difference between one to one tutoring and one to three tutoring to make it worth the cost.

Obviously, if I'm one tutor and I can tutor three students at the same time, you cut your costs down quite a bit in one on one. We're not seeing a huge difference. Um, one to three seems to be mostly just as effective. And so what this allows us to do is we look into districts. Um, like, like some of the districts across New York City and some of the districts in L.A. and other places, and we see, okay, these students are struggling, not just pandemic struggling, they've always been struggling, the pandemic just exposed it. And how can we put a tutoring, um, tutoring program in those areas? That's cost effective that they can afford and they can and they can use across the board.

And I think I will help us in some of those areas. Get answers more quickly. You know, at the beginning of the pandemic, everybody got scared with how are we going to educate? We all went online. Nobody knew how to do it. And millions and billions of dollars were wasted on different programs, just trying to figure out what what would work tutoring was very similar.

A lot of companies went to text based tutoring, audio only tutoring using tutors all over the world. And we found that was that like 

Deep: them sort of filling in gaps that yeah, what is it about that 1 to 1, 1 to 2, 1 to 3 ratio? That changes the dynamic from a larger setting in the classroom. And you were talking about COVID, um, bringing up a lot of sort of making it obvious the need for tutoring.

Brent: The problems have been there, right? The problems have been there for a generation that at risk students, students that are in poverty stricken areas, highly minority areas, they've always been slightly behind those in districts that have a lot more money or that have parents to have a little bit more money to pay for tutoring.

Or to be able to help the students themselves, right? We've always known it was there. COVID, COVID exposed it and, and made it worse, right? Because now they all went home. Um, many of them just disappeared. 

Deep: You know, in Seattle, we lost like 8, 000 students. Just, yeah. 

Brent: And nobody knows what they've been doing for those months.

Nothing. 

Deep: Well, nobody knows where they are to this day.

Brent: And even worse, you know, it is. And even worse during that time is that their parents. Were the essential employees to different places like grocery stores and others that had to be at work. So there, these kids were home by themselves, like do your own education.

How's that going to work out? So now they're even further behind than they already were. It's also much harder for those students to catch up. They don't have the resources. They don't have. Um, the support system to catch them up. And so we're going into districts and we're and we're showing them ways they can do that.

I mean, we've had districts where they increased the reading 200 percent in one quarter of tutoring. Now our tutoring is three days a week or more, um, is what we hope for one to one is great. One to three seems to still work. We try not to go past that. We've had districts where 86 percent of the district increased or caught up a gap bigger than six months.

In just one session of tutoring, just been a session in tuning, you know, over like what, three months, there's this huge need to step in and help. I don't think it's replaced the classroom, right? It's not possible for us to have a 1 to 3 ratio of teachers across the country. We barely can do a 1 to 30 right now and keep teachers in there.

But if we can augment that with tutoring, I think we can make a difference. But it's costful. 

Deep: So now what's happening there is it is it like students in a larger like classroom context can space out and they can just fall through the cracks and nobody notices, but in a smaller social setting, they can't just space out like there's more holding them into 

Brent: the conversation or something.

As a teacher, I'm looking at a classroom of 30 students. I can't keep track of who came in tired. Who came in, you know, the same clothes they were wearing yesterday. Something's going on at home. It's a much harder thing to do as a tutor and all of our tutor is done over video. So they see our tutors faces and they see the kids faces.

That's a requirement. The other thing is our students always have the same tutors. So they meet with them three or four days a week. They meet with the same tutor every time that tutor can see that student. They get to know them. They learn their they get to know their learning style. They get to know their their emotional state.

They get to know their, um, social behavior and how they act and how to interact with them to get the best results. And you can do that in a smaller group, right? It's much harder to do in a room of 30. Plus, if I have 10 students that need help in a room of 30 that need very particular help, That's much harder than if I have one student out of three.

You're, you're going to like try 

Deep: to group into common questions and then stack break them by the one you think most people are affected by. And none of that addresses the attention drift issue, you know, I mean, and, and, you know, it's, it's interesting, like one of the things that I've noticed in the States is.

You know, tutoring has sort of had this like connotation of like, Oh, you're falling between the cracks. So you need tutoring, but you know, I mean, like growing up, like all of my, you know, cousins in India, they all had tutors and it was seen as like mandatory for for like true academic excellence. So it was never optional.

If you wanted to go really far, and it seems kind of backwards here where it's, it's sort of only seen as a catch up mechanism. Are you seeing Any interest in the districts at the other end of the spectrum where you know the hey, there's students that weren't falling through the clerks before, but you can like optimize and get them 

Brent: learning even more, or is it we, I think we're seeing interest but it's always, you know, does the money follow that interest do they have the money in order right now.

They get a limited pool of money. Obviously they're focused on the students that need the most help. And so, and so that's where we see a lot of it. Um, the students that are getting by are getting by and, and so they don't spend as much. And it, it's an interesting dynamic. So my, my wife is a, a child and family psychologist, and she would say the same thing of, um, counseling and, and, and getting that kind of support in the US is once you have problems, you go get a counselor.

Right? And her argument is always. Everybody needs it. You should be doing it ahead of time. So the problems don't ever become a problem. It's no different than tutoring, right? We wait till the students so far behind that we have to throw all these resources to catch them up where honestly, if we just threw those resources to everybody at a more even rate, probably spend the same amount of money and time over a longer period of time and you get better results.

Deep: So maybe walk us through. Like your, your platform or your approach a little bit from the eyes of a student. Like what, how would they engage with your platform? Does the teacher kinda select them, pull 'em out of the class and, you know, and, and, and is it always virtual or is there a physical presence to, and then is it directly aligned with the curriculum that.

A given teachers marching through or is it like they can move at their own individualized pace and there's assessment involved like, you 

Brent: know, maybe walk us through. What's interesting about our platform and I think makes us stand out from a lot of our competitors is that we are very much a SaaS platform with a lot of plug and play involved, meaning that you have your own content.

You can use that in our platform. You have a specific curriculum you're using nine times out of 10, we're probably already partnered with that curriculum and we can use it directly in it, or we can, we have other ways to bring your curriculum into our platform. We have our own virtual classroom that we built in house with our own whiteboard and everything.

We can even plug and play with that. Say you have a school that says, well, we really like zoom and we don't want to leave zoom. We can plug that in. So what we go into a school and it can be very customized. Where are you guys at? What's your curriculum? What is the kind of classroom you want to have? And then we set that up.

So so I think that's what makes it unique in the play. It can be in person so they can do all of this in person and then use a mobile device to record what happened. So basically, I open up my device and I say, okay, I did my session. This kid paid attention, did great, ready to move on to the next section and add those notes in.

That all gets captured then in our SAS platform. And then what we do with it is we can, we can give them reports out on it. We can give them, you know, analytics and they can start to look at how's the program going, even in person. We can mix that in person and virtual. So, um, and they can do that during the school day.

They can go into a lab during the school day, throw headphones on, meet with one of our virtual tutors. Uh, they're all US based. They're all trained, uh, and, and ready to go. Like these are, these are legit tutors. And they can go in and they can do that, uh, that way during the school day, they can do it after school or before school as well.

So it's, it's a very open customized to the school. And I think that's important because every kid is not the same and every district isn't the same. They have different needs. They have different, uh, hurdles to get over. And so we try to make the platform as. You know, easy to use and plug into a school district as possible.

They have so much technology they have to try to use. We want to make ours as easy for them to plug into. 

Deep: So this is an AI podcast, so maybe we'll shift gears slightly, but maybe walk us through the data from which you might be building some models today, and then maybe the data that you're talking about in the future.

So for example, There's, it sounds like there's obviously the video recording, but there might be, you know, other data like, hey, what part of the curriculum, you know, like what, what topic, what subject, um, who's the tutor, and then there may be like assessments. It sounds like they might be taking. Solving problems like in the platform as opposed to, you know, I don't know, holding up a piece of paper with a math problem or something like, maybe walk us through all that.

Brent: They're doing all of their work in the platform. So they have a whiteboard, they can do math problems in there. They can draw on it. They can bring in PDFs and have them read on the screen. All of that is built in. Um, so w you know, I would say first and foremost, once in the classroom where we can capture anything and everything, if we want, right, we can, we can capture who's talking the most.

Who's not talking? We can, we is the tutor talking too much. We can capture all that information using analytics and using the, using our, our backend. Um, and then we can do something with that. We're, we're, we're, we're, you know, not quite there yet. We're not, we haven't added AI in yet. It's something that we are working on right now and, and we're probably gonna be partnering with somebody to do some of that, but.

All of the data we're capturing already. We know what they're doing in the classroom. We know how often they're meeting. We know what subject they're meeting on. We know what tutor they're meeting with. The student at the end gives their feedback on how they felt it went. Even the young ones. We make it very simple.

A little, you know, smiley face or a sad face. Pick one kind of thing. Or for, you know, the more advanced students, they might have a row of stars. So we capture that data. We get feedback from the tutor themselves. So the tutor comes out and says, Hey, you know, Molly over here was not, not really engaged, didn't really talk, struggling with her, doesn't seem to be getting the reading.

And, and so we, we capture that each week. Does that change? And then you ask about the assessments. Well, the assessments come on the, on the school side, they'll give the assessments with their content there. And then they'll compare those results. And what we're working on right now is getting an ingestion back of those assessments so that we can add that into the reports that we give to them to show them how they're progressing and how their students are progressing.

So there's, there's, there's a lot of data we're capturing right now. We're just trying to figure out what is most important to the districts and how do we give them, you know, a direct feed because it's overwhelming if they just look at the data in the raw. Yeah. I mean, 

Deep: I imagine there. Kind of biggest question is like, what's the efficacy look like?

You know, and, and like, are you able to actually, you know, increase ultimately like scores on standardized tests or something? That's my, my guess, 

Brent: but a hundred percent, they'll do their assessments and we'll look at their scores and we're even looking at like how we would bill and we could go into a school district and say, look, we'll only bill if you get results.

So completely results driven and and so that and we feel comfortable doing that because we know the platform will do what it's supposed to do as long as we have the right curriculum. It's also been interesting because I think what will happen. Not I think I know we're already seeing evidence of it. Is that we're going to start to be able to be able to say, Yeah, this curriculum is actually working.

Or, you know, this curriculum is not because now I can compare across classrooms, right? But this classroom is using this curriculum for reading, and they are doing using the exact same tutors, the exact same methods, everything, but they've only caught up on a month gap. Whereas this classroom over here is a six month gap that they've cleared.

So we're gonna be able to start to tell districts even is their curriculum effective in what they're doing. And we can see the effectiveness of our tutors and weed out the ones that aren't effective as well. 

Deep: Perhaps you're not sure whether AI can really transform your business. Maybe you don't know what it means to inject AI into your business.

Maybe you need some help actually building models. Check us out at xyonix.com, that's X Y O N I X dot com. Maybe we can help.

Deep: So you've got these like lower level signals on the tutor. Uh, you know, like you mentioned, um, talk time ratios between tutor and students that I imagine is probably revealing. And then, you know, on the tutor, I don't know, there could be all kinds of stuff, maybe tone of voice.

Is a signal, maybe their tenure, you know, how long they've been doing this. Maybe their prior track record plays a role. And then on the student side, you know, um, you, you have a bunch of attributes. You've got their scores on like regular assessments, presumably, and on specific modules. And then you have like all kinds of demographic info too, about students.

So you might know if they're from a low income or a higher income. Bracket and then you've got various curriculum. So it imagine you could, you can leverage some ML in a lot of different capacities, like, you know, on the, on the tutoring side, I think you mentioned like they're writing a free form natural language characterization of the student.

Maybe some of them aren't doing it. Very well. Maybe some of them are just doing the bare minimum and putting in a half a sentence or something, and others are actually taking the time. So that alone, like a feedback loop there makes sense. Um, 

Brent: and then the quality of that information back to the teachers, right?

Because the teachers in the classroom are the ones actually teaching the students. We're just trying to help them keep pace and we could take that information, feed it back to the teacher in the classroom. And then they know, Hey, what I'm doing half of my classes in tutoring right now and they're not getting it.

I need to back up. Yeah. 

Deep: So like some kind of, I don't know, near real time dashboard or something for, you know, so that students so teachers know, like, Hey, 33 percent of the students passed this, uh, you know, quiz or whatever. Um, so I got to back up and cover material better or something. Yeah, I mean, it seems really right for machine learning, right?

Like People change their behavior when they know something's being measured. It's it's. And so if a tutor knows that they're free form assessment at the end of a session is being measured because you give them a score. on what they wrote and maybe on different attributes, like maybe you're giving them a score based on the thoroughness of it, based on, you know, the, the utility of it or the action ability, if you will, uh, you know, like, cause the student be able to do anything, this information.

And those are the things that like, you know, large language models are pretty darn good at. If you've given it some training data and set up some, some custom. 

Brent: Like editorial guidelines and they proven they've been around for a while. They're proven, right. This isn't, this isn't new. Like generative AI is new.

And I think that's an area we still have to figure out what that means and what it does. Machine learning's not, it's been around, it's been used. And I think that that is ripe for education right now. Um, they like to have stuff that's got a track record, , well, even 

Deep: on the generative side, there's, I mean, the generative models are fairly new in the last five to 10 years, but it all depends on how the user experiences.

Is like presented, right? So like if it's presented to the tutors, so there's two parts of it, right? Maybe some, some classic ML to sort of project for a given tutor, like what level of boost you predict that tutor is going to have for this student population. That's maybe more classic, but to give them some near real time feedback on the assessments they just wrote up.

That's something that, you know, you can feel pretty. You can just basically scan a few thousand of those and be like, okay, we're cool with you giving this to them. 

Brent: Cause it's going to no, a hundred percent. Yeah. And I think, I think figuring that out is going to be, it's going to be an interesting next couple of years, I think, in, in, in our space and how we use it.

I mean, there's no doubt AI is going to have an impact and we're going to use it across the board, you're going to have to, I sometimes wonder, you know, when I hear. People talking at conferences and coming out on like AI is going to solve all of our problems. It's going to fix everything. It's going to be the new teacher.

It's going to be the new tutor. I'm always cautious to say, look, technology is a tool. It's not an answer. And when we start to use it as the answer is when we really mess up. But when we use it as a tool to help the teachers do a better job to help the analyst analyze stuff a little better. That's where I think it really becomes a real help in, especially in the education space.

And that's what I hope to see here. I hope to see AI start to play a big role in helping us understand the data we're looking at. You know, not that a group of high end analysts and researchers can't figure this out on their own, but using AI, they can do it a heck of a lot better and a heck of a lot faster.

And, and that's where I think this is going to play, play nicely. Yeah, I mean, you could imagine, 

Deep: like on the assessment side, you've got the tutors, but like up the normal chain, you've got teachers assessing their classroom. Then you've got principals assessing a set of teachers, um, relative to maybe their grades and, and, uh, the grades they're teaching.

And then you've got, uh, you know, district. Superintendents assessing principles and all this stuff kind of like ripples up to ultimately to like, you know, national level policymakers and and, you know, having all of that dashboarded and in place. And then weighing in on how is this teacher's performance for this student population relative to peer benchmarks?

How is this principal? Same question. How is this district? Same question. And then you can use the machine learning to kind of flip it and try to understand, well, what attributes Are leading to poor performance or better performance. So then you can start to like help policymakers and like folks to hire up make more strategic decisions.

I mean, it's Yeah, I 

Brent: mean, a no brainer as we start to get ourselves in. Um, you know, when we started selling, we were selling more into the district level and a school by school. Which we found was, it was working fine, but it wasn't really giving us the results we wanted. We started to work with what we call tutor service providers.

These are overarching agencies. They might be for profit, they might be non profit. And we've partnered with people like Carnegie Learning and Edmentum and others that are a big overarching content builders. But we've also partnered with non profit state organizations. And I think that's where what you're saying is a perfect example where They're going to be tutoring in schools across their entire state in poverty areas in wealthier areas are everywhere.

Now we can give them that data from every school and we can put it together in one place and go, here's what's effective. And what's cool about that is not every district in these States uses the same content. So not only are we assessing. Is the tutors working? Is the tutoring program working? Is the teaching working?

Is the principal working? We're also looking at the content they're using. What's most effective, you know, at teaching a, uh, a second, first grader to read? And, and things like that. And, and, you know, I, I have a, a second grade son and, and when he comes home, uh, He, you know, I look at some of the stuff he's doing and it's mind boggling to me, even just like his math.

I passed three levels of calculus and I never actually went to class, but I can't do his math homework because of the new way they do things. It can 

Deep: be quite odd. Yes. Pointless and quite amazing. Yeah. 

Brent: And so it's just like looking at that as like, okay, is that really effective? You know, let's compare that.

It's not using it. I don't think we've done a lot of that. It's all been kind of. You know, well, it seems like this state's doing it this way. This state's doing it this way. But we're not comparing everything. And I think AI will let us do that very quickly. 

Deep: Yeah. I mean, I think having, you know, like at the national level, there's some basic data that gets gathered, like mostly like performance on standardized tests.

And then that is sort of like one thing, but To really have a lot of these decisions historically, you know, I think have been made through a combination of politics and expert advice. You know, that's maybe loosely backed in literature or maybe even backed by the literature, but not in a political sense.

Bottoms up data centric manner like you're describing here, 

Brent: right? Like this is taking everything into consideration, right? It's not taking, you know, how many of those kids actually got a full night's sleep and breakfast that morning. It's not, it's not taking into consideration the full picture of what these students are going through because look, their time in the classroom is one thing.

It's completely affected by everything else in their life that we kind of ignore for the most part. 

Deep: Yeah. I'm wondering what, what do you think is like the evolutionary path of You know, of how this stuff gets rolled out, maybe from your vantage in your and Littera's vantage, like, do you, where do you start to, like, introduce some of this, like, higher level analysis?

You know, do you start on the assessment side? Do you start on the tutoring side? You know, do you start start in the social kind of glue side, like kind of helping the the social forces where students feel better, you know, interacting with their tutors? Like, how do you see it like today and tomorrow and a year and five and ten years out?

Like, how does it march 

Brent: forward? It's a good question. I don't know that I know the answer yet. I don't think anybody does. I think we're still figuring it out. Um, if I, if I had to guess right now, We're going to start with looking at assessments that are built by the same people that built the curriculum right looking at the curriculum assessments like not the standardized testing that happens over a year.

I think that's great. That's important. A lot of schools care about that because that's how they get their state funding. But if we really want to know, is this kid learning? Are they, are they keeping pace? We need to look at the curriculum they're learning from and assess that, right? Assess like, uh, in the moment, week by week, month by month, not over a year's time.

I think that's where we have to start. And we have to set some benchmarks of, okay, this is the progression. This is where this is, where this, this child's heading and then starting to look at the lie after that. Now we've got a benchmark. Now we know what students are progressing fine and which ones are.

Now the question is why, what's the difference between them? If they have the same tutoring program, they have the same teachers. They're in the same school system, the same curriculum. But they're doing two different directions. Why? What is, what is the piece we're missing? What's the information we don't have yet?

And I think that's what we'll start to figure out as we capture more and more data and be able to present that back to the districts to say, here, here's our findings. Here's what we're looking at. And I think it's going to take some expertise. I mean AI is great, machine learning is great, but it's still only as smart as the people creating it.

And at some point we have to figure out, like, are we feeding it enough information to give us the right, the right answer? Um, you know, I just read an article, uh, the other day, uh, that talked about how, uh, the biggest problem with AI is it can be completely skewed by the data you feed it and, and how you teach it.

Absolutely. Yeah. And so I think in the education world, that's my biggest worry is, are we going to feed it the right information? Are we just going to continue the same skewed education system we've had for generations because we're still not feeding it the right information? 

Deep: Yeah, I mean, you know, we build these kinds of systems all day long.

So, you know, I run an AI consultancy. So we Bad data equals bad models, and it's pretty simple. But in your case, because you're working across curriculums, it feels like you need some kind of normalized assessment of your own that is not curriculum dependent. Is that how you're thinking about it, or are you using each curriculum's assessment, but then I guess you're in a bubble per curriculum to sort of 

Brent: assess?

You kind of you do kind of have to be and I think even regionally. I mean, it doesn't I think there's some things that we can gather when we look at it nationwide. And that's great. Right. But it still doesn't tell the whole picture because students in New York City and students in Cincinnati, Ohio.

They're not the same. They don't deal with the same life stuff. They don't have the same. Nothing for them is identical enough for us to just compare apples to apples there. So curriculum differences. It is. You know where they live, differences, poverty, differences, all those differences have to come into play and we have to look at all of them as as we try to assess these students and where they're going.

So I think The biggest problem is when we start to get up too high and try to look at the data and try to generalize it too much. I think it does need to be a bit local, a little bit hyper. Then once we figure that out, once we figure out those different pieces, then I think it's a lot easier to come up here and go, okay, how do we assess across?

How do we know that this curriculum is working versus this curriculum? There's gotta be, I think that's where your standardized testing comes in. That seems to be a little bit more general across the boards, district to district, at least within a state. Um, nationally, I think it's going to get more complicated.

It's really hard to compare an Ohio to a New York or a New York to California or California to Texas or Florida. It's very different. Their education systems are different. Their standards are different. Uh, their textbooks are different. Yeah, I mean, 

Deep: maybe, you know, when you're at the more granular level, right, like you're looking, let's say you're within one curriculum and let's just take math, for example.

Um, and you're within, you know, Singapore math or Saxon math or something, and and they have their own quizzes and their own like kind of assessment tools and their own grading standards, math is math. And at the end of the day, all of those curriculums are going to be assessing whether or not you can add some numbers together or deal you with your subtraction or division or whatever it is.

It seems like you could take the the assessment tools and kind of categorize them based on what it is that they're actually assessing. And from that vantage, you can't necessarily normalize scores across curriculums, but within curriculum, you could sort of assess The kind of progression, but you might be able to like you might be able to actually go down to the actual quiz that covers addition and subtraction across the curriculums and say, yeah, these are all comparable.

And a score of seven on this curriculum is equal to a score of six on that, which is equal to a five on this, you know? 

Brent: Yeah. And I think some of that, you know, we we've, we've done a research project with like the Bill and Melinda Gates foundation. I think those kinds of overarching research projects are going to help figure that out.

And they're, they're looking a lot at. Math and reading. I believe are there two big things and I think those kinds of nonprofits and coming in and saying, okay, how do we how do we find a standardized to figure out what's working and what's not, you know, that that's that's how we're going to figure that out.

That's how we're going to answer those questions. And, and we're going to work with experts that know how to do that. 

Deep: You're listening to your AI injection brought to you by xyonix. com. That's X Y O N I X. com. Check out our website for more content, or if you need help injecting AI 

Brent: into your organization.

Deep: So, you know, kind of back to the tutoring side a little bit. Are you looking, for example, in just at the tutoring session level? Like, so, okay, given that a tutor is talking to three students, do you have a, an approach to figuring out how to make that itself, like within that constraints more effective?

Brent: Yeah, we're, you know, we're working on it right now. Um, you know, we're, we're a three year old company. We're, We're growing, um, very, you know, nicely right now, but there, there's still a lot, a lot that we need to do. And I think the next stage for, for me, the thing I'm excited about and the thing I'm kind of putting some strategy around is how do we, how do we start to assess tutors without having to have a live person on every session, assessing them and writing notes using AI?

How can we have somebody give them feedback? And I think live feedback sounds great for us. Uh, at least for me as a non tutor, um, you talk to tutors are like the last thing I need is a distraction of AI popping me up saying, Hey, stop talking so much. And then I get off and I'm not, you know, so I think our first step for me would be, could we deploy?

And there's companies out here doing this. I don't think this is something we're going to create from scratch. But could we deploy something in our virtual classroom that monitors everything going on and then afterwards gives feedback to the tutor about language, about tone, about how often they talk versus others, about getting the other kids to interact a little bit more, or did they interact, did they talk, or, you know, calling out, like, sometimes in the moment, you don't realize that out of your three students, this one over here hasn't actually asked a question the entire time.

Even with three students, that can happen. And so I think when you deploy AI in there, it'll remind you later, Hey, by the way, next time you have one of those sessions, Johnny over here said three things the entire time, try to figure out how to bring him into the conversation a little bit more. And so I think that, that to me is going to be a really interesting spot to deploy AI and use it to help those tutors just become better tutors.

Um, and then obviously, it's going to tell us something. It's going to tell us tutors that are effective. It's going to tell us tutors that are not effective, that we maybe either need to train or move out of those roles. 

Deep: Imagine there's, you know, there's humans who do this really well, right? Like, I think principals play this role where they go, they sit in with a new teacher, um, they watch, observe, and then they come back with like, you know, concrete feedback, uh, and actionable steps.

So it seems. Like the first step to building a more automated system for, you know, tutor feedback on a given session or set of sessions is to start capturing what it is the expert humans do in that context. So maybe gathering, gathering a few experts, having them watch a video of a tutoring session and like, what are the kind of different elements they talk about, you know, like talk time is one, but there's probably like a whole vocabulary and approach and like methodology.

And, you know, you can do a lot with transcript analysis too on the AI front, right? Like, so if you've got the transcripts and you've got the speakers diarized, so, you know, like, you know, the voices of, like, you got three students and a teacher, I think that would be fairly straightforward. Maybe you confuse the students a little bit because their voices are in very similar spectral ranges, but the tutor shouldn't be, like, there should be a lot of, 

Brent: actually interesting enough, the way our platform works is when you're talking.

It knows who you are because of where you're from. Oh, great. So you don't even have that problem. Yes. We don't have to worry about figuring out who's who when we're, when we're talking about on video like this. Now, in the classroom, you know, if I had AI in my phone, I'm going to set it on the table. That's where it gets a little bit harder.

Like, okay, which student was that kind of thing? We just have students talking. So I think that, that kind of using AI in that way might be a little more difficult in person. But when we're talking, your feed is your feed. I know when you talked, I know what you said. I have your name attached to it. Yeah, I mean, everything that we need is there.

Our next step now is, okay, how do we deploy the AI to actually use it? And we've got, we've talked to several companies about it that are trying to do this. They're not very far along. The thing I always find interesting is everybody has talked to me like, Oh, I hear AI is just solving everything. It's doing great things.

And I'm like, you know, honestly, I haven't really seen a lot of that yet. I mean, it's, it's there where we're starting to look at it, but. Using it in real life is very different than, you know, asking a question on chat, GBT and having to write my board deck for me, you know, you'd 

Deep: be surprised that it's definitely different, but you'd be surprised at how good GPT for is at providing feedback.

If you're prompting it, well, yeah, because we, you know, we, and it's not about like one prompt, one that solves all prompts. It's like. You know, we build a lot of systems where we're orchestrating across hundreds of prompts that are themselves built off of a lot of kind of more complexity, but, but it's amazingly good at if you give it a set of attributes, like, so let's take the tutoring context.

I'm guessing there's like specific things that tutors should be doing, like asking. You know, asking questions, one axes, like, like, like if you imagine a report card against a set of attributes or a rubric, you know, one of which is like, are you successful at getting students to kind of partake? And that's probably a big part of the assessment.

I imagine. 

Brent: Yeah. Yeah. No, you're right. If you 

Deep: ask the, you know, if you go to GPT four with the transcripts and, you know, of course you have to massage and manipulate, but if you're saying like, Hey, Um, I want you to provide specific feedback on, um, how this tutor can, quote, increase, uh, engagement, what specific, with specific examples in their conversation, um, that's like one attribute.

And then you can do that across all the attributes and you'll wind up with like really solid feedback, which will probably, in my experience, I think you'll, you'll be at high quality principle level feedback in a very short 

Brent: order. Yeah, it's 

Deep: better than most humans at most things like, 

Brent: especially at the speed at which you can do it.

Right? I mean, it's, it's in the moment and, and, you know, and again, like, go back, go and taking this all back to what I originally said, which is. We want to make sure tutoring can happen in every district, regardless of how much money that district has. Right. And in order to do that, you've got to make it cost effective and it's not cost effective to have a live person overseeing every single tutoring session, trying to figure out if the tutoring is doing a good job or the students learning, you know, and there's not enough staff to do that.

Um, you know, we do provide tutors. Um, we have over 1000 tutors that we can provide, but yeah. Districts also like to do their own tutoring and that that's their end goals. They want to build a tutor their own students. And so we're trying to make it so they can do that. And if I can deploy AI to help them where they don't even have to have somebody, you know, overseeing these and and helping to teach their tutors to do a better job.

I mean, it's a win for everybody. So, 

Deep: yeah. So it sounds like you're looking at feedback to tutors, but there's also probably like on ramping of tutors where, you know, where you're bringing them up to speed, yeah, where you're teaching a lot of methodology and technique, and so that would make sense to plug in some, some capabilities there as well.

Yeah. 

Brent: Like some practice sessions, right. Having them practice with somebody and get feedback there before they even tutor their first student. That kind of thing. And I think, you know, another one, another thing I'm really interested in is, is virtual reality and augmented reality. And, you know, years ago when it first came out and I was getting into it from a gaming standpoint, it wasn't ready for mainstream.

Now you throw AI in there. Could I throw an AI headset, you know, or a VR headset on a teacher or a tutor, have an AI student, you know, replicating the real estate in the classroom and have them practice. On that, you know, or in a situation like this, there's a couple of pieces of software out there where I can sit and have a conversation with what looks like a real person, that's an AI generative person, you know, using, using an engine and, and I can practice, you know, my tutoring on that person to get, to get better at it and get some feedback on how I'm doing.

And, and basically just teaching teachers and teaching, uh, tutors to help them do a better job and be more effective. Yeah. I've got a 

Deep: friend of mine who years ago set up a VR company for training surgeons and, uh, yeah, they, they, they're, they're actually quite a decent size, but like a lot of surgeons go into like robotic and laparoscopic surgery.

Um, it's too expensive to have them. Obviously, you can't have them practicing instead of humans, but even in like pig carcasses, it's too expensive. Yeah, but like in a lot of what they're trying to train them is like the physical manipulation techniques of the robot. But yeah, I mean, you can imagine, you know, a lot of a lot of capabilities that you could evolve over time to like optimize the training of the tutoring itself.

Yeah, and how they talk to students matters, you know, like the tone they use the matters, how often they interject matters, how often they, you know, and also just like their grasp of the materials matters, but, you know, those things are like a bigger deal at the higher, you know, grades and 

Brent: yeah, or just, you know, like, I think you had said it earlier, but asking questions.

Getting, getting the student to think about it rather than just telling them, okay, this is what you did wrong. Do it this way. And we're asking questions. Okay. Do you think that, that what you did was the right way? Or do you think there's a better way to get to your answer or, you know, getting them to think and, you know, critical thinking is one thing.

You know, what I see is missing in a lot of our younger younger generation right now, coming up through education. Um, you know, they're supposed to get that in college, but quite frankly, they should have a long before college so that if they don't go to college, they have that skill. And I think that's another area of like, how do we teach them that in these, in these kinds of sessions?

Um, yeah, and 

Deep: there's also probably like some subject specific kind of more advanced systems that, you know, need to evolve. So, for example, providing feedback on essays and grading essays and natural language content is fairly straightforward with these large language models, but doing so. In an algebra class where you've got symbolic manipulation.

Um, that's not a solved problem there. You know, you have to like combine some symbolic. I mean, the actual symbolically the symbolic math programs, you know, like Mathematica are quite evolved and solved, but communication of that to a student in context is like a whole other 

Brent: game. Yeah, it is. It's tough.

And, you know, even reading, it's one thing for me to like, Have a child practice reading to an AI and the I came back. Okay, you know, try, you know, sound this word out. You got this word wrong. Whatever. It's kind of straightforward, right? The word to the word. But yeah, you get into math. You get into some of the science you get in some of the other stuff.

It gets a little more difficult. 

Deep: So one of the questions I have for you is, you know, we've got a lot of listeners and viewers on the podcast that are, you know, they're looking at their businesses that, you know, maybe education based, but may not maybe in very different domains. But they're, they're all trying to ask kind of the same questions.

Like, how do I assess how AI can help me and how do I take that first step getting some machine learning or AI systems into my product? What kind of advice, you know, would you give those folks about pulling in some AI and ML into their products and how to think about it, you know, and how to think about it, maximizing the return on their investment, I guess.

Brent: Yeah, it's a great question and something we actually have been talking a lot about within Latera even and with our board and others. It's if the first thing I tell everybody is don't go out and head down the A. I. Road just to head down the A. I. Road, right? What problem are you trying to solve? What is the area where you need help?

Either? It's just too costly for you to solve the problem. It's given. You don't have enough people to solve the problem. You don't have the right people start there. Once you just figure out what your problem you're trying to solve is, it's much easier than to go out and talk to some consultants and talk to some companies and folks like yourself to say, Hey, here's the problem I'm trying to solve.

Has anybody done this yet in AI? And if they haven't, that's where you get to experiment and you get to be the first one to try to try to conquer it. But probably nine times out of 10, somebody is At least attempted to solve the problem whether it's solvable or not. And so the thing I always encourage people is start with your problem.

Figure that out first. Um, too many companies, you know, saw all the fanfare on chat, GPT and others and immediately went out there and said, Oh, we got to have a I to keep up like, but do you, you know, I don't understand your business model yet. So do you what's the problem you're trying to solve? 

Deep: No, I think that's great advice.

Like, Okay. The thing that that all sort of domain experts, you know, like yourselves, you know, you have the luxury of having a very unique perspective, and you have you're in a very specific context. So within that context, articulating exactly what your kind of big expensive meaty problem is. Is a fantastic starting point to go off and to talk to, you know, folks like us or whoever, you know, has the, the sort of narrower technological expertise that matters to you.

Yeah. Uh, it's like a, it's a, it's a, it's a much better way of like approaching it, but that's, 

Brent: that's really good. And even, uh, even a business that like, you know, my, my parents own a construction company, uh, and they build homes and they have a website and, you know, I've maintained it and done things for him for years, but I finally sold that like, look.

You can go out and have AI actually build this website for you, maintain it, feed it the information, have it do everything. And it makes sense because you don't have the time, the money, or the staff to do this. Let, let, you know, let an AI do it. And it works perfect for a company like his. It's small. They don't want to spend a lot of money on this.

It's not the main way that people find them anyway. And so I think there's a lot of cases like that for businesses where, um, it's just replacing their, their technology need that I have a couple of buddies that we had started a business together, just building websites on the side. And I kind of let them do it.

And I told him the other day, I go, we need to find a new business. There's no reason for us to exist anymore. AI is going to do this for us. So let's find something else to do with it. And they went off and started a drone business instead, which I think is great. But, but point being is, you know, I think there are areas where AI is going to upset industries and that's okay.

But it's also going to create a lot of jobs in the process. And so I think it just depends on, you know, what your needs are and what you're trying to do. What I do hate to see is I hate to see companies spend millions and millions of dollars heading down these AI roads because it's the new best thing.

only to get to the end of it and lay off a bunch of people because they realized they didn't actually need it or what they actually needed was clear over here and they went the wrong direction because because they just got excited and ran before they ever walked.

Deep: This has been like a super interesting conversation. Um, so thanks so much for coming on. I wanted to end with like, you know, one final question that it's, it's kind of my favorite way to end, but, you know, walk us out five or 10 years into the future, given what you know today and what you guess might happen, you know, Technologically and like maybe socio politically with respect to education.

Describe for us like what does the world of tutoring look like in this sort of grand vision of yours, like 510 years down the road. What's different about a student and their tutoring experience. And why is it so much better, you know, 

Brent: in that, in that future? I mean, first of all, I think tutoring is going to become a mainstream thing, a thing that is required in education for students to continue to move forward.

I think the pandemic exposed that a little bit and made us, it required it. And now they're getting the taste of it. They're seeing what it does. I think it's going to be hard to let go of that. So I think tutoring is here to stay. I think it's going to grow across the board. So now the question is, how do we do it effectively?

How do we do it so that they can afford afford it? And I think that's where AI is going to be really cool. Um, I think a future five to 10 years from now, AI is going to be running behind the scenes telling me, you know, throwing up red flags on. The student over here long before anybody else realizes saying, Hey, they're falling behind.

Let's catch them now before they're a year behind. Let's get them while they're just starting to fall behind. I think that's one area. I think tutors and teachers are going to become more effective as they can get direct feedback on how they're doing in the classroom and virtually. I think both both are going to be there.

I don't think we're going to see a future where AI is The tutor or A. I. Is the teacher until a I can actually look at a student and know what's going on in their lives and understand them emotionally and other ways. I think taking the human interaction out of that is not a good thing at all. And so I, you know, I told you all the time, like teachers, you're not, you're not in jeopardy being replaced by a I anytime soon.

Not to mention that education, especially K 12 is very slow to adapt. And so I think it's gonna be a while before we're there. But I love the idea that it's a tool to help that teacher be better. Or help that, that, that, uh, um, tutor be better. Um, and then you look at like the older kids, high school, um, even the college level where I see it there is it could be quick homework help.

The chat gpt model is a really cool idea. Like I can jump in and ask it a quick question and get an answer that helps me keep moving the same thing I would do if I was going to go ask my teacher assistant in college or somebody else. I can ask this This person, and I can save the big, more meaty questions for class or for that person later on, and they're not answering every little thing.

So I, I think that's kind of where I start to see it separate out, probably more adapted in the higher education and the other students. And then with the younger students, we still need that human interaction and we're going to need that probably forever, if not for a long 

Deep: time. Awesome. Well, thanks a ton.

Awesome. That's all for this episode. I'm Deep Dhillon, your host saying check back soon for your next AI injection. In the meantime, if you need help injecting AI into your business, reach out to us at xyonix. com, that's X Y O N I X dot com. Whether it's text, audio, video, or other business data, we help all kinds of organizations like yours automatically find and operationalize transformative insights.