Can AI help motivate unmotivated learners?
Plus emerging profiles of AI "early adopters," new resources, and a reimagined holiday poem from CRPE Director Robin Lake (and ChatGPT)
The holidays are nearly upon us! I’ll be taking advantage of down time to finally read Yuval Noah Harari’s (very long) book Nexus. I hope to have a pithy synthesis and analysis for you come January.
In the meantime, AI research is on my mind. I’ve seen a few thought-provoking articles and had the pleasure of talking with some very smart friends about the challenge of evidence building in the time of AI.
A major question for philanthropists and governments interested in supporting research on AI in education is: What are the highest priority evidence building questions and projects? I think about this question a lot, as regular readers may have noticed. So here is an important question that my meanderings lately bring to mind:
Can AI tutors help students who are not motivated to learn?
This little mini-case study presented in Education Next explores the question that Laurence Holt has raised: Will AI simply help the “top 5%” (e.g. the academically strong and motivated) to get even better? This is an important question, and the article’s experiment suggests that the answer is yes (though it’s only an “n” of one!). The author had his 16-year-old intern work with an AI tutor and observed a highly successful and enjoyable tutoring session that rivaled one led by an expert teacher.
As much as the intern enjoyed the experience, he also raised concerns about whether it would help more unmotivated peers. He writes, “A human tutor’s main purposes are to teach and to motivate. It’s nearly impossible to teach a student who doesn’t want to learn… I think LLMs work well for motivated learners, but in the cases where the user absolutely does not want to be learning, an AI tutor is not effective because it lacks the strategies to motivate them.”
This is an important question for the field, and one that must be tracked transparently. Too often, ed tech studies report on the efficacy of the tool when kids stick with it. Educators need to know if the tool helps motivate unmotivated learners.
How and why does any given tool reach beyond the 5%?
It’s also a question that must be understood in the context of schooling. If a tutoring tool is used in a classroom or school in tandem with great classroom management techniques or a teaching and learning strategy designed to intervene quickly if students are not staying on track, etc., does the tool help kids who are not especially self-motivated? We urgently need to start conducting studies that look at interaction effects between AI learning tools and other strategies and contexts.
Skeptics are right to suggest that AI will only exacerbate existing inequalities if simply layered onto the existing systems. In that case, as my friend Travis Pillow smartly observed recently, the promise of AI will be “domesticated” rather than fully explored. To get transformative results and ones that reach the kids most in need, we’ll need large-scale experiments that assess how intentional designs to go beyond the 5% can and do work.
Now, with these questions in minds, a couple of recent studies are worth considering (and credit to Nick Potkalitsky’s Substack, where I saw mention of them).
More evidence that AI can motivate the top 5%
(Note: study conducted among Harvard students only!)
A randomized controlled experiment in a large undergraduate physics course (N = 194) at Harvard University measured the difference between 1) how much students learn and 2) students’ perceptions of the learning experience when identical material is presented through an AI tutor compared with an active learning classroom.
The AI tutor was developed with the same pedagogical best practices as the lectures.
Students learned more than twice as much in less time when using an AI tutor, compared with the active learning class. They also feel more engaged and more motivated.
These findings offer empirical evidence for the efficacy of a widely accessible AI-powered pedagogy in significantly enhancing learning outcomes, presenting a compelling case for its broad adoption in learning environments.
Evidence that AI tools can help well beyond the top 5%!
AI applications to support human tutoring might improve learning outcomes, but engagement issues persist, especially among students from low-income backgrounds.
An AI-assisted tutoring model, combining human and AI tutoring, could have positive impacts on learning processes.
To investigate this hypothesis, the authors conducted a three-study quasi-experiment across three urban and low-income middle schools consisting of: 1) 125 students in a Pennsylvania school; 2) 385 students (50% Latinx) in a California school, and 3) 75 students (100% Black) in a Pennsylvania charter school, all implementing analogous tutoring models.
Compared to students using math software only, students using human-AI tutoring experienced positive effects, particularly in student’s proficiency and usage, with evidence suggesting lower achieving students may benefit more compared to higher achieving students.
Emerging profiles of AI “early adopters”
You may remember that CRPE has been surveying and interviewing AI early adopter school systems this fall. These districts and CMOs are pursuing diverse problems and experimenting with AI in unique ways. We just wrapped interviews this week and are sorting through the data. While findings are preliminary, something I found interesting is the emerging profiles of these early adopters. Here's a preview of how we see folks organizing their efforts:
Dabblers: Aspirational AI use, or primarily using AI to maintain “innovator” reputation. Allowing educators to use AI tools, but at this point, they lack a clear purpose and plan for spreading the use of AI in their district.
Emerging Users: Encouraging the use of AI among educators and administrators, but adopting AI without a coherent strategic plan. Their AI strategies are disconnected from broader district goals, and professional learning is one-off and optional.
System Changers: Have a clear vision for change, usually connected to a strategic plan, and using AI to help them with that vision.
Reimaginers: Rethinking what school can be in the age of AI, exploring how AI could fundamentally reshape every aspect of the educational system.
At this point, most early adopter districts fall in the Emerging Users category, but these categories are dynamic and show that AI adoption in education is not a monolithic process, but a complex, varied journey with different organizational approaches and levels of sophistication.
Stay tuned: We'll be putting out a full review of the data in early 2025!
New resources
On December 17, a House bipartisan task force released a report on artificial intelligence. One of the key findings? “Despite federal and state efforts, the U.S. has a significant gap in the appropriate talent needed to research, develop, and deploy AI applications—and this gap is growing. Educating and training American learners in AI topics will be critical to continued U.S.leadership in AI technology and for America’s economic and national security.” The report recommends investing in more K-12 STEM and AI education, as well as empowering teachers with training and other resources.
EdWeek is out with this nice special report on “The Transformative Potential of AI: 6 Big Questions for Schools.” The individual reports dig into topics that we are deep into, including AI for special populations and for teacher professional development (PD). We’ll have more data on PD soon.
OpenAI released a “Student Guide to Writing with ChatGPT.” Not sure I agree with some of the advice. I feel like OpenAI could use some good education advisors, maybe an advisory panel or something.
Happy Holidays!
How could I resist? Here’s a poem for my loyal readers from me and my co-author, ChatGPT:
'Twas the dawn of GenAI, and through every school, Teachers were stirring, adapting each rule. Lesson plans stacked by the smartboard with care, In hopes that new tech might ease their despair. Students were nestled with screens in their hands, Crafting bold essays from AI's commands. Principals worried 'bout cheating and trust, While pondering futures they knew were a must. When out in the district arose such a clatter, Leaders sprang up to see what was the matter. Away to the forums they flew in a flash, Reading of progress — and fears that might clash. The glow from the laptops, both wondrous and bright, Lit pathways to learning deep into the night. When what to their curious eyes should appear, But bold innovations that silenced their fear. With a visionary driver, nimble and keen, They knew in a moment it must be their team. More rapid than Wi-Fi, ideas came alive: "Let’s rethink assessments! Let new models thrive!" "Now project-based learning! Now AI-assisted! Let’s teach how to question and think unresisted! To equity’s summit, through challenges vast, Education's future is forming — and fast!" As data informs where blind guesses once led, New pathways to learning take root in its stead. So up through the cloud, solutions they flew — With grants full of promise and researchers too. And then, in a twinkling, they heard on the ground The buzz of new programs reshaping what’s sound. As teachers embraced both the old and the new, Creative AI helped learning shine through. It spoke not of shortcuts, but tools to enhance, Of fostering minds through balance and chance. From essays and feedback to projects profound, AI in education gained solid ground. Its impact was clear — not a fleeting affair — A bridge to new worlds built on hopes and care. And teachers proclaimed, as they turned out the light: "Happy learning to all, and to all a future bright!"
Thank you for joining me on this exploration of GenAI this year! I’m looking forward to what’s next in 2025.