My First Car Ride With AI
Plus new survey results on AI use from teachers and districts, a chatbot for learning about Holocaust survivors, and more.
I was in San Francisco last week to drop my son off at college and had my first ride in an autonomous vehicle. Readers, it was so amazing! I am not ready to hop onto the freeway in one, but I felt safer in that car than I typically do in an Uber. I don’t know how a person could ride in an autonomous vehicle and fail to understand how transformative AI technologies will be. At the same time, this chilling news story from South Korea about deepfake abuse reminds one of negative transformative potential.
As always, there is a lot going on in the AI world. Here are some of the things that have caught my eye and imagination recently (as usual, thanks to John Bailey, Tom Coyne, Dan Silver, and the CRPE team for leads).
Help Versus Harm?
Hechinger Report recently reported on a new study out of UPenn. The study has yet to be peer reviewed or published, so consider the findings preliminary. That said, it was a well-designed study with interesting findings.
The paper’s provocative title, “Generative AI Can Harm Learning” emphasizes the negative effects AI might have on learning. The authors found that when students used ChatGPT-4 (without any tutoring interface) to reinforce math concepts, they got a lot of answers wrong (due to error or AI hallucinations), and this usage actually diminished students’ ability to perform well on exams. In effect, students tended to ask GPT for the correct answer and thus did not actually learn the concepts or skills.
However, when an AI tutoring program (which screened out incorrect answers and prompted students to work through problems rather than just giving the answer) was used for the same purpose, students performed significantly better on practice problems (with 127% improvement) but did not perform better on the exam. This interesting and somewhat puzzling outcome should be explored further in other research studies on AI-enabled tutoring.
It’s important to learn when and how AI can diminish learning and students’ ability to solve problems for themselves, but the basic finding of this study should not surprise us. Over-reliance on calculators can also become a crutch. We need to consider which skills kids need to learn without a calculator, but we also need to make sure kids know how to use a calculator to save time or to compute things they will never need to retain. It’s also worth noting that we can expect generative AI to become much better at math and math tutoring over time.
In all, the interpretation of these findings is very much a “glass half empty or half full” question. The title could just as easily have been: Chatbots with better constraints and training can help students.
Most importantly, beware of claims that studies like this “prove” that AI is overhyped, dangerous, etc. Take a deep breath, folks: there will be a lot more studies like this coming. We need to look honestly at what such studies do or do not tell us and carefully examine this body of knowledge as it grows.
I know from many years of studying charter schools that early studies on controversial topics tend to be used as weapons rather than neutral sources of information. An excellent recent post by teacher and blogger Nick Potkalitisky contrasted two help vs harm studies to show how the presentation and press coverage of research tend to fall into black-and-white thinking. He makes the case that exhausted teachers deserve more nuanced research and reporting. We should keep this in mind as the field of AI research continues to develop.
Why Rigorous Ed Tech Research Is so Hard to Find
The CEO of Once wrote a terrific article about the challenges and disincentives around conducting Randomized Control Trials (RCTs) in ed tech. He makes a strong case for doing them anyway.
I believe K-12 education deserves some knock your socks off randomized controlled trial results and I want to serve them up. Yes education is more psychology than biology (harder to standardize, harder to scale), but the impact of education on people's lives is no less than health and is often directly related to health in complex causal relationships.
How Educators Are Using AI
Cambium Learning released the results from their survey of teachers and administrators, which echo other recent survey results on use cases and perceived barriers or risks. More than half of survey respondents (56%) reported that they are leveraging AI to create personalized learning experiences for students. Other uses included providing real-time performance tracking and feedback (cited by 52% of respondents), helping students with critical thinking skills (50%), proofreading writing (47%), and lesson planning (44%).
On the administrator side, top uses of AI included interpreting and analyzing student data (61%), managing student records (56%), and managing professional development (56%).
One interesting data point I’d not seen before: 20% of respondents said their schools and districts have created new job positions to meet the need for ed tech and AI expertise.
How Districts Are Using AI
Digital Promise conducted a survey of the 31 districts in their League of Innovative Schools and found that districts continue to be hesitant to adopt AI policy (as we reported here). Here are a couple of interesting data points:
Most districts (93%) report that at least some schools are using AI technology in the classroom, and 41% of districts report purchasing AI-powered tools in the last 12 months.
Three quarters (75%) of districts report offering professional development to teachers on the safe and effective use of AI, but only a quarter (25%) report having specific policies or guidance for the effective use of AI.
Addressing Fragmented Policy
K-12 Dive did a nice piece on a central theme in our Wicked Opportunities paper: what to do about the fact that states are taking disparate approaches to AI policy. Some ideas:
Consider innovative ways AI could transform education. District leaders can use AI to rethink and redesign schools, but they first need a clear vision of what that would look like.
Help districts strategically use AI tools.
Give priority to funding initiatives that test AI solutions in low-income and historically marginalized communities.
Provide districts with detailed, actionable implementation plans (and funding) for adopting AI effectively.
Interesting Educational Tools
This chatbot from the Shoah Foundation is really worth trying out in the microphone mode. It’s a powerful experience to be able to have something like a conversation with a Holocaust survivor. Imagine how these kinds of interactive tools could change how History classes are taught.
Closing Words
When students trust the process—when they feel safe to experiment, fail, and try again—AI becomes more than just another tool in the box. It becomes a gateway to deeper learning, a spark for creativity, and a platform for innovation. This is where the real magic happens—when students engage with AI not just confidently, but boldly, knowing that the journey they’re on is supported, encouraged, and celebrated every step of the way.
-Teacher and blogger Nic Potkalitsky
“Tomorrow belongs to those who can hear it coming.”
―David Bowie (referenced in Andrew Maynard’s terrific book, Future Rising)