AI tools have been installed in thousands of U.S. classrooms with the fanfare of a major technology rollout, but teachers and students report that the promised boost to learning has not materialised. Activation rates are high, dashboards show near-total deployment within days, yet the impact on achievement remains flat.
Why technology alone is not enough
The pattern mirrors earlier edtech waves. When interactive whiteboards, laptop carts and pandemic-funded platforms arrived, schools focused on installing hardware and logging usage, leaving the instructional model and teacher training to chance. The result was a familiar alibi: the tool was clunky or the migration was botched. MIT quantified the problem last year, finding that 95% of corporate generative-AI pilots produced no measurable return, a figure that now echoes in education.
Department of Education raises the bar
Last week the U.S. Department of Education issued guidance that separates recreational screen time from instructional technology. The document insists that products be judged by demonstrated learning outcomes rather than mere usage. It calls for evidence built into contract renewals and for vendors to publish independent evaluations.
Judge products by demonstrated learning outcomes rather than usage.
This shift does not call for less technology; it demands proof that the technology works.
Early evidence from AI tutors
A two-year randomised trial in 18 Tennessee middle schools showed that while 96% of students tried an AI tutor at least once, they turned to it in only 17% of moments when they made a mistake. A companion study found that gains were strongest when the AI was embedded in a mastery-based workflow that forced students to slow down and review their work. The lesson is clear: a capable tool does not automatically translate into better outcomes without thoughtful design and adult guidance.
Implementation challenges and policy response
Implementation costs extend far beyond a login screen. Schools must train teachers, redesign curricula and create workflows that integrate AI meaningfully. Without these supports, students treat the AI as a shortcut, asking it to produce answers without engaging in critical thinking, a habit that will cost employers in the coming decade.
State legislatures are moving faster than the federal government. Alabama will require an AI-inclusive computer-science course for graduation by the class of 2032, Idaho is building a statewide AI framework, and the OECD plans to test media and AI literacy through PISA in 2029. Each initiative hinges on qualified adults who can teach AI literacy, a workforce that is currently in short supply.
What happens next
As the Department of Education's guidance takes effect, vendors will need to embed training and evidence generation into their products, and school districts will have to allocate resources for professional development. The next wave of AI in education will be judged not by the number of logins but by measurable improvements in student learning and the development of responsible AI use skills.
The tools are extraordinary; the real question now is whether educators and policymakers can build the surrounding ecosystem that turns potential into performance.