---
title: "Is AI tutoring better than human tutoring? What the randomized trials measured (2026)"
description: "Is AI tutoring better than human tutoring? Two randomized trials found real gains, but neither tested AI against a tutor, and both kept a person in the room."
url: "https://theinternetninja.com/blog/is-ai-tutoring-better-than-human-tutoring-what-the-trials-measured-2026/"
updated: "2026-10-26"
---
# Is AI tutoring better than human tutoring? What the randomized trials measured (2026)
By Maneesh Behera, Head of AI Deployment. Published 2026-10-26. The Internet Ninja, analysis.
> Two randomized trials of AI tutoring measured real learning gains: 0.31 standard deviations in Nigeria and 4 percentage points of topic mastery in a US district. Neither tested AI against a human tutor, and in both the gain came with a person in the room.
## Case files this argument rests on
- [A World Bank RCT in Nigeria: a six-week GPT-4 tutoring program lifted learning by 0.31 standard deviations](https://theinternetninja.com/stories/world-bank-nigeria-generative-ai-tutoring-rct-edo-state-2025/)
- [Tutor CoPilot: a Stanford randomized trial found AI real-time coaching made tutors more effective, with the biggest gains for the weakest tutors](https://theinternetninja.com/stories/tutor-copilot-stanford-fev-rct-4pp-mastery-9pp-lower-rated-tutors-2024/)
Every parent and every district buyer is being sold the same promise this year. A chatbot that tutors each child one to one, at the price of a subscription, and never gets tired.
The question underneath the pitch is whether it beats the tutor it would replace. There is a small record that tests AI tutoring with randomized trials, and it answers a different question than the one being asked.
The headline result is real. In a World Bank trial in Nigeria, students using an AI tutor for six weeks scored 0.31 standard deviations higher on a composite assessment than a control group ([source](https://ideas.repec.org/p/wbk/wbrwps/11125.html)).
## What AI tutoring is
AI tutoring is the use of a language model to guide a student through learning, either by talking to the student directly or by coaching the human tutor who does.
That split matters more than any product feature. The two trials in this post tested one of each.
## Is AI tutoring better than human tutoring?
Nobody on this record has measured that. Neither randomized trial put an AI tutor against a human tutor, and in both the measured gain came with a person present.
The Nigeria trial compared students in an after-school AI program with students who had no program. Its sessions ran "between June and July 2024" in school computer labs under teacher supervision ([source](https://blogs.worldbank.org/en/education/From-chalkboards-to-chatbots-Transforming-learning-in-Nigeria)).
The Stanford trial never let the AI talk to a student. Tutor CoPilot gives "expert-like guidance to tutors as they tutor" ([source](https://arxiv.org/abs/2410.03017)). The comparison was tutors with the tool against tutors without it.
So the honest answer to "better than human tutoring" is that the evidence measured AI with humans, not AI instead of them. Anyone quoting these trials as proof that a chatbot beats a tutor is citing a study that did not run that test.
## Does AI tutoring work?
Yes, in both trials, and by more than most education programs manage. The size of the gain depends on who the AI was helping.
The World Bank program "deployed Microsoft Copilot (powered by GPT-4) to support first-year senior secondary students in English language learning over six weeks" ([source](https://ideas.repec.org/p/wbk/wbrwps/11125.html)). About 800 students took part in after-school classes in Benin City ([source](https://www.ictworks.org/genai-advance-learning-outcomes/)).
The paper reports 0.31 standard deviations on the composite and "the effect on English, the main outcome of interest, was of 0.23 standard deviations" ([source](https://ideas.repec.org/p/wbk/wbrwps/11125.html)). The gain carried into a broader end-of-term exam, at 0.206 standard deviations ([source](https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099548105192529324)).
The evaluators put the gain at "1.5 to 2 years of 'business-as-usual' schooling" ([source](https://ideas.repec.org/p/wbk/wbrwps/11125.html)). That is a cost-effectiveness translation, not a claim that students advanced two grades in six weeks.
Against a review that found "a median effect of 0.10 standard deviations in overall test scores," the paper places its result "at least at the 80th percentile of all RCTs" ([source](https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099548105192529324)). On language outcomes alone it sits "near the 70th percentile" ([source](https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099548105192529324)).
## Does AI math tutoring help weaker tutors?
Yes, and that is the most useful finding on this record. In the Stanford trial the AI lifted the weakest tutors most.
The trial involved "900 tutors and 1,800 K-12 students from historically under-served communities" ([source](https://arxiv.org/abs/2410.03017)). The deployment partner was the tutoring company FEV Tutor, in "a large school district in the South" ([source](https://www.k12dive.com/news/ai-tutor-effectiveness-stanford-university/728980/)).
Students whose tutors had the tool "are 4 percentage points (p.p.) more likely to master topics (p<0.01)" ([source](https://arxiv.org/abs/2410.03017)). For lower-rated tutors the effect more than doubled: their students improved "mastery by 9 p.p." ([source](https://arxiv.org/abs/2410.03017)).
The tool cost "only $20 per-tutor annually" ([source](https://arxiv.org/abs/2410.03017)). Across more than 550,000 messages, tutors with it were "more likely to use high-quality strategies" such as guiding questions, and "less likely to give away the answer" ([source](https://arxiv.org/abs/2410.03017)).
Tutors also flagged a flaw: the AI sometimes generated "suggestions that are not grade-level appropriate" ([source](https://arxiv.org/abs/2410.03017)). The human filtered them. That filter is part of what was measured.
## Who does AI tutoring help most?
It depends on the design, and the two trials point in opposite directions. That is the reading the sales pitch skips.
In Nigeria, where students used the AI directly, "the largest effects are for female students, and those with higher initial academic performance" ([source](https://ideas.repec.org/p/wbk/wbrwps/11125.html)). The paper notes the program still helped "across the baseline ability distribution" ([source](https://ideas.repec.org/p/wbk/wbrwps/11125.html)).
In the Stanford trial, where the AI coached the adult, the biggest gain went to students of the weakest tutors ([source](https://www.the74million.org/article/study-ai-assisted-tutoring-boosts-students-math-skills/)).
Put the AI in front of the student and the stronger students pulled further ahead. Put it behind a weak tutor and the gap closed. If equity is the goal, the second design is the one with evidence behind it.
## The proof
TIN's case file on the Nigeria trial quotes the World Bank paper's abstract and results tables, and prints the one place press and paper disagree (0.24 against 0.238 on English): [a six-week GPT-4 tutoring program and a 0.31 standard-deviation gain](https://theinternetninja.com/stories/world-bank-nigeria-generative-ai-tutoring-rct-edo-state-2025/).
TIN's case file on Tutor CoPilot quotes the preregistered Stanford paper and two independent education outlets that report the same effects: [4 points of mastery, 9 for students of lower-rated tutors](https://theinternetninja.com/stories/tutor-copilot-stanford-fev-rct-4pp-mastery-9pp-lower-rated-tutors-2024/).
Read the limits. Each rests on a single working paper that had not been peer-reviewed in a journal or replicated by a second team. Neither measured durability past the trial window, and the Nigeria program ran in nine schools.
## What each trial tested
| Trial | Who used the AI | Compared with | Result | Gained most |
|---|---|---|---|---|
| World Bank, Nigeria | Students, teacher present | No program | 0.31 SD composite, 0.23 SD English | Stronger students |
| Stanford Tutor CoPilot | Human tutors | Tutors without AI | +4 pp mastery | Students of weaker tutors (+9 pp) |
## The bottom line
The question "is AI tutoring better than human tutoring" assumes the choice is one or the other, and the only randomized evidence on file never made that choice. Both measured gains came from AI working alongside a person, and the cheapest one, $20 a tutor a year, made a weak human tutor better rather than removing them. Before buying any AI tool that claims to replace a skilled person, ask which trial tested it against that person, and who in the trial gained most.
## Sources
1. World Bank, "From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria" (Policy Research Working Paper 11125), abstract via RePEc/IDEAS, 2025-05. https://ideas.repec.org/p/wbk/wbrwps/11125.html
2. World Bank, "From Chalkboards to Chatbots" (Policy Research Working Paper 11125), full text, 2025-05. https://documents.worldbank.org/en/publication/documents-reports/documentdetail/099548105192529324
3. World Bank education blog, "From chalkboards to chatbots: Transforming learning in Nigeria, one prompt at a time," 2025-01-09. https://blogs.worldbank.org/en/education/From-chalkboards-to-chatbots-Transforming-learning-in-Nigeria
4. ICTworks, "Generative AI Now Proven to Advance Learning Outcomes in Nigeria," 2025-01-23. https://www.ictworks.org/genai-advance-learning-outcomes/
5. Rose E. Wang and others (Stanford University), "Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise," arXiv 2410.03017, v2 2025-01-26. https://arxiv.org/abs/2410.03017
6. K-12 Dive, "How AI can improve tutor effectiveness," 2024-10-07. https://www.k12dive.com/news/ai-tutor-effectiveness-stanford-university/728980/
7. The 74, "Study: AI-Assisted Tutoring Boosts Students' Math Skills," 2024-10-07. https://www.the74million.org/article/study-ai-assisted-tutoring-boosts-students-math-skills/
## Questions
**Is AI tutoring better than human tutoring?**
AI tutoring has not been shown to be better than human tutoring by either randomized trial in this post, because neither one compared them head to head. The World Bank trial in Nigeria compared AI-supported after-school sessions with no program and found a 0.31 standard-deviation gain. The Stanford Tutor CoPilot trial gave AI to human tutors and found students 4 percentage points more likely to master a topic.
**Does AI tutoring work?**
AI tutoring worked in the two randomized trials documented here, with limits. A six-week program using Microsoft Copilot in nine Nigerian public schools raised a composite score by 0.31 standard deviations and English by 0.23. Both studies are single working papers that had not been replicated by a second team.
**Does AI math tutoring help weaker tutors?**
AI math tutoring helped weaker tutors most in the Stanford Tutor CoPilot trial. Students of lower-rated tutors improved topic mastery by 9 percentage points, against 4 points across all students, and the tool cost about $20 per tutor per year.
This post is analysis, not proof. The evidence lives in the case files: https://theinternetninja.com/stories/