Skip to main content

1.1 Potentials & Risks

If you remember only one sentence form this page: AI has made its way into schools like a medicine that comes without a patient information leaflet. As a teacher, you don’t know when to use AI, you lack dosage instructions, and there is no information on the side effects. This chapter serves as the patient information leaflet for the Playbook.

Should AI come with a leaflet like a medicine?

You know that slip of paper tucked inside every medicine box? The one you likely toss aside without a glance? It may seem dull, but this little paper is a quiet triumph of the last century. That leaflet spells out four essentials: what the medicine treats, how much to take, the surprises it might bring, and who should steer clear.

Behind every line on that leaflet stands a whole machinery: clinical trials, regulators, and a responsibility to reveal mistakes. No one believes the leaflet itself heals you. Its real power is making the medicine safe for anyone. That is the magic we are missing when we let AI teach without guidance.

The AI tools now quietly settling into your school inherit a problem that is older than AI: educational technology has rarely come with instructions. In the UK, a review found that only 7 out of 100 education technology companies had put their products through a randomised trial, and only 12 out of 100 had sought outside certification. Schools rarely ask for more. In a survey across 17 American states, only 11 out of 100 teachers and administrators had requested peer-reviewed evidence before adopting a new tool.

The result, in many cases, is wasted money. In the US, on average two-thirds of education software licences went unused, and 98 out of 100 were never used intensively. The EdTech Genome Project, also in the US, looked at about 7,000 teaching tools worth 13 billion dollars and found that 85 out of 100 were either a poor fit or implemented incorrectly.

This chapter may look like the dry, necessary section, but it reveals the hidden stories behind the rules. It tells what really happened, who was affected, and shines a light on the side effects, even those that quietly reached children who never signed up.

Potentials of AI

Most studies on AI's potential focus on university students. School pupils have so far featured relatively rarely in these studies. And most studies relate to short-term interventions lasting from a few hours to a few weeks. Consequently, the so-called novelty effect may have positively influenced the results. A large review of 68 experiments found exactly this pattern. The shorter the study and the smaller the number of particpants, the larger was the reported benefit of AI.  

(1) Let AI explain a new topic and keep the practice for your lesson

Imagine this. Tomorrow, when a pupil misses a topic or needs that tricky concept explained yet again, let an AI tutor handle the first explanation. Reserve the practice for your lesson, where real learning comes alive.

Why this matters. In a well-controlled trial, an AI tutor taught new material at least as well as a good lesson. In a Harvard study, 194 physics students learned two topics. For one topic, they used an AI tutor developed for this purpose. For the other topic, they learned in a well-run active-learning class – with discussion and problem-solving in pairs, not a lecture. Each student completed both approaches, so no one can claim that the stronger group received the better treatment. In a test conducted immediately afterward, the students taught by the AI achieved significantly higher results. The researchers recorded a median time of 49 minutes to complete the task, compared with the 60 minutes allocated for the lesson in the timetable. The authors are cautious in their interpretation of these results: the material was new to the students, and much of the task simply involved explaining it well. They explicitly decline to extend their result to tasks that require pulling several ideas together. And these were undergraduate students, not eighth-graders.

A large review pooling the results of 228 studies points the same way, with one important limit. Generative AI produced larger gains in knowing and understanding than older forms of AI such as intelligent tutoring systems and adaptive practice software. But this was the only one of the four outcomes the review measured where generative AI had the edge.

Go deeper. Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15, Article 17458. Free access, DOI 10.1038/s41598-025-97652-6. The article describes how the tutor was designed and controlled. This is useful for anyone wishing to develop an AI tutor themselves.

(2) Making better use of lesson planning time

Imagine this. Save some of the time you need for lesson planning by using AI to draft your first lesson plan and then revise it thoroughly.

Why this matters. No one disputes that teachers’ workloads are heavy. Therefore, one potential benefit of AI is using working hours more efficiently. A trial in England investigated whether targeted use of AI for lesson planning can reduce teachers’ workload. Researchers randomly assigned 259 science teachers at 68 schools either to a group that used ChatGPT together with an online guide to using it well, or to a group that was asked not to use generative AI at all. Teachers in the ChatGPT group spent about 56 minutes a week planning science lessons for Years 7 and 8 (ages 11 to 13), compared with about 81 minutes in the comparison group. A panel of subject experts assessed the quality of the lesson plans. They did not know whether the plans had been created with or without the aid of AI. The experts found no evidence to suggest that the quality of the lesson plans differed. However, two points must be highlighted. (1) The teachers used the tool less and less as the weeks went on. The evaluators read this as teachers moving from a learning phase into routine; it may also mean that the saving does not sustain itself without support. (2) The trial measured the time spent by teachers on lesson planning, not their pupils’ learning outcomes.

Voices from the field

"Everything comes back to time. Even teachers who see the value of AI rarely have the time learn how to use it."

Go deeper. Roy, P., Poet, H., Staunton, R., Aston, K., & Thomas, D. (2024, 12 December). ChatGPT in lesson preparation: A Teacher Choices Trial. NFER, commissioned by the Education Endowment Foundation and the Hg Foundation. Free at educationendowmentfoundation.org.uk.

(3) Providing better support for learners who do not benefit fully from standard lessons

Imagine this. Your school decides how it wishes to spend its funds. It wants to purchase an AI chatbot. How likely do you think it is that the chatbot providers will provide you with evidence of the bot’s benefits for different subjects?

Why this matters. The provider may point to research on learners with disabilities – one of the longest-studied fields of AI in education, and one that is easily overlooked when funds are allocated. In an analysis of 29 studies involving 41 groups of learners with disabilities, researchers found moderate benefits across a wide range of settings. But read the small print. In seven out of ten of these studies, ‘AI’ meant a robot, usually a humanoid one, often steered by a researcher out of the child’s sight. Only one in five studies used software. So this is no proof that a chatbot will help an autistic pupil in your class. Nor could the reviewers say what makes these interventions effective. Based on their research, they could not identify why the studies that worked differed from those that did not.

Go deeper. Zhang, L., Carter, R. A., Jr., Liu, Y., & Peng, P. (2026). Let's CHAT about artificial intelligence for students with disabilities: A systematic literature review and meta-analysis. Review of Educational Research, 96(1), 215–257. Free, DOI 10.3102/00346543241293424. The real gem is the table of included studies: it instantly reveals just how little of this field actually focuses on the tools currently being marketed to schools.

(4) Teaching learners how AI fails

Imagine this. Ask your class to create ten images of 'a doctor' and ten of 'a nurse.' Tally up who appears in each set, then spark a discussion: where did these patterns come from, and who made those choices?

Gemini_Generated_Image_mfaur1mfaur1mfau.jpg

Fig. 1. Image created with NanoBanana 2, created August, 20, 2026,1 One shot prompt: a doctor, a nurse (AI-generated image, NanoBanana 2) 

Why this matters. The classic version of this lesson feels outdated, and that is exactly why it deserves attention. For years, the demonstration was predictable: ask for a doctor, and a man appears. In a test by the non-profit AlgorithmWatch, Midjourney almost always pictured doctors as white men, while Adobe Firefly made visible, if imperfect, attempts to diversify. A peer-reviewed study of 2,400 AI-generated images of surgeons found the same split between products: two of three popular generators showed almost no women, while the third came close to the real proportion of women in surgery.

Try the same prompt today, and you might see a female doctor or a male nurse appear (Fig. 1). But do not be fooled into thinking the bias has vanished. A 2026 analysis of 1,344 images from three popular generators revealed the opposite: when asked for a 'competent person,' one system showed no women.

What changed is not the bias but the layer it lives in – and that is the important lesson. Human decisions stand behind any of these pictures: what the system was trained on, and what its makers afterward decided it should show. The second is adjustable, invisible, and made by a company.

The first layer of human decisions is what the system has absorbed. Millions of images, crafted and labeled by people, each one echoing the world as it already was. This is the layer that the classic bias lesson reveals, and pupils grasp it easily: the machine became a reflection of what it was fed.
The second layer is shaped by what a company chooses to show you next, yet this influence leaves no trace in the image itself. Three things unfold here, each one described by the creators in their own words.
  • Your words are rewritten before the model receives them. From OpenAI’s own developer documentation: “we use GPT-4 to optimize all of your prompts before they’re passed to DALL-E” — and “this feature isn’t able to be disabled at the moment.” What your pupil typed is not what the machine was asked.
  • Appearance gets filled in when the prompt leaves it open. OpenAI, describing its diversity technique: “This technique is applied at the system level when DALL-E is given a prompt describing a person that does not specify race or gender, like ‘firefighter.’” Afterwards, users were twelve times more likely to say the images showed people of diverse backgrounds. “A doctor” is exactly that kind of prompt.
  • And that tuning can miss. Google, after pausing Gemini’s image generation in 2024: “Our tuning to ensure that Gemini showed a range of people failed to account for cases that should clearly not show a range.”
So when a pupil types “a doctor” and a woman appears, the image offers no clue about which layer brought her into being. That uncertainty is not a flaw in the lesson. It is the lesson itself.
The old story claimed the machine simply soaked up society’s prejudices. The new story cuts deeper and lasts longer: A company made a choice! You cannot spot that choice in the outcome, and it might shift on a Tuesday, quietly, without warning. That truth endures through every new model. The stereotype lesson fades away.

This is the lesson that deserves a student's attention, and it will outlast the next model update: the image is not a simple reflection of reality, and it did not happen by chance. Someone made a choice.

Zeichenfläche 1-100.jpg

Fig. 2.2 Why the picture based on a prompt like "draw a doctor" looks like it does and why the picture itself won't tell you. (AI-generated image, NanoBanana 2) 

This lesson can start early. When 209 Finnish students in 12 classes, grades 6 and 7, explored this topic, the number who could explain the bias using data jumped from about 7 out of 100 to 4 out of 100. The honest reservation: the study measured what children could explain, not what they would do later.

Voices from the field

"Bias is not unique to AI. Even textbooks can contain one. What matters is knowing about chance, about the data behind an answer & about possible distortions."

Go deeper. Vartiainen, H., Kahila, J., Tedre, M., López-Pernas, S., & Pope, N. (2025). Enhancing children's understanding of algorithmic biases in and with text-to-image generative AI. New Media & Society, 27(9), 5342–5368. Free, DOI 10.1177/14614448241252820. The study had no comparison group – read it as a promising classroom design, not as proof.

Weinmann, H., Messingschlager, T. V., & Appel, M. (2026). Gender bias in text-to-image generative artificial intelligence: Neglect and stereotypical presentations across three popular platforms. New Media & Society. Online first. Free, DOI 10.1177/14614448261435197.

Ali, R., et al. (2024). Demographic representation in 3 leading artificial intelligence text-to-image generators. JAMA Surgery, 159(1), 87–95. DOI 10.1001/jamasurg.2023.5695.

The vendors’ own statements: OpenAI (2022), Reducing bias and improving safety in DALL·E 2; Depue, W. (2023), What’s new with DALL·E 3?, OpenAI Cookbook; Raghavan, P. (2024), Gemini image generation got it wrong. We’ll do better, Google. 

How solid are the research results on the potential of AI at this very moment?

The most important thing. If someone quotes a study to you, ask two questions before agreeing: "Who were the learners, and how long did the study last?"

Why this matters. In 2025, a meta-analysis of 51 studies reported a large benefit of ChatGPT for students’ academic performance. The article was read hundreds of thousands of times and served as the basis for numerous confident statements in staff rooms. On 22 April 2026, the journal retracted the article. A retraction means that a journal officially withdraws something it has published; the article remains visible, marked with the note ‘RETRACTED’, so that anyone who has cited it can recognise this. Two researchers who had not been involved in the study had identified discrepancies in the analysis. The journal's editor wrote that the problems ‘undermine the editor’s confidence in the validity of the analysis’. The authors did not respond to correspondence on this matter.

This does not mean that AI has no impact. There are reputable studies that demonstrate moderate benefits. This field of research is also not yet well defined because it is relatively new and highly dynamic. It means that the figure that was frequently cited was incorrect, and that it took a year and two external experts to realise this.

Another reservation is worth mentioning. In mathematics, at least, the closer the evidence gets to school, the smaller the effect. Across subjects the picture is mixed: some recent reviews find larger effects in schools than at universities – but the school studies are still few and small.

Go deeper. The retraction itself is one page, free, and worth reading precisely because it is short: Wang, J., & Fan, W. (2026). Retraction note: The effect of ChatGPT on students' learning performance, learning perception, and higher-order thinking. Humanities and Social Sciences Communications, 13, 528. DOI 10.1057/s41599-026-07310-z.

The retracted original was HSSC 12, 621 (2025). Still standing: Wu, X., Zhu, P., Zhang, J., Yin, M., & Wang, Y. (2026). ChatGPT's impact on student learning outcomes: A meta-analysis of 35 experimental studies. Humanities and Social Sciences Communications, 13, 684. Free, DOI 10.1057/s41599-026-07019-z.

The school-only figure for mathematics: Yi, L., Liu, D., Jiang, T., & Xian, Y. (2025). The effectiveness of AI on K-12 students' mathematics learning: A systematic review and meta-analysis. International Journal of Science and Mathematics Education, 23(4), 1105–1126. DOI 10.1007/s10763-024-10499-7.

A broader, more recent pooled analysis: Han, X., Peng, H., & Liu, M. (2025). The impact of GenAI on learning outcomes: A systematic review and meta-analysis of experimental studies. Educational Research Review, 48, 100714. DOI 10.1016/j.edurev.2025.100714.

Risks of AI

(1) Doing something is not the same as learning

Imagine this. Before you set a homework task a chatbot can complete, think about the purpose of the task. Might it be a good idea to change the task?

Why this matters. Do you learn faster with a chatbot? Around a thousand pupils in Years 9 to 11 practiced maths in three groups: one using a standard chatbot, one using a version programmed to withhold the answer and ask a follow-up question, and one without a chatbot. Whilst using the chatbot, the group with the standard chatbot completed 48 percent more practice exercises than the group without one. Afterwards, everyone sat the same exam without a chatbot. The result, which seemed surprising at first, was that the group using the standard chatbot performed 17 percent worse than the pupils who had not used a chatbot. The group that had worked with the chatbot which withheld the answer showed no drop in performance in the test compared to the group without a chatbot, but practiced more. The pupils using the standard chatbot were neither lazy nor did they cheat. They worked harder and learnt less, because what the tool eliminated was not the effort, but the cognitive challenge that constitutes learning. 

Voices from the field

“AI makes it very easy to get a solution without understanding it.”

Go deeper. Bastani, H., Bastani, O., S ngu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. *PNAS, 122*, e2422633122. DOI 10.1073/pnas.24 2633122.

(2) AI Detectors punish the wrong pupils

Imagine this. You receive a student’s essay to grade. Curious about its authenticity, you feed it into an AI detector. The result flashes: the essay was almost certainly written by AI.

Why it matters. You use the AI detector to ensure fairness, yet for a particular group, the detectors have exactly the opposite effect. A detector does not recognise machine-generated text. It measures how predictable the wording is. And that is precisely where the problem lies, for example, when a pupil is writing in a foreign language, is unpractised in writing, or has a limited everyday vocabulary. Their writing is predictable. When seven widely used detectors analysed 91 essays written for the TOEFL English test by non-native speakers – collected from a Chinese online forum – they wrongly flagged them as machine-generated in around six out of ten cases. Essays by American eighth-graders writing in their native English were almost all classified correctly. A tool that triggers false positives for exactly one group is not neutral. Furthermore, this group is the least able to defend itself.

Go deeper. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. Free, DOI 10.1016/j.patter.2023.100779.

(3) Where on earth will the pupils' data be?

Imagine this. You use an AI tool your school offers. Try to find out where the pupils' data go.

Why it matters. Can your school ensure that the data is not passed on to a third party? A third party is a company that is neither your school nor the app’s developer. Typically, these are advertising or analytics companies that receive data whilst the pupil is working. No one at the school chose this company. When researchers carried out a technical analysis of 163 learning products recommended by governments during the pandemic-related school closures, 145 of them processed children’s data in a way that jeopardised or infringed their rights by transmitting data to 196 third-party companies or granting them access, predominantly from the advertising industry. Of the 42 governments that developed their own products rather than purchasing them, 39 had developed products with the same problem. The study covers products from 2021, and some may have changed since then. However, we cannot rule out that the same applies to AI applications.

Go deeper. Human Rights Watch (2022). "How dare they peep into my private life?: Children's rights violations by governments that endorsed online learning during the Covid-19 pandemic. Free at hrw.org. The country annexes let you look up what was endorsed where.

(4) AI  arrives last where it is needed most

Imagine this. How much training did you already receive on learning about AI or subject-specific teaching with AI?

Why this matters. In one experiment, adults with and without a university education solved a work-style business task. With an AI assistant, about three-quarters of the gap between the two groups disappeared. When the researchers then took the assistant away, a sizeable part of the gap came back. The authors conclude that the AI did much of the work, and that only part of the gain survived once it was gone.

A survey of American school principals adds the second half of the picture: the more disadvantaged a school’s pupils, the less the school had done to integrate AI – fewer policies, less teacher training, less guidance on pupil use. That is an association, not proof of cause.

Taken together: a good tool can narrow a gap while it is in the learner’s hands. But it tends to reach the schools that need it least, and only part of its benefit stays behind when it is removed.

German figures show the same pattern. Among 1,590 young people aged 14 to 20, 80 percent from the most affluent families but only 55 percent from the least affluent saw AI as an opportunity for school. Usage follows the same gradient: among young people who use AI tools, 70 percent of Gymnasium pupils use them for school or homework, compared with 58 percent at other secondary schools.

What about teacher professional development? A survey of more than ten thousand teachers in England found that 45 percent of teachers at private schools had already received formal training on AI, compared with 21 percent at state schools. Within the state system, the figure ranged from 26 percent at schools with the most affluent intakes to 18 percent at those with the least affluent.

Go deeper. Cruces, G., Fernández Meijide, D., Galiani, S., Gálvez, R. H., & Lombardi, M. (2026). Does generative AI narrow education-based productivity gaps? Evidence from a randomized experiment (NBER Working Paper 34851). A working paper, not yet peer-reviewed. Campos, C., & Singleton, J. D. (2026). AI diffusion gaps: Unequal integration of AI across K-12 schools (NBER Working Paper 35347). Latham, K., & Montacute, R. (2025). Artificial advantage? AI in the classroom and the inequality gap. The Sutton Trust. Free at suttontrust.com. Vodafone Stiftung Deutschland (2024), Pioniere des Wandels; mpfs (2024), JIM-Studie 2024. Both free, in German.

(5) What improves when learning with AI is not necessarily what you see

Imagine this. You introduce a new chatbot into your teaching and the pupils are enthusiastic about using it.

Why this matters. A large review of 228 studies looked at four different things AI might improve: knowing and understanding; applying what one knows; the affective side, such as motivation and engagement; and the ability to plan, monitor and correct one’s own learning. Three of the four improved. The last one did not, as far as the studies could tell.

  • For knowing and understanding, generative AI did better than older forms of AI such as intelligent tutoring systems and adaptive practice software.
  • For applying knowledge, and for the affective side, generative AI did no better than the older software many schools already own.
  • For self-steering, the pooled effect was small and could have been chance – and there were too few studies to ask which kind of AI might help at all.

Only a handful of the results came from randomised trials, and the enthusiasm you see in the first weeks is exactly what short studies capture best.

Here’s a quick side note: in this review, the greatest gains were found in the arts and humanities.

Go deeper. Yeo, G. H., & Lansford, J. E. (2025). Effects of artificial intelligence on educational functioning: A review and meta-analysis. Educational Psychology Review, 37(4), article 110. DOI 10.1007/s10648-025-10085-5.

Go deeper. Yeo, G. H.  & Lansford, J. E. (2025). Effects of artificial intelligence on educational functioning: A review and meta-analysis. Educational Psychology Review, 37(4), article 110. DOI 10.1007/s10648-025-10085-5.