DfE Generative AI Product Safety Expectations: What Schools Need to Know

The DfE's generative AI product safety expectations are now referenced in statutory safeguarding guidance, but many schools have never read them. Here is what they say, what has changed, and how to use them when choosing AI tools.

What are they?

Generative AI: product safety expectations is a DfE document, first published in January 2025, that sets out what generative AI products and systems used in education should do to be considered safe. It is aimed mainly at developers and suppliers, but it is equally useful to schools as a checklist when choosing tools. KCSIE links to it from its filtering and monitoring section — see our KCSIE and AI timeline.

What they cover

In summary, the expectations say AI products used in education should:

• Filter content — reliably prevent access to harmful and inappropriate content.
• Monitor and report — keep activity logs and alert the right people, including routing safeguarding concerns to the DSL.
• Be secure — protected against misuse and attempts to get around safety features.
• Protect privacy and data — comply with UK GDPR, and not use pupil or staff data or work for commercial purposes or model training without a lawful basis or consent.
• Respect intellectual property, including pupils' work.
• Be designed and governed responsibly — transparent, tested and accountable, with child safety prioritised.

What changed in 2026

The expectations were updated in 2026 to address wider harms — including effects on cognitive development (tools that simply hand pupils answers rather than supporting thinking), emotional and social development (for example, tools presenting themselves as human-like companions), mental health (detecting distress and routing it to humans) and manipulation (engagement-maximising or deceptive design). Check the current version on GOV.UK for the exact wording.

How schools should use them

1. Use them as supplier questions. Before approving a pupil-facing AI tool, ask the supplier how it meets each area, in writing.
2. Link them to your DPIA. The privacy points map directly onto a data protection impact assessment.
3. Include them in your filtering and monitoring review. See KCSIE filtering and monitoring for AI.
4. Brief governors. They should know which AI tools pupils use and how they were checked.

Questions to ask any AI supplier

• How does your tool filter harmful content, and how is that tested?
• What activity logs exist, and how are safeguarding concerns flagged to our DSL?
• Is pupil data or work used to train your models?
• Where is data stored and processed, and will you sign a data processing agreement?
• How does the tool avoid simply giving pupils answers?
• How do you prevent the tool presenting itself as a friend or companion?

The last two questions matter most for chatbots. See safeguarding risks of AI companion chatbots.

Applying them to common tools

The expectations apply most directly to pupil-facing tools — chatbots, tutoring tools and AI features pupils use directly. Teacher-only tools used without pupil data are lower risk, though data and security points still apply. For pupil-facing tools, a written response from the supplier covering each area should sit alongside your DPIA.

Red flags when reviewing a tool

• No clear answer on whether pupil data trains the model.
• No activity logs available to the school.
• A chatbot that talks like a friend or encourages long sessions.
• No way to route concerning conversations to staff.
• Vague claims about educational impact with no evidence.