HARNESS D1.6: Reference Use Cases

This deliverable examines how the EU AI Act applies to artificial intelligence systems in education and vocational training, with particular focus on Annex III, which designates certain educational AI uses as high risk. The guiding research question is: What are the different obligations and challenges of actors of various scenarios in the education sector based on the education clause Annex III of the EU AI Act? Rather than surveying the full landscape of educational technology, the analysis develops two carefully chosen reference scenarios that sit at opposite ends of the legal spectrum: one that falls clearly within the Act's high risk framework, and one that exposes its limits. The methodology is analytical and scenario based, mapping actors, affected rights, legal obligations, and regulatory gaps for each case.

The first scenario, AI assisted admission and student assignment, represents a paradigmatic high risk application under Annex III point 3(a). When an AI system ranks, scores, or assigns applicants to educational programs, it directly shapes access to educational opportunity and long term life chances, justifying the Act's strongest compliance requirements. Obligations fall unevenly across actors: providers bear pre market duties around design, documentation, and risk management, while deployers such as universities or public authorities must ensure meaningful human oversight during use. The central practical challenge is that oversight is only substantive if institutional staff genuinely understand and can override AI generated outputs; where that competence is absent, compliance risks becoming a procedural formality rather than an effective safeguard. The scenario also intersects with GDPR, non discrimination law, and national education law, none of which the AI Act displaces.

The second scenario, AI dependency and the erosion of critical thinking, illustrates where the Act's risk based logic falls short. When students routinely rely on generative AI tools to draft essays, summarize readings, or construct arguments, the harm is gradual and diffuse rather than tied to any single formal decision. Because general purpose writing assistants are not designed to admit, grade, assign, or monitor students, they sit outside Annex III's four binding education categories, even though Recital 56 explicitly recognizes media literacy and critical thinking as values the Act seeks to protect. The AI literacy duty in Article 4 and transparency obligations for synthetic content apply, but are insufficient on their own: a student can know an answer is AI generated and still accept it uncritically. Addressing this gap requires education policy responses including curriculum and assessment reform, teacher training, and institutional rules rather than legal classification alone.

Taken together, the two scenarios reveal the Act's structural logic and its limits. It is robust where AI produces or materially supports a formal decision about a learner, and comparatively weak where the harm concerns the quality of learning itself. Obligations are clearest for providers and deployers in the admissions context and nearly absent in the critical thinking context. The deliverable concludes that the AI Act provides an important but incomplete framework for educational AI governance: it establishes a clear protective core around decision making systems while leaving a significant regulatory gap around the longer term cognitive and developmental effects of everyday AI use in education.