OAE Evaluation Framework & Toolkit: For OAE NAECs, Centers & Nodes

This Framework and Toolkit has been designed to assist astronomy educators within the Office of Astronomy Education (OAE) network (NAEC, Centers and Nodes) in systematically evaluating their educational programmes and developing evidence-based practices that can be applied across various contexts. Employing a ‘realist approach’ to evaluation, the resource delves into what works, for whom, and under what circumstances. This approach accounts for the intricate nuances and complexities of different environments, including cultural and practical components that vary across local geographies, and in diverse school settings.

The document guides NAECs, Centers and Nodes through evaluating their programmes from start to finish, through stages including: setting objectives; creating a theory of change; deciding what type of data to collect, when to collect it, and from whom; ethical considerations; and which data collection tools to use.

The Toolkit component comprises a set of data collection tools designed to capture valuable information and feedback from the programme’s audience. Each tool is accompanied by a comprehensive description of how to conduct an evaluation with it. This description includes information on the required resources, such as equipment and space, guidance on implementation with the audience, suggested questions and prompts for participants, a summary of the tool’s advantages and disadvantages, and an overview of analysis procedures to use once the data has been collected.