CoActD7.2: Interim Impact Assessment Report

DISCLAIMER:  The present Project Deliverable has been submitted to the European Commission for review. The information and views set out in this report are those of the author(s) and do not necessarily reflect the official opinion of the European Union. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein.   

The main aim of the evaluation and impact assessment in CoAct is to bring evidence of the impact that the project’s Citizen Social Science activities have on the involved actors, such as co-researchers, citizen scientists, knowledge coalition members, and professional researchers, as well as on their socio-cultural contexts. Additionally, the formative evaluation aims at the assessment of user-acceptance factors, such as ease of use and perceived usefulness, of the involvement activities, offered materials, developed prototypes, and the research process as a whole. This input will iteratively shape our interaction activities, the materials and the prototypes, trying to detect the non-conformances that may occur during the Citizen Social Science co-research process as well as drivers for engagement and usage.

In CoAct we follow a co-evaluation approach, which is a form of participatory evaluation that initiates the conversation on expectations, objectives, and impact already at the start of the project. The approach has been elaborated in the Deliverable D7.1 CoAct Impact Assessment Plan (Schäfer et al. 2020) and has raised international attention in the citizen science community. The strong interest in CoAct outreach activities with regards to our co-evaluation approach has been confirmed by participants in our webinars, workshops and the recently launched call for papers in the Special Issue “Participatory Evaluation and Impact Assessment in Citizen Science” of the fteval Journal for Research and Technology Policy Evaluation.  

After two years of project work, the co-evaluation efforts in the three main R&I Actions of the project, namely Citizen Social Science on mental healthcare in Barcelona, on youth employment in Vienna and on environmental justice in Buenos Aires, have already led to important insights. On the scientific level we see e.g. that the research questions are shaped by the social issues at stake. Scientific objectives are less visible in communication and motivation than the specific concerns, confirming a strong problem-driven approach. At the level of participating actors we can observe some fluidity in the roles for certain actors, such as co-researcher or members of the knowledge coalitions, who reveal diverse motivations for participating. In the case of mental healthcare in Barcelona we observe e.g. a growing feeling of ownership across co-researcher and very high expectations towards achieving the project objectives. At the socio-ecological and political impact level most evidence is still referring to expectations and not yet measurable impact. These are however very high across the three cases and reveal a demand for stronger participation and citizen advocacy. 

These first insights from co-evaluation are important to shape the remaining year of each case action, while also benefitting the gender action cases. It also contributes to the further development of co-evaluation strategies and learnings from a methodological point of view. On the other hand, we are also experiencing the limitations of co-evaluation as we had foreseen it at the beginning of the project, which will lead to important lessons learned to be documented at the end of the project.