CoARA WG TIER – Report of the In-Person Meeting of 7 February 2025, Milan, Italy

Abstract

This document reports the key outcomes and highlights of the in-person meeting of the CoARA Working Group TIER (Towards an Inclusive Evaluation of Research), held on 7 February 2025 at the University of Milano-Bicocca.

The meeting brought together researchers, practitioners, and policymakers from across Europe to take stock of the Working Group's progress and to discuss the structural and systemic barriers that prevent gender-inclusive research evaluation. Three invited plenary speakers contributed perspectives grounded in major European projects — GENDERACTIONplus, MINDtheGEPs, and GRANteD — alongside presentations from the two TIER Task Forces and a report on the Working Group's large-scale survey on bias perception among CoARA members.

Key themes addressed:

The meeting opened with a critical examination of why gender bias in research evaluation remains so difficult to address. Six structural barriers were identified: the perceived objectivity of evaluation systems; cognitive dissonance among researchers whose careers have been validated by the current system; lack of awareness of systemic and unconscious bias; the entrenchment of masculine norms in academic culture; fear that inclusive practices lower scientific standards; and the taboo status of gender equality discussions in research environments.

The second plenary presented findings from the MINDtheGEPs project, highlighting how the dominant model of academic "excellence" — built around high mobility, continuous quantitative productivity, and funding acquisition — disproportionately disadvantages women researchers, who bear greater caregiving responsibilities and perform more unrewarded institutional service.

The third plenary focused on the definition of excellence itself, arguing that excellence is neither objective nor fixed, but context-specific, negotiable, and shaped by the criteria that evaluation systems choose to apply. Empirical evidence was presented documenting multiple assessment flaws, including differential language in recommendation letters and the Matthew effect in CV-based evaluations.

A presentation on the TIER Survey on Bias Perception — addressed to researchers and professionals across CoARA member organisations, from PhD students to full professors — outlined the survey's objectives: to investigate key biases in research evaluation, self-reported bias, organisational climate, and the visibility and effectiveness of existing mitigation strategies; and to collect evidence on how excellence is currently defined across institutions.

The two Task Forces then reported on their progress. Task Force 1 (Data Collection & Analysis) presented findings from case studies across Finland, Italy, Switzerland, the UK, and at EU level, and shared a list of approximately 30 best practices for bias-free evaluation for participant scoring. Task Force 2 (Capacity Building & Communication) reported on a catalogue of open-access tools and training resources to counteract bias in assessment contexts, and outlined plans for expanded outreach and stakeholder engagement.

The meeting concluded with small-group discussions on three themes: how to question the concept of excellence; best practices; and outreach strategies.