Coalition for Advancing Research Assessment
Published October 9, 2025 | Version Final 1.0
Publication Open

A whitepaper on reforming research assessment for a digital and AI-driven science future

Creators

  • 1. ROR icon TOBB University of Economics and Technology
  • 2. EDMO icon Technical University of Munich
  • 3. CODATA IDPC International Data Policy Committee (IDPC), Committee on Data (CODATA) of the International Science Council (ISC)
  • 4. CoARA Ethics and Research Integrity Policy for Responsible Research Assessment in Data and Artificial Intelligence (ERIP)
  • 5. ROR icon Universidade Católica Portuguesa
  • 6. 5Jožef Stefan Institute (JSI)
  • 7. EDMO icon Technical University Dortmund
  • 8. OpenAIRE

Contributors

Description

This whitepaper proposes a strategic framework to advance Responsible Research Assessment (RRA) in Europe, responding to the need for a science system fit for the AI era. The initiative is rooted in the ethical principles developed by the CoARA Ethics and Research Integrity Policy for Responsible Research Assessment in Data and Artificial Intelligence (ERIP) Working Group. We introduce two co-designed tools, the Ethics Assessment Alignment Matrix (EAAM) and the Modular Assessment Configurator (MAC), to enable value-aligned, customizable evaluation workflows that integrate with open infrastructures like the OpenAIRE Graph. By fusing ethical principles with operational solutions, this initiative provides a practical pathway to embed open science directly into research and career evaluation.

Abstract

This whitepaper presents a strategic framework for advancing Responsible Research Assessment (RRA) across the European digital research ecosystem. Responding to the European Commission’s call for a science system fit for the AI era, we propose a common roadmap to transition from outdated, publication-centric evaluation towards inclusive, transparent, and context-sensitive practices.

Our initiative is built upon a foundation of core ethical principles, translated into actionable tools through a coordinated, systemic approach with three key pillars: (1) participatory design with Research and Innovation Actors (RIA), (2) flexibility and adaptability to diverse disciplines and missions, and (3) interoperability within the wider open science ecosystem.

We introduce two key innovations developed through extensive co-design: the Ethics Assessment Alignment Matrix (EAAM), which bridges normative values with assessable research domains, and the Modular Assessment Configurator (MAC), a digital tool that enables institutions and funders to create configurable, explicable, and value-aligned assessment workflows. These tools are designed to integrate with open infrastructures, notably the OpenAIRE Graph, which provides a provenance-enriched, FAIR-aligned evidence base for assessments.

By fusing ethical foresight with operational, machine-actionable solutions, this work provides a practical pathway to embed open science principles directly into research and career evaluation. It demonstrates how aligning policy, ethics, and technical infrastructure is essential for building a trustworthy, resilient, and innovative research assessment system for the future.

Files

CoARA-ERIP Whitepaper on reforming RA for a digital and AI-driven science future 25-10-09_Final.pdf

Files (1.2 MB)

Additional details

Identifiers

DOI
10.5281/zenodo.17308056

Related works

Cites
Publication: https://data.europa.eu/doi/10.2848/931335 (URL)

Funding

European Commission
CoARA Boost - Strengthening CoARA and Enabling Systemic Reform of Research Assessment - A Booster 101131826

Dates

Copyrighted
2025-10-09
CoARA-ERIP MAC-EAAM reforming research assessment for a digital and AI-driven science future
Available
2025-10-09
CoARA-ERIP MAC-EAAM reforming research assessment for a digital and AI-driven science future

References

  • 1. European Commission. Directorate General for Research and Innovation. (2021). Towards a reform of the research assessment system: Scoping report. Publications Office. https://data.europa.eu/doi/10.2777/707440. 2. European Commission. Directorate-General for Research. (2008). Developing world-class research infrastructures for the European Research Area (ERA): Report of the ERA Expert Group. Publications Office. https://data.europa.eu/doi/10.2777/9697 3. European Commission. Directorate-General for Research. (2010). A vision for strengthening world-class research infrastructures in the ERA: Report of the Expert Group on Research Infrastructures. Publications Office. https://data.europa.eu/doi/10.2777/79690. 4. European Commission. Directorate General for Research and Innovation. (2020). Supporting the transformative impact of research infrastructures on European research: Report of the High Level Expert Group to assess the progress of ESFRI and other world class research infrastructures towards implementation and long term sustainability. Publications Office. https://data.europa.eu/doi/10.2777/3423. 5. CoARA. 2022. Agreement on reforming research assessment. Retrieved from https://coara.org/wp-content/uploads/2022/09/2022_07_19_rra_agreement_final.pdf. 6. European Commission. Directorate General for Research and Innovation. (2022). European Research Area policy agenda: Overview of actions for the period 2022 2024. Publications Office. https://data.europa.eu/doi/10.2777/52110. 7. European Research Executive Agency. (2024). Report on research assessment. Publications Office. https://data.europa.eu/doi/10.2848/931335. 8. Barcelona Declaration on Open Research Information, Kramer, B., Neylon, C., & Waltman, L. (2024). Barcelona Declaration on Open Research Information. https://doi.org/10.5281/ZENODO.10958522. 9. Cagan, R. (2013). The San Francisco declaration on research assessment. In Disease models & mechanisms (Vol. 6, Issue 4, pp. 869–870). The Company of Biologists Limited. 10. Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. 11. Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., Da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), 160018. https://doi.org/10.1038/sdata.2016.18. 12. Parsons, M. et al. (2019). CARE principles for indigenous data governance. 13. Lin, D., Crabtree, J., Dillo, I., Downs, R. R., Edmunds, R., Giaretta, D., De Giusti, M., L'Hours, H., Hugo, W., Jenkyns, R., Khodiyar, V., Martone, M. E., Mokrane, M., Navale, V., Petters, J., Sierman, B., Sokolova, D. V., Stockhause, M., & Westbrook, J. (2020). The TRUST Principles for digital repositories. Scientific Data, 7(1), 144. https://doi.org/10.1038/s41597-020-0486-7 14. Manola, N., Tzouganatou, A., Kuchma, I., & CoARA WG on OI4RRA. (2025). Open Infrastructures for Responsible Research Assessment: Principles and Framework. https://doi.org/10.5281/ZENODO.14844582. 15. Manola, N., Vergoulis, T., Tzouganatou, A., CoARA WG on OI4RRA, Mannocci, A., & Pecoraro, F. (2025). Conceptual Architecture for the Implementation of a Responsible Research Assessment Framework Built on Open Infrastructures. https://doi.org/10.5281/ZENODO.15487415. 16. Manola, N., Tzouganatou, A., Vergoulis, T., & CoARA WG on OI4RRA. (2025). Policy Briefs: Transitioning from Closed Proprietary Systems to Open Infrastructures for Responsible Research Assessment, for Research-Performing & Research-Funding Organizations. https://doi.org/10.5281/ZENODO.15490704.