Challenges in Assessing Contributions to Reproducible Research and Open Science

• DORA Panel: Addressing Roadblocks in Research Assessment Reform
• Panel organiser and moderator: Helen Sitar
• Speakers: Malvika Sharan (presentation in this Zenodo deposit) with Elena Simukovic, David Carr, Ulrich Drinagl, Anne Letho, Serhii Nazarovets

About the panel: Research evaluation at the institutional level frequently focuses on researchers’ academic outputs, primarily publications. Many evaluations fail to consider the myriad of unique contributions an individual researcher can make to the scientific community. If we want to transition towards an open and collaborative research culture, then universities, funders and institutions must incentivize behaviours and reward contributions which underlie the desired research ecosystem. For that reason, many leaders have called for an assessment of researchers’ contributions to open science and team science, data sharing, mentorship, teaching, leadership, academic service, invisible work or societal impacts. But changing standing assessment procedures is far from easy. Often when one proposes procedural changes at the institutional level, unforeseen challenges arise, for various reasons, and prevent positive changes from being implemented. At DORA, we seek to support people seeking to enact assessment reform in their communities. We know, however, that challenges are the rule, not the exception. Before we can surmount these roadblocks and reform the research evaluation system, we must first understand the common challenges: What barriers exist due to administrative difficulties, or financial limitations? In what cases is it difficult to gain buy-in from decision-makers? What is needed to facilitate the leap from procedural reform to implementation? In this session, we will address these questions, and others, through short talks and group discussion in a world-cafe format. Invited speakers will share accounts of challenges in pushing for research assessment reform. Small groups will discuss difficulties in rewarding and recognizing diverse contributions to the research enterprise in assessment procedures. We will learn from each other’s challenges, and DORA will develop tools and strategies to support change-makers in the future, based on the input from this session."