How research libraries can contribute to reform the research evaluation system

The intersection between open science and research assessment is not the easiest to clarify. Because it concerns at once national research policies, international rankings, bibliometric tools, for profit organisations, research institutions and management, it can be difficult to get involved in this topic. On the other hand, the open science community is aware of the need to change the research assessment. As Bernard Rentier [1] used to say in his lectures: ‘we can talk for hours about open science but as long as we did not change the research assessment system, nothing will happen’.

Since the end of 2021 important signals regarding an evolution, or even a reform, of the research assessment system have appeared. For example the european fundraising appeal ‘Services and tools to underpin a research assessment system that incentivises open science practices’ [2] and the creation of a coalition commited to reforming the research assessment system [3]. Furthermore, at a more technical level, the limits of traditional tools (Wos & Scopus) have never been more apparent, and new tools, led by the open science community like Open Alex [4], are arriving.

This paper comes from a training on bibliometrics and open science given to PhD Students and librarians. At the end of the session, fellow participants wondered ‘what can we, as research librarians, do to help to reform the research assessment system ?’. The purpose of this paper is to begin to answer this critical question in three parts.

The first concerns bibliometrics, that is the statistics that are produced to account for the scientific production of a research institution. The goal is to be collectively aware of the limitations of traditional tools, which have been used for decades in bibliometrics. I will draw a criticism of Wos and Scopus mainly with the term exhaustivity and objectivity, and also with an approach based on cost and services. Finally I will describe some aspects of the new Open Alex tool related to research assessment and open science. The second part will be devoted to the DORA declaration [5], I will emphasize recommendations that can be disseminated by research libraries. To stay focused on practical aspects, the last part will give examples. I will explain the steps taken by two universities to facilitate open access practices. The first has released an “open access bonus” for the laboratories, and the later has made its institutional repository the sole source of its evaluation system.

[1] Rentier, B. (2019). Open science, the challenge of transparency. Académie royale de Belgique. http://hdl.handle.net/2268/233905

[2] Horizon Europe funding call (2022). Services and tools to underpin a research assessment system that incentivises open science practices. https://ec.europa.eu/info/funding-tenders/opportunities/portal/screen/opportunities/topic-details/horizon-infra-2022-eosc-01-01

[3] European Commission (2022). Process towards an agreement on reforming research assessment https://ec.europa.eu/info/news/process-towards-agreement-reforming-research-assessment-2022-jan-18_en

[4] Chawla, Dalmeet Singh (2022). "Massive open index of scholarly papers launches." Nature https://doi.org/10.1038/d41586-022-00138-y

[5] San Francisco Declaration on Research Assessment (2013), https://sfdora.org/