Beyond just counting the beans: Reimagining research indicators to favour diversity

Talk given as part of the Research on Research Institute's Culture Shift series on 15 September 2022

Abstract

The increase in the availability of data relevant to research performance evaluation over the past ten years has been transformational. Despite this and the parallel increase in the power of computational tools the actual indicators used in practice remain stubbornly limited to citation counts and simple derivatives such as impact factors, h-indices and field-normalised counts. The lack of diversity in indicators, along with a lack of diversity in data sources, has aligned with a lack of diversity in the academy to strengthen and perpetuate a status quo where high prestige researchers at high prestige institutions gain greater resources, leading to more outputs in which they cite each other, increasing citation counts and propelling the whole cycle forward.

This talk will propose some simple, yet radical, shifts in how we think about research performance indicators. Using open data and transparent analysis it will imagine a world in which we stop asking how to count more beans, but instead how different they are.