What is the h-index and what are its limitations? Or: Stop using the h-index [Animation]

 

Educational content and script: Stefanie Haustein, Maddie Hare, Heather Woods, Isabelle Dorsch, Carey Ming-Li Chen

Storyboard, drawing and animation: Julie-Ann Déry

Production and editing: Julie-Ann Déry, Marie-Josée Archambault

Narration: Sarah Komendat

Audio and sound effects: Marie-Josée Archambault

 

This video is licensed under a Creative Commons CC-BY license: https://creativecommons.org/licenses/by/4.0/

 

The video was created with funding from an Insight Grant (435-2021-0108, PI Haustein) from the Canadian Social Sciences and Humanities Research Council (SSHRC).

 

Excerpts from the script:

The academic world is competitive. Researchers compete for jobs, funding, prizes, and awards to advance their careers. The h-index was created by physicist Jorge Hirsch who wanted to create a measure that combined research productivity and impact: “A scientist has index h if h of his or her Np papers have at least h citations each and the other papers have less than or equal to h citations each.” Mathematically, the h-index is simple to compute by ranking publications by the number of citations they received. You can look up an author’s h-index in many citation databases such as Google Scholar, Scopus, or Web of Science. Since the h-index depends on the number of publications and citations, senior researchers who have been active for longer periods of time have the advantage of being able to accumulate more publications and citations over time. Therefore, they usually have higher h-indexes than their junior colleagues. For this reason, the h-index should never be used to assess early career researchers or to compare authors who are at different stages in their careers. Different fields have different publication practices and authorship protocols. Since the h-index does not account for these field-specific differences, it should never be used to compare researchers from different disciplines. While fields such as Engineering often include several, sometimes dozens, of authors on a publication, authors in English literature tend to work alone. The length of reference lists (and thus the average number of citations per article) also varies between disciplines. Additionally, the extent to which researchers publish in journals rather than books or conference proceedings differs from one discipline to another. All these publication practices affect the average number of publications per author and citations per paper. This causes h-indexes to vary significantly between fields, so authors from different fields should not be compared using the h-index. Each citation database has different criteria regarding which sources to index (such as document types, language, and geographical focus). They may therefore rely on different sets of publications and citations to calculate the h-index. This means that the h-index of the same author may vary across databases. It is therefore important to indicate when and from what database the h-index was obtained. The h-index is a metric that arbitrarily combines the number of publications and citations. Despite its many limitations the h-index is heavily used to assess researchers. If used at all, the h-index should only complement qualitative research assessment. Ultimately, one quantitative metric will never be able to fully capture a researcher’s work.