Altmetrics for German medical research: what leads to research articles achieving policy impact?

One fundamental drawback considering the use of policy citations as altmetric indicators is that coverage of medical research within policy documents overall seems low, with only 2-3% of articles from our sample receiving any policy citation at all. Thus, policy citations will most likely only be an informative indicator in settings in which analyses can be based upon large samples of available research publications. Another challenge regarding the use of policy citations – both within studies like this one and for ‘real world’ assessment exercises – results from the diversity of sources that data aggregators like Altmetric.com or Overton fetch policy documents from. For instance, the concept of ‘policy documents’ as tracked by Altmetric.com covers a heterogeneous multitude of sources and document types, ranging from government guidelines over publications from independent think tanks to reports from international development organizations. While the simple API-based approach taken in this study does not differentiate between such different kinds of policy sources, in practical situations differentiation will often be desirable. The lack of best practices and field-tested frameworks for a more differentiated handling of different kinds of policy citations is an aspect that future scientometric research should address. In our own future work, we will expand on characterizing the biomedical literature cited in policy documents regarding further properties, e.g, involved authors, institutions, or specific research fields.