Investigating the Link Between Research Data and Impact - Phase II

The Institute for Methods Innovation – a research charity registered in the United States and United Kingdom – was commissioned by the Australian Research Data Commons (ARDC) to investigate how research data contributes to non-academic impacts by analysing existing case studies from the Australia Research Council (ARC) Engagement and Impact Assessment 2018. This represented a second phase to this work on the impacts of research data, with the first phase focusing on United Kingdom Research Excellence Framework (REF) impact case studies (ref.ac.uk).

Project overview
The research involved analysing impact cases from the ARC’s Engagement and Impact Assessment 2018. Only high scoring cases have currently been published by the ARC. These cases were sifted for the present research to focus our analysis on cases with an emphasis on ‘data’. Relevant text segments from the published engagement and impact (E&I) case studies were extracted from the E&I case study documents. A content analysis was conducted on these data to identify patterns linking research data and impact. This analysis achieved a high level of scientific quality, based on established methodological standards.

What type of impact was developed from Australian research data?
The most prevalent type of research data-driven impact was Practice impact (44%). This category of impact includes changing the ways professionals operate and improving the quality of products or services through better methods, technologies, and responses to issues through better understanding. It also includes changing organisational culture and improving workplace productivity or outcomes. Government impacts were the next most prevalent category identified in this research (20%). These impacts include the introduction of new policies and changes to existing policies, as well as reducing the cost to deliver government services, enhancing the effectiveness or efficiency of government services and operations, and more efficient to government planning. Other relatively common types of research data-driven impacts were Economic impact (14%) and Public Health impact (8%).

How was impact developed from research data?
Impact from research data was developed most frequently through Improved Institutional Processes / Methods (33%). This relates to improving the way an institution operates, making it more efficient or effective at delivering outcomes. The second most common way of developing impact was via a Report (25%) of some kind; that is, the presentation of information based on the analysis and interpretation of relevant data. Analytic Software or Methods (12%) comprised the third most frequently used way of developing impact. Here, research data are used to generate or refine analytic software or methods which, in turn, generated impacts. In general, research data itself rarely contributes directly to any impact. Instead, 99% of research data-linked impacts are indirectly associated with the identified impacts. Data need to be processed and conclusions or other value need to be drawn from them so that they can yield nonacademic impacts.

Who benefited from the research data-linked impact?
Government, Policy, or Policymakers (28%), Industry / Business (21%), and Specific Publics (16%) were the most common types of beneficiaries from the research data-linked impacts we analysed. This finding is indicative of a two-step flow of research data-linked Who benefits from research data-linked impact? impact that ultimately reaches publics or wider non-academic stakeholders. Intermediaries such as the government, policymakers, and businesses are typically the primary beneficiaries of research data-based impacts. However, they often in turn use what they have gained to develop further insights, services, products, and policies that deliver broader public benefits.
Looking at patterns in this analysis, the following statistically significant associations were identified:

• Reports or other types of static information tended to be used most with government stakeholders, and significantly less with industry / business.
• Analytic software or methods as well as shared technology or software were used more to develop impacts with industry / business, and significantly less with specific publics.
• Professionals as opposed to government stakeholders tended to benefit most from improved institutional processes.
• The general public tended to benefit from other impact instruments, e.g. gaining benefits via policy change.

Where did the research data in these impact case studies come from?
We found that research data used in our sample of EI 2018 case studies were sourced from a range of different categories of people and organisations. Specific Publics (21%), such as hospital patients, were the largest category. We found that Specific Publics also tended to be beneficiaries of impacts related to data sourced from Specific Publics. Research data in our sample also commonly originated from Industry / Business (17%), and the Natural Environment (17%) and Other Organisations (17%). Most of the impact-linked research data seemed to have been sourced by Research Performing Organisations (96%), such as universities. However, other contributing parties leading on primary sourcing and data collection may not have been included in the case study narratives.

Conclusions
Various connections were found between predictor variables and certain outcomes such as identified types of impact and beneficiaries, for example:

• The Field of Research, that is, which type of academic discipline the research data-linked impacts were attributed to- was found to be a moderate predictor of impacts. For instance, practice impacts were significantly more commonly associated with the field of Psychology and Cognitive Sciences than with other research fields, and specific publics as research impact beneficiaries were disproportionately associated with the research field of Aboriginal and Torres Strait Islander Research.
• On the other hand, whether the university associated with the impact-linked research data was part of the elite Group of Eight (Go8) overall did not or only weakly influenced impacts.

The analysis found that research data on their own rarely lead to impact, but instead they require analysis, curation, product development or other interventions to leverage broader non-academic value from the research data. These interventions help to bridge the gap between research data – which might otherwise go unused for the purpose of developing impact – and the diverse range of potential primary and secondary beneficiaries. As such, impact from research data may be increased through closer links between government, industry and researchers, as well as capacity building at each of these levels. Capacity building initiatives can be aimed at potential impact beneficiaries, including supporting them to access useful sources of research data, and either understand and make use of this data or adapt it to serve new purposes. As such, the way that research data are made available, and the nature of the support available for interpreting and using this data, can affect how feasible it is to use that research data to develop new and creative pathways to impact. Finally, there were strikingly high ‘uniqueness’ scores for the impacts linked to research data (93%), suggesting that most of the research data-linked impacts may have only been possible to develop through research data. However, limitations inherent in impact case studies have to be taken into account before drawing firm conclusions on this point. Finally, it is worth noting that the findings from the Australian EI 2018 case studies are broadly similar to the UK impact case study findings from Phase I of this work. This similarity suggests that there may be structurally parallel patterns internationally in how research data are used to develop non-academic impacts.

Should you have any questions about the report or dataset, please contact the Institute for Methods Innovation research team at research@methodsinnovation.org