Journal article Open Access
The Institute for Methods Innovation – a research charity registered in the United States and United Kingdom – was commissioned by the Australian Research Data Commons (ARDC) to investigate how research data contributes to non-academic impacts by analysing existing case studies from the Australia Research Council (ARC) Engagement and Impact Assessment 2018. This represented a second phase to this work on the impacts of research data, with the first phase focusing on United Kingdom Research Excellence Framework (REF) impact case studies (ref.ac.uk).
The research involved analysing impact cases from the ARC’s Engagement and Impact Assessment 2018. Only high scoring cases have currently been published by the ARC. These cases were sifted for the present research to focus our analysis on cases with an emphasis on ‘data’. Relevant text segments from the published engagement and impact (E&I) case studies were extracted from the E&I case study documents. A content analysis was conducted on these data to identify patterns linking research data and impact. This analysis achieved a high level of scientific quality, based on established methodological standards.
What type of impact was developed from Australian research data?
The most prevalent type of research data-driven impact was Practice impact (44%). This category of impact includes changing the ways professionals operate and improving the quality of products or services through better methods, technologies, and responses to issues through better understanding. It also includes changing organisational culture and improving workplace productivity or outcomes. Government impacts were the next most prevalent category identified in this research (20%). These impacts include the introduction of new policies and changes to existing policies, as well as reducing the cost to deliver government services, enhancing the effectiveness or efficiency of government services and operations, and more efficient to government planning. Other relatively common types of research data-driven impacts were Economic impact (14%) and Public Health impact (8%).
How was impact developed from research data?
Impact from research data was developed most frequently through Improved Institutional Processes / Methods (33%). This relates to improving the way an institution operates, making it more efficient or effective at delivering outcomes. The second most common way of developing impact was via a Report (25%) of some kind; that is, the presentation of information based on the analysis and interpretation of relevant data. Analytic Software or Methods (12%) comprised the third most frequently used way of developing impact. Here, research data are used to generate or refine analytic software or methods which, in turn, generated impacts. In general, research data itself rarely contributes directly to any impact. Instead, 99% of research data-linked impacts are indirectly associated with the identified impacts. Data need to be processed and conclusions or other value need to be drawn from them so that they can yield nonacademic impacts.
Who benefited from the research data-linked impact?
Government, Policy, or Policymakers (28%), Industry / Business (21%), and Specific Publics (16%) were the most common types of beneficiaries from the research data-linked impacts we analysed. This finding is indicative of a two-step flow of research data-linked Who benefits from research data-linked impact? impact that ultimately reaches publics or wider non-academic stakeholders. Intermediaries such as the government, policymakers, and businesses are typically the primary beneficiaries of research data-based impacts. However, they often in turn use what they have gained to develop further insights, services, products, and policies that deliver broader public benefits.
Looking at patterns in this analysis, the following statistically significant associations were identified:
Where did the research data in these impact case studies come from?
We found that research data used in our sample of EI 2018 case studies were sourced from a range of different categories of people and organisations. Specific Publics (21%), such as hospital patients, were the largest category. We found that Specific Publics also tended to be beneficiaries of impacts related to data sourced from Specific Publics. Research data in our sample also commonly originated from Industry / Business (17%), and the Natural Environment (17%) and Other Organisations (17%). Most of the impact-linked research data seemed to have been sourced by Research Performing Organisations (96%), such as universities. However, other contributing parties leading on primary sourcing and data collection may not have been included in the case study narratives.
Various connections were found between predictor variables and certain outcomes such as identified types of impact and beneficiaries, for example:
The analysis found that research data on their own rarely lead to impact, but instead they require analysis, curation, product development or other interventions to leverage broader non-academic value from the research data. These interventions help to bridge the gap between research data – which might otherwise go unused for the purpose of developing impact – and the diverse range of potential primary and secondary beneficiaries. As such, impact from research data may be increased through closer links between government, industry and researchers, as well as capacity building at each of these levels. Capacity building initiatives can be aimed at potential impact beneficiaries, including supporting them to access useful sources of research data, and either understand and make use of this data or adapt it to serve new purposes. As such, the way that research data are made available, and the nature of the support available for interpreting and using this data, can affect how feasible it is to use that research data to develop new and creative pathways to impact. Finally, there were strikingly high ‘uniqueness’ scores for the impacts linked to research data (93%), suggesting that most of the research data-linked impacts may have only been possible to develop through research data. However, limitations inherent in impact case studies have to be taken into account before drawing firm conclusions on this point. Finally, it is worth noting that the findings from the Australian EI 2018 case studies are broadly similar to the UK impact case study findings from Phase I of this work. This similarity suggests that there may be structurally parallel patterns internationally in how research data are used to develop non-academic impacts.
1_ARDC_P2 Report - Investigating the Link Between Research Data and Impact - Phase II.pdf
2_ARDC_P2 - Analysis Data.csv
3_ARDC_P2 - List of Cases.csv
4_ARDC_P2 - List of Variables.csv
5_ARDC_P2 - ICR Data.csv