A New Approach to Impact Case Study Analytics

The 2014 Research Excellence Framework (REF) assessed the quality of university research in the UK. A fifth of the assessment was allocated according to peer review of the impact of research, reflecting the growing importance afforded to impact in UK government policy. The impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia. Each university submitted a set of four-page impact case studies. These are mainly free text that describes and evidences the impact of research. There have been several analyses of these case studies, but these have either used qualitative methods or basic forms of text searching and analysis. These approaches have limitations, especially in terms of the time needed to analyse the data manually, and due to the often poor quality of the answers generated by applying computational analysis to free text data that lacks structure and context. This paper describes a new system we have built that that takes an alternative approach to overcome these problems. At its core is a structured, queryable representation of the Impact Case Study data. We describe the design of the ontology used to structure the information and how semantic web technologies are used to store and query the data. We show that this gives two main advantages compared to existing techniques: improved accuracy in question answering, and the ability to answer a broader range of questions, including by integrating data from external sources.