The Sustainable and TrAnsparent Research data (STAR) study: Final report

This report covers findings from the UKRN-led STAR (Sustainable and Transparent Research Data) project. Project collaborators included 21 diverse UK Higher Education Institutions (UKHEI), geographically spread across English regions and UK devolved nations, varied in their relative funding wealth; and in the nature, focus, and/or breadth of the subjects and disciplines in which their students, teachers, researchers and/or practitioners specialised. Participants included executive and professional services staff with leadership and delivery responsibilities for institutional open research data (ORD) policy and implementation. STAR project collaborators took part in interviews, focus groups, site visits and workshops, that set out to gauge progress and barriers to ORD implementation, since publication of the 2016 Concordat on ORD. Outcomes and co-designed recommendations to support future UKHEI ORD implementation, are reported here.    

The STAR project found that since the publication of the Concordat on ORD in 2016, UKHEIs had made significant progress. Participating institutions were actively demonstrating their commitment to a future in which publicly accessible ORD is a routine and normalised output of publicly funded research. ORD specialists had been recruited into open data posts and were champions of institutional ORD implementation, often the authors of institutional ORD policy and guidance. ORD policy and guidance was mostly available to academic communities in visible and accessible formats. Dedicated institutional ORD repositories were in place and ORD workers provided expert assistance to support effective data preparation and appropriate use of ORD repositories. ORD support included detailed data scrutiny, curation and training in the standards and skills required to promote FAIR (Findable, Accessible, Interoperable, Reusable) standards of open data publication. Training in ORD processes and standards was regularly offered to postgraduates and also to established research and teaching staff and postgraduate supervisors. There were many and varied examples of research data that had been successfully openly published and stored within institutional repositories.

Despite these strong indicators of increasingly established ORD policy and infrastructure, it was evident that making research data open remained a relatively small-scale and exceptional, rather than routine and ubiquitous, activity among institutional research communities. In general, it appeared that a majority of research communities remained unfamiliar with repository technologies, open data curation standards and practices, ORD publication and reuse. Apart from those mandated as part of postgraduate courses, academic staff did not readily take up the ORD training that was increasingly on offer to them. It appeared that for the most part, when researchers sought to openly publish data, this was to satisfy stringent journal publication requirements, rather than institutional ORD policy or guidance, which was readily ignored.

There were multiple, simultaneously operating barriers to uptake of ORD activities. Inclusivity was a recurrent issue. Academic communities with divergent disciplinary practices, outputs, and epistemic values, did not always identify their work as ‘data’, or even necessarily as ‘research’. Similarly, the presumed value of ORD as a public good, makes sense in the context of tangible anticipated positive impacts (e.g. generative plans for open data reuse). For work with less obvious reuse potential, any inherent justification for spending time, effort and costs on ORD activities, remained weak.

Even among research communities able to share in a unified ORD vision, or who recognise an inherent value in making data open, because of obvious reuse potential, many were not yet fully prepared to take on ORD work within their funded projects. They lacked data curation skills and were unlikely to have allocated sufficient time or costs to ORD activities. Researchers and the finance teams who approved their project costs and budgets, did not yet routinely or effectively recognise costs associated with ORD practices within funding applications. ORD workers had also not yet established systems to efficiently monitor a plurality of ORD databases, to capture and evaluate their research communities’ ORD outputs and impacts. As a result, ORD work lacked the visibility of regular reporting, and neither the threat of compliance checks nor the promise of recognition and reward served to incentivise ORD practices. In this context ORD work remained irregular, mostly invisible labour. Given the additional impact of funder, institutional, departmental and project financial pressures, ORD was unlikely to be routinely prioritised.

To compensate, ORD workers, typically enthusiastic proponents of ORD, described spending considerable time on the ‘translation work’ of inclusion, attempting to expand the concept of ‘open research data’, to accommodate the work and interests of their varied academic communities. ORD workers also described providing detailed data curation assistance to promote effective ORD publication among inexperienced academic staff. As a result, ORD workers and teams, typically a relatively small and finite presence within their institutions, were often working at capacity, despite serving relatively small proportions of their overall academic communities. It was widely recognised that it would not be possible to scale up ORD activities, to include entire institutional research communities, on the basis of the current intensity of dedicated specialist ORD assistance. It was clear that open research data workers and teams needed their academic colleagues of all disciplines to get more solidly and autonomously on board with the vision and values of ORD implementation and the skilled work of data stewardship. In this vein, they offered regular training and published accessible open research data curation and repository-use guidance.

However, the push for academic communities to take on this work, appeared muted. Despite notable progress, there was a sense of stasis and cautious anticipation. In the context of strongly perceived financial, workload and staff capacity pressures, most felt an external pressure (e.g. funder compliance checks, or more stringent ORD REF requirements) would be needed, for institutions to risk diverting additional resources to scale-up ORD implementation. As such, open research data implementation appeared to have hit a glass ceiling. Institutions and dedicated ORD staff continued to work hard, but were already over-stretched in their efforts. The journey towards ubiquitous open research data publication appeared to have reached a point where open research data publication was encouraged, was possible, and where standards of excellence were promoted, generating prominent examples of best practice. There was little sign of institutional or research community push, beyond this early emergent phase, to make open research data activities routine and normalised practice.

Co-developed recommendations began with the need to increase the inclusiveness of the language and examples of ORD, to better connect with disciplinarily diverse researcher communities to the work of ORD, including implementation decision making processes. Core to all further recommendations, was the need to reconnect ORD values to ORD work, outputs and anticipated impacts, as a balanced moral and pragmatic concern for each research project. To support this, we recommend the establishment of formalised institutional or departmental processes of ORD review as an addition to current ethical review processes or as part of data management planning or, ideally, as an integrated process. Formal ORD review processes would allow researchers to strengthen and better formulate their ORD outputs, by setting out and justifying the time, skilled labour and costs involved in generating ORD outputs, and any additional work required to connect proposed ORD outputs to relevant audiences beyond their proximate research community and to increase opportunities for anticipated scrutiny and impact. This process of review would allow projects to better specify, justify and account for ORD costs in their project budgets, removing the invisibility of ORD labour and better connecting research communities to the inherent value of ORD outputs in relation to their own work. This would also allow projects where justification for ORD outputs remained weak, and other research priorities and impacts were better supported, to avoid ORD work, costs and outputs, through formal exemption, allowing research time, costs and efforts to be focused more appropriately on other areas. This would move us beyond the current, generalised, future-oriented approach to ORD, that treats all research as of equal ORD value, and requires all researchers to produce ORD, not because this is justified in relation to their particular program of work, but as a contribution to a future in which ORD is a routine, normalised output. This future framing may be necessary at the early emergent phase of ORD implementation, but is seen to in the long run to alienate researchers, ignoring moral and pragmatic concerns, that will continue to lead researchers, who face multiple pressures on limited project resources to continue to avoid ORD activities, but at increasing risk of reputational damage (both their own and their institutions). Better formulating institutional processes of ORD review that connect the value of ORD outputs to the relative costs, and support formal exemptions where appropriate, would allow researchers to consider these values more carefully, and to account for the work of ORD more effectively, connecting proposed time, work and costs to specific outputs and impacts. This would be less alienating and would also support engagement from currently disengaged research communities, for whom the relevance of engagement with formal review processes, would be clear, especially if these review processes were linked to compliance, monitoring and evaluation processes, and would ensure that those with formal justification for exemption from ORD outputs would not be penalised for failure to generate ORD.

The ability to cost, describe and assess impacts, that would inform formal ORD review, provides a backbone for the implementation of compliance and evaluation processes, while protecting those currently in need of exemptions, but creating a framework for their future engagement, given changing costs, skills, and reuse possibilities.

Other recommendations are proposed that address the perceived disconnects between ORD requirements, ORD specialist work and many research communities in terms of values, as well as pragmatic and technical concerns. Implementation of these recommendations would complement the adoption of the institutional review processes noted above.