Accompanying process evaluation of FWF's Emerging Fields

Launched in 2022 as part of the Excellent=Austria initiative, the first call of the Emerging Fields (EF) Programme attracted 45 applications from a range of multidisciplinary Austrian research teams. Five of these were granted a total of €31m via a three-stage selection process. This report presents the findings of the evaluation of that selection process.

Our headline conclusion is that the Emerging Fields (EF) programme has served a critical signalling and galvanising function. By launching the programme, the FWF provided an opportunity for Austria-based researchers to propose a wide range of highly novel research ideas. Many of these were fully formulated only in response to the EF call.

In the wider context of the Excellent=Austria initiative, the EF call has thereby fulfilled an important function by adding funding for basic research and by providing a vehicle for innovative ideas to take shape.

The three-stage EF assessment process itself was well designed and works well. In line with the original design-intentions, Stage 1 had a strong focus on innovative aspects of the research ideas. By focussing on short synopses, this stage also ensured that the burden of extensive peer reviewing of full applications was minimised (though applicants still had the burden of writing the full applications).

Stage 2 was a full formal external expert peer review focused primarily on scientific quality, while the Stage 3 Jury hearings allowed a broad range of questions to be asked in relation to many criteria but also provided insight into team composition, dynamics and hierarchy. The designated decision-making body, the FWF Scientific Board, played only minimal roles at Stages 1 and 3 but a major role at Stage 2.

We find a small number of challenges. First, applicants were dissatisfied with the feedback provided and what they saw as lack of process transparency, especially at Stage 1.

Second, while the initial pool of applications and applicants was diverse, outcomes in terms of gender and discipline are less so, with no humanities or arts applications being successful and no female coordinators among the five winning awards. While we find no overt evidence of bias or discrimination, the FWF needs to continue to monitor these figures in future calls.

Finally, there was much need throughout the process to define several key terms and how to treat them (novel, interdisciplinary, field, what it means to be ‘emergent’, etc). This extended to applicants and institutions being able to contextualise the scheme and what exactly was expected, extending to the difference or similarity to the FWF’s SFBs.

In the final section of this report, we set out our full list of recommendations. These include creating synergy between the EF scheme and the SFBs, as well as better definition and clarification of key terms (‘novel’, ‘interdisciplinary’, etc) and what role they play in the assessment. We make various additional recommendations for minor process adjustments.

A statement by the Austrian Science Fund (FWF) on this evaluation can be found at:  https://doi.org/10.5281/zenodo.13918295